Photography: post-processing – and some amazing free, open-source software workflows

Introduction

This article describes some free, open-source photo/video processing software that I use since I moved from Windows to Linux.  Most of this software also runs on Windows, and it is also free.  As much as I like Adobe CS5 and Sony Vegas, I would struggle to move back to them (excluding Illustrator/InDesign) now that I’ve got used to the open-source offerings.

If you are comfortable with building programs from the source (or use Arch Linux, which makes this really really easy) then do so and you could get significant performance improvements on modern processors.  Unlike the somewhat expensive offerings from Adobe, these open-source programs don’t install nasty viruses on your PC*.

* Some Adobe products (when legally purchased) come with a rootkit virus called “FlexNet” which can cause data-loss and corrupt boot-loaders (making systems unbootable).  Amusingly, the pirate copies of Adobe’s products generally have this virus removed from them.  That’s how I wasted over £100 by buying Adobe software, only to end up using a pirate copy…

Photo processing on Linux

Before I left the UK, I ran on Arch Linux.  When travelling, I boot public PCs with Debian from my phone via DriveDroid.  This means that Photoshop and Lightroom are not available for me to use in my photography workflow, as they are amongst the few programs that can’t be run in Linux via Wine.

This isn’t much of a problem though, as the open-source offerings for photography are generally very powerful and well maintained.  They have a steeper learning curve than Adobe’s products, but allow you to do more in less time when you get used to them.

When mentioning open-source image-editing software, people usually think of “The GIMP“, which is probably the ugliest piece of software I have ever seen (even if I include commercial scientific software).  Its numerous text-rendering issues (on all operating systems) make it generally a pain to work with.  Running an old version of PaintShop Pro or PhotoDeluxe under Wine is preferable to using The GIMP, despite PSP being incredibly buggy and PD being ancient.  Thankfully, there are many alternative free/open-source image processing programs available which are powerful, consistent and usable.

On the camera

On the camera, I would love to use some custom firmware similar to Magic Lantern, which is available for Canon DSLRs.  I suppose I can wait for Vitaliy Kiselev’s hack to expand and mature.  For lack of this, I just shoot RAW in the usual ways, using the standard delay/repeat/timelapse/remote/long-exposure features.

Cataloging and RAW processing

After taking the RAW files off my camera, I first catalogue and process them with RawTherapee.  I hear that there is a stable Windows port of this program available too.  RawTherapee has the best noise-reduction that I have come across yet, has brilliant colour correction, and makes batch processing effortless (which should be a minimum requirement for RAW processing software).  The only downside is that it can not read the white-balance setting from Nikon’s RAW files, but this is because Nikon encrypt that data it in an attempt to force their slow, unstable, bug-ridden ViewNX/CaptureNX upon people.

If I just want to process a load of files with the same processing settings, then I can use RawTherapee’s batch processing although for simple jobs I often use ImageMagick instead.

Video processing

For video processing I’ll use Kdenlive (which has completely replaced Sony Vegas for me).  As for time-lapses, I’ll usually stick to ffmpeg or slowmoVideo, the latter being pretty powerful – it supports non-linear time flow, optical flow interpolation, and optical-flow driven motion-blur.

HDR

HDR?  What HDR?  I probably should start learning this style, as it does produce some really nice results when people take the time to do it properly (i.e. go beyond just “pressing the HDR button”).

Post

As a general rule, I only use effects which operate “globally”, i.e. to the whole image.  If an effect requires me to click on the actual image, then I won’t use it (excluding crop).  This is just a personal rule to prevent me from spending hours on a single image, developing amazing art like some others do.  I occasionally break this rule to clone out annoying people, parked cars, or lens flares but this is quite rare.

Summary

There’s lots of cool free/open-source image processing software out there.  Get rid of any bad impressions that you might have from The GIMP and branch out into the purpose-built software.  The learning curves are steep, but the rewards are greater.  A list of software mentioned is given below:

  • The GIMP – included for completeness, but DO NOT USE this unless you hate yourself.
  • Magic Lantern – Like CHDK, but for Canon DSLRs.
  • Vitaliy Kiselev’s hack – Like CHDK but for Nikon DSLRs.  I haven’t tried this yet.
  • RawTherapee – RAW processing and batch-processing software with the best noise-reduction I’ve come across yet.
  • ImageMagick – command-line image processing, useful for automated tasks and batch processing, e.g. resize to fixed width and watermark all images.
  • Kdenlive – Powerful non-linear video editor with great shake-reduction.
  • ffmpeg – command-line video processing, Debian users will probably have the libav fork instead.
  • slowmoVideo – very good timelapse/slowmotion software supporting non-linear time flow, optical flow interpolation and nice motion blur.
  • DriveDroid – not free/open-source but useful when travelling: it allows me to boot Arch Linux onto any hostel PC from my Android phone, and have a personal desktop complete with all my software.

All of these programs also work on Windows 2003, excluding the camera firmwares and also DriveDroid which is a mobile Android app.

Reverse-tunnel SSH through NAT: Remote-access a computer that is behind a NAT router/firewall without manually forwarding ports on the router

Using port-forwarding over an SSH connection allows one to set up a SSH reverse-proxy server, allowing SSH access to a machine that cannot receive incoming connections (e.g. behind NAT firewall/router) by proxying them through a machine which can (e.g. a cheap VPS)

Problem: Dynamic IP, NAT and a bad router

While travelling around Europe (and now, living in Estonia), I occasionally want to connect to my home PC in order to backup photos to it from my camera, or to copy music from it to my phone. Unfortunately my PC is at the family house (now that I don’t have a UK house) and they have a crappy ISP router which “forgets” its settings every time the ISP pushes some new pointless stealth update to it.  The dynamic IP problems arising from the cheap ISP are easily resolved via my Dynamic DNS package, however that isn’t even necessary for the solution in this article.

Reverse port forwarding

This means that my PC in England essentially cannot receive connections from the outside world. However it can make connections. This is where “port backwarding” or rather “reverse port forwarding” comes in.  In typical port forwarding over tunnels, you connect to some other device and messages sent to some port on your end are forwarded down the tunnel to the other end.  In a reverse port forward, you connect to some other device and instruct it to forward packets back down the tunnel to you.

When my PC is powered on, a service loads which creates a tunnel to one of my cloud servers. On the remote end of this tunnel, a port on the server is forwarded back down the tunnel to my home PC. Therefore, connections made to that port on the server are forwarded down the tunnel to my home PC. Hence, my home PC can receive connections from the outside world, as long as they are passed through the tunnel that it created to the outside world. This is similar to a VPN, but much simpler and also quick and easy to configure.  In order to prevent me from having to enter a password for SSH login every time the service starts, I use public key authentication*.  If security is not an issue, you can also create port forwarding (reverse and forward) via netcat and pipes.

* I actually use this between ALL my secured PCs and servers, and disable password login.  This vastly reduces the risk of Chinese botnets getting into my networks.  The server hosting this blog (and others) gets on average three failed login attempts per second from China, plus some from other parts of the world too.  An IPTables rule-set restricting connection initiation rates on certain ports also hardens security against these bots somewhat.

Remote wake-on-lan without VPN or DDNS

First, to power my PC on remotely, I issue a command to one of my cheap cloud servers. A service running on a Raspberry Pi in the England house creates a connection to my cloud server.  The cloud server uses this connection to notify the Pi when a “power on” command is received by the server.  The Pi uses Wake-on-lan to power my PC on – much cheaper and simpler than the 1980’s power-relay method.  My PC powers on and the “port backwarding” service starts.  This method does not require a VPN or my Dynamic DNS service.  Nor is my old remote wake-on-lan interface needed any more.

The code

The service on my home PC runs the following:
[code language=”bash”]ssh -nNTR remoteIntf:remotePort:localHost:localPort remoteUser@remoteHost[/code]
This creates an SSH tunnel between localHost and remoteHost, and forwards any packets bound for remoteIntf:remotePort on remoteHost to localHost:localPort. Using this, we can forward ports from the remote host to any host/port on our local network. In this case, the machine creating the tunnel is also the target for the port forwarding, so we have:
[code language=”bash”]ssh -nNTR localhost:9000:localhost:22 server-user@cloud-server[/code]
which listens for packets via local loopback, port 9000 on the remote host, and forwards them over the tunnel to local loopback, port 22 (SSH port) on the home PC.

I could of course use this on the Pi instead, to tunnel from Pi to server, forward from server throuh Pi to PC:
[code language=”bash”]ssh -nNTR localhost:9000:home-pc:22 server-user@cloud-server[/code]
But I see no advantage in forwarding through the Pi, and it will definitely slow things down a little. Also if I decide to reboot the Pi then I would lose my connection to the home PC via this method.

Using the first method (service on home PC initiates reverse forwarding), to create an SSH connection to my home PC, I simply log onto my cloud server via SSH/Putty and issue:
[code language=”bash”]ssh home-user@localhost -p 9000[/code]
which creates an SSH connection through local loopback to port 9000 – which is forwarded over the previously-created tunnel to my home PC’s SSH port (22).

If I bind the tunnel to the public IP of my server, then I can connect to my home PC from my SSH client here, rather than logging into my server first.

This has security risks, so to enable it you must first set the following in /etc/ssh/sshd_config:
[code language=”bash”]GatewayPorts=yes[/code]
Service on target PC:
[code language=”bash”]ssh -nNTR cloud-server:9000:localhost:22 cloud-user@cloud-server[/code]
Connection from other PC/phone/tablet:
[code language=”bash”]ssh home-user@cloud-server -p 9000[/code]
Obviously, change the user-names and server names/ports to your own preferred values, I use home-user/cloud-user/cloud-server/9000 as examples here.

Helix image switcher

I saw a rotating blinds -style advert outside a fuel station and figured that it would be a flashy way to show a set of images on a web page. The demo is available here, and the source (including a generator shell script) is available in a github repository. This animation uses pure HTML/CSS3 – forget outdated stuff like Flash, there isn’t even any JavaScript in use for this demo!

I may use some variation of this for the images in the gallery viewers over at Kuckian Arts, once more importants tasks for that site have been completed…

You can also trigger the animation via on-mouse-over / on-mouse-leave by using the :hover pseudoclass, the animation direction specifiers, and a single iteration of the animation rather than “infinite” as it currently does. Thanks to CSS3, making those changes is incredibly easy for anyone who wants to.