Having looked around at many web hosting companies and cloud VPS providers, I eventually settled on Digital Ocean. Besides being the cheapest by far, they also have no hidden charges (or at least none that I’ve come across), their website leaves you at most two-clicks away from any settings, and their support team are quite fast to respond, although not as fast as the friendly and very helpful community on the forums.
Having your own virtual private server means you can run whatever you want on your site: WordPress plus all the bells and whistles for no extra cost, with no restrictions on file types; a DNS server; an email server; a git repository – all for less than $0.01 per hour. On the very remote chance that their service completely dies, losing all your data, you can have the important files on your server replicate to Skydrive, Dropbox, or to another VPS on another continent with a little extra effort. This beats my last provider, heart internet, who wanted to add extra charges for pretty much everything.
A nice feature for analysis and data-mining: their virtual servers are billed per-hour. This means that I can fire up a new 8-core/16GB virtual server in Amsterdam, leave a job running on it, collect the result a few hours later and destroy the sever – for the grand cost of one US dollar. Nice.