Twitter is making it decreasingly attractive for 3rd party developers to bother creating applications for their platform:
[I]f you are building a Twitter client application that is accessing the home timeline, account settings or direct messages API endpoints (typically used by traditional client applications) or are using our User Streams product, you will need our permission if your application will require more than 100,000 individual user tokens.
In effect, this limits the total number of users of a 3rd party application to a maximum of 100,000. If selling 100,000 copies of your application isn't profitable, you might be interested in backing app.net.
I am often asked the question, "How do I figure out how to diagnose this programming problem?" Most often I point to experience and practice as the best ways to know what to do first. Rob Pike gives some of his advice from his time working with a colleague:
When something went wrong, I'd reflexively start to dig in to the problem, examining stack traces, sticking in print statements, invoking a debugger, and so on. But Ken would just stand and think, ignoring me and the code we'd just written. After a while I noticed a pattern: Ken would often understand the problem before I would, and would suddenly announce, "I know what's wrong." He was usually correct.
I've been hosting sites for myself and for clients since 1995, and over those years have use many different web hosts. I've had good experiences and bad, and am always surprised when clients come to me having already paid for their hosting but haven't really thought to shop around or considered their needs. Hopefully, this page will serve as a guide for people that are looking for an appropriate host for their sites at any stage of their site's lifetime.
Assess Your Needs
Before you can shop for a host, you need to consider what the hosting needs of your site might be. Depending on the size, expected traffic, and technology needs of your site, you will need different hosting.
Estimating Size and Traffic
It may be difficult to estimate size and traffic on a site that doesn't already exist, but you can get a rough estimation. Don't forget that your CMS will consume some space (for example, WordPress can occupy about 20MB after custom themes are installed), and that the data in your database may count toward your hosting allotment.
If you publish primarily a text site, you will take up considerably less space than a site that routinely publishes photos and audio. A video blog can consume not only a significant amount of storage resources, but if the video is meant to be served through a streaming server, it can consume processor resources as well.
Traffic is probably the most difficult thing to estimate for a new site. Sites that have been running for a while have the benefit of analytics to gauge their traffic. Unless you know you have a built-in audience for your site, you can probably assume that your traffic needs are fairly low. You should still take into account the amount of data that your visitors will transfer over their visit, because depending on the host you select, this can factor into your costs.
Technology Needs - Content Management Systems
There are many different Content Management Systems (CMS). Different CMSes have different server requirements that must be met to have a site that performs well. If all you are publishing is a set of static HTML pages and photos, then you likely do not need much more than basic web hosting. If you're running an instance of Drupal, you may need more horsepower in your to generate the pages for serving.
Consider the minimum requirements of your CMS. For example, if you use WordPress, your host should supply at least the last supported version of PHP, and a current version of MySQL as the database. Your CMS of choice may require a different set of baselines, including specific versions of Python (or another server-side language) or MongoDB (or another database server).
Types of Hosting
There are several types of hosting, each with pros and cons. The type of hosting you choose will depend on the needs of your site and your budget.
The types listed here are not the only types, but if you need hosting other than these, you're very likely at a stage where you'd be better off hiring a contractor to help you decide.
Hosted Services offer the content management component as the sole offering of the hosted service.
- Very cheap for starting sites
- Simple to start up and use
- Typically, reasonable support options are available
- May not offer flexibility for custom themes
- Will not provide capabilities for services outside of what they offer
- Some features (like custom domain names) may have a premium
If you want a no-frills package, a hosted service is a low-headache option that works well, at the cost of loss of customization.
There are many hosts that offer hosting for no cost.
- It's free
- May be restricted to a certain subset of features that cripple most CMSes
- There is typically a very low site size limit
- The servers are often overloaded with free "customers"
- The servers are typically under-powered
- Support is nearly impossible to get for free
- Some hosts delete inactive sites
- Some hosts force ads onto your pages
- Practically everything else
One of the major take-aways about all types of hosting is that you almost always get what you pay for. In the case of free hosting, you're practically getting exactly what you pay for. If you choose free hosting, prepare to either abandon your site or move it to a capable host very soon after launch.
Shared hosting is characterized by a service that offers its customers space on a single server that is shared with other customers.
- Typically, shared hosting is the most affordable hosting
- Support is often available by email, message board, or phone
- Often has a very simple-to-use control panel
- Shared resources with others means less resources for your site
- Shared resources opens potential security issues
Shared hosting is probably the most popular self-hosting option because of its low price and capable features. Still, remember that you are sharing this server with other customers, which means that when they get more traffic, your site can slow down. Also, you're only as secure as the least secured customer on a shared host. If there's someone running insecure scripts in their site, there's the possibility it can affect yours. It probably will.
Overselling on shared hosting is an important concept to understand. There are many hosting services that offer unlimited storage, bandwidth, and CPU for prices as low as $7 per month. How are they able to offer this rate to everyone? They bank on you not using everything they offer.
If each site only uses 50MB of storage on a server, it's possible to fit hundreds of customers onto a single server, and still offer them unlimited or high capacity. Some hosts, if you start to use more space than other customers, will move your site to a server that has more capacity. This can be an inconvenience.
In a worst-case scenario, a shared host can claim that by using more space or CPU you are violating their Terms of Service by affecting the performance of other customers' sites on that server. Sometimes, they may send a warning email. Other times, they may cancel your account without notice. The lesson here is to be sure to read your host's terms of service before entering an agreement to allow them to host your site.
Platform As A Service hosting offers a host for a specific technology platform, but are often unique in the way that you deploy to them. Examples of a technology platform include Ruby on Rails, or WordPress.
- The platform is maintained by the host
- The platform is usually tuned for performance
- Allows customization that shared hosts don't
- Can be costly
- Not usually configured for multiple applications
- Sometimes deployment requires knowledge of SCM
Many PaaS offerings require you to know git to deploy code to their servers, which isn't hard but can be daunting if you've only ever used FTP to transfer sites. Because the platform is managed, it's usually costlier than shared hosting, but it's also usually not as costly as managed VPS or dedicated hosting, and can be a great option if your site uses only the technology that the platform supports.
A Virtual Private Server is software that simulates a whole physical server running inside of a larger host server. The large host server can hold many of these virtual servers, which each have been allocated a specific amount of resources of that host server. The virtual servers do not interact with one another.
- A full server environment all for your site
- Can host multiple sites
- Can usually be quickly changed in size to allow for growth
- More expensive
- Some server maintenance knowledge is usually needed
One of the best abilities of VPS servers is that they can easily scale. If you find that your server is not powerful enough to handle the current load, you can request that your host "resize" your server to provide it more capability. Be wary, though; many of these hosts charge for metered CPU usage, storage, and transfer.
The obvious downside to a VPS server is that some knowledge of how to run a server is often required, which is typically not the case for shared hosting. There are hosts available that will provide you with "managed hosting", in which they manage your server for you. These options are typically much more expensive, but a poorly maintained server is a ripe candidate for security problems, which at best can lead to unwanted downtime for your site, and at worst can be used infect other more-secure machines more easily. Security is everyone's responsibility!
Dedicated hosting is the simplest to explain -- It is a single physical box connected to the internet that you rent to host your site.
- Possibly the most powerful hosting option
- The whole server is yours to do with as you please
- Very costly
- Server maintenance knowledge is usually needed
- Single point of failure
Dedicated hosting can be very useful for sites that have specific back-end requirements. Unless your blog is doing millions of page views per day, this is probably too much power and maintenance headache for a blog.
Just like the VPS option, there are services that offer managed hosting on dedicated servers, and these have commensurable cost. Unlike VPS hosting, dedicated servers are difficult to scale. This is one reason why cloud-based hosting, where the server can be resized and grouped with other hosts, has become more popular for larger sites than dedicated hosting.
There are standard features that you should expect to receive along with your hosting, and many features that hosts offer that you should be aware of beyond simply hosting a site.
I've put this at the top of the list because it's the easiest thing to check, one of the most important things to get when you need it, and the hardest thing when you miss it. Before you pay a host, call their support line. Oh, they don't have a support line, you say? Glad you know that now, aren't you?
Be sure that there is some way to contact a human in the case of a problem. Know what it costs to have their customer service address issues that you've created, like restoring things form backup or fixing mangled server configurations. Be aware of whether the host offers a downtime/status monitor at a dedicated location on the web. (There's nothing worse than a whole host going down and their status page going with it.) Check out the host's knowledge base to see what kinds of questions their customers typically ask. This can be enlightening and save a lot of trouble over the long run.
Most often the web server used for hosting is going to be Apache, but some hosts offer specialized web servers for the sake of performance or the technology platform used. Knowing whether the web server they provide can handle your site is important if you have a specialized CMS with unique requirements.
There are two types of control panels. One lets you control your hosting plan as a whole - arrange payment and configure new servers. The other lets you configure features on individual domains or sites. cPanel is a popular example of a shared hosting control panel. Some control panels allow you to deploy software on your site with the push of a button. These are convenient, but be aware that the control panel may not be updated with the latest version of the deployed software, and you should make sure that your software is fully updated.
Some hosts provide email for your domain as part of the hosting offering, including web access to the email. Usually, the email that comes with the web host is not as feature-rich as found in dedicated email hosts. With the ubiquity of GMail for domains, you often need a specific reason to use email that your host provides. Still, if needed, host-provided email can be essential.
Hosting logs are nice to have access to, not so much for analytics (which is now handled better by third-party services) but for access to error reporting. If something goes wrong on your site, having access to the error logs can be essential in sorting out the problem. PaaS and Shared Hosts sometimes make these logs difficult to find or obtain.
Domain names can also be part of your hosting package. Depending on the configuration of your site, your host may even require that your DNS records are hosted with them. Be aware of the situation before you buy, especially if your host offers to register the domain on your behalf.
You are the only person responsible for your site backups, but your host can provide you with the resources to make backups easy. Even if you are able to create "images" of your whole hosting system in case of emergency, you should see if your host offers any way to store those images (or other backups) in a remote location that is compatible with other platforms. A host that offers compatibility with other hosts isn't afraid of losing your business, and that's a good sign.
Shell access is something that sounds advanced, but if you're using a server that has it, it can be invaluable for a few reasons. If you use source code management tools, like subversion or git, having shell access on your server will allow you to use those tools to deploy to the server directly.
If you use FTP to transfer files to your site, you should stop! Shell access allows you to securely connect to the server and use SFTP to transfer files securely. While the files you transfer might not be important to some hacker snooping on your traffic in a coffee shop, the FTP password to your site definitely would be useful for uploading whatever botnet software they like.
Things to Look Out For
Here are some random things that you should look out for or be wary of when choosing a host.
You Get What You Pay For - As I mentioned above, if the price is too good to be true, then there is usually some deep flaw hidden under the surface. Any hosting that offers a price disproportionate to the features is probably worth skipping.
Get Recommendations - Don't jump immediately at the cheapest host you can find having the features you need. Be sure to solicit recommendations from peers or user groups so that you know you're dealing with a reputable hosting company.
Spread Your Eggs Around - Many hosts offer one-stop shopping for all your hosting, be it web, email, or DNS. I recommend that you do not host your site at the same place you register your domain, for a little added security if you need to transfer your site to a different host.
Be Wary of Overselling - As mentioned earlier, one of the easiest ways to get shafted by a host is by using oversold services. If you choose a shared host, be sure that you get some recommendations from other users so that you know how the host behaves when your site gets the traffic you're hoping for.
Switch Hosts at Whim - After you deploy your site, you'll want to keep regular backups of all your code and data. This isn't just to mitigate catastrophe, but to make it easy to deploy your site to a new host at a moment's notice if they're not performing to your standards. With so many competitively priced hosts out there, there is no reason to stick with a host that doesn't provide the features or support you need. Arrange your backups in such a way that migration to a new host is trivial.
The Admin Matters - There should be a web interface for your interaction with the host, for doing things like initial configuration of the server, submitting support tickets, and payment. If the web interface is difficult to use, or is riddled with upselling ads, this is often a sign of the kind of thing you can expect from support.
Keep an Eye Out for Lifetime Hosting - Sometimes hosts offer customers the ability to pay one large sum for a lifetime hosting account -- hosting that is ever after free of cost provided that the company remains in business. Usually these accounts are pretty low on the support attention span, but if you can live with that for a simple site, it's a reasonable way to get hosting cheaply from an otherwise reputable host.
Server Farm photo by Simon Law
When presented with the idea that either Serif or Sans-Serif fonts might be more legible, you might think there was some credibility to the idea:
Back in 1998 when Times New Roman was still widely used on the web, my then boss made sure we always designed our web sites with Arial, as she hated the look of serif fonts on the web. Was it the case that sans serif fonts were more legible, or was it just a matter of taste?
As it turns out, there's a lot more to it.