I have some advice for those in the business of building large websites with content management systems.

1) Do not implement search yourself.

Your CMS sucks at search, and so do you. I see this again and again and again. Everyone is implementing search on large websites instead of using Google. Developers are afraid of looking unprofessional. Managers are answer yes to the question "do you want advanced/faceted search" (the correct answer is no - user's don't like it and don't use it). As a result a lot of resources (both server and developer) go into implementing something that Google is awesome at. Even some very smart people, like Jeff Atwood roll their own search, and their users end up going to google.com and typing "foo site:stackoverflow.com".

Users are very happy with Google CSE, and don't mind the text ads. Those text ads - well, that's revenue that you would otherwise would not have, however small this is. If you absolutely can't do Google CSE - buy their search appliance. If you can't do that either - well, you better be using Solr.

2) Do not implement comments yourself (unless comments are what you do for a living).

It is extremely difficult to get comments right. Users absolutely abhor comments. Spammers - well, they love it. Luckily, you can just go and get DISQUS to do all the heavy lifting for you. The time saved on using DISQUS can be used on building something else, meanwhile users absolutely love leaving comments through it, while spammers hate it.

3) Physically separate your admin interface from the stuff that is going to be used by your users.

Maciej Ceglowski has some words of advice about not having your blog hacked: cache your output in flat files and hide the admin interface. The benefits of this are tremendous: cached files are fast and secure. You will need to do some fancy footwork to serve up parts that change a lot, but you can do it the same way DISQUS and Google CSE do it - through the magic of AJAX.

4) Sanity check: calculate the amount of RAM in the home computers of all of your interns. Compare that to the amount of RAM in your server farm. Who wins?

5) Use a CDN and/or caching proxy, don't be cheap. These things will save your butt when Yahoo and Digg will come a-knocking at the same time. I'm not even going to mention Memcached - you can't get big without it at all.

6) Fight WYSIWYG editors. These things are the worst. They are the Devil. They are a security hole. You never get what you see. People paste from Word. Do I need to go on?

The best middle of the road solution is something like Markdown.

Do not underestimate the user's ability to learn a few simple rules. When I worked at TV Guide there was this movie database application. Very non-technical editors were using a very scary-looking Unix-based interface at an amazing speed. When I rewrote it as a web interface, it became more "user-friendly", but they could not enter stuff as fast as before.

7) Make sure you have good backups

8) I know you won't be able to follow my advice, I know I can't either. Life is a constant compromise.

There's something that has been bothering me for a while, something that I call "Homer Simpson's toothpick school of blogging". In one of the Simpsons episodes Homer is marauding a grocery store at brunch, making a meal out of free samples. He proceeds to eat a few non-sample items by proclaming that "if it has a toothpick in it, it's free" and sticking his toothpic into a variety of items. He even drinks a beer, piercing it with a toothpick. The most successful blogs are basically like that: they either paraphrase or directly quote juiciest pieces of online articles. There might be a little bit of commentary (the snarkier - the better), but the meat of these blogs is in the quotes.

This is known as "curating" - the successful toothpickers have excellent taste in content. The people they quote and take images from are very glad to receive traffic from these A-listers. BoingBoing.net, kottke.org, daringfireball.net are like that: short, high volume (once you get the hang of it, it does not take much to turn that interesting site in your firefox tab into a pithy little wrapper around a juicy quote), very enjoyable. More so than mechanized versions of the same thing like digg.com and stumbleupon.com. For one, submitters don't do a very good job of quoting or paraphrasing, and you find yourself clicking on links more. Very successful blogs stick their toothpics into so much content that you don't really need to click through to the originals much: I can read BoingBoing, Gothamist or Lifehacker without clicking too much - the juiciest stuff is already there. In fact Gothamist seems to be almost completely pulled from from New York Times and New York Post headlines. It's a bit like a segment on some NY TV news stations where they read the latest headlines from local papers.

Now, there isn't anything unethical about quoting and paraphrasing - it's all squarely in the realm of fair use. These blogs are a bit like suckerfish that attach themselves to whales or sharks in that they benefit immensely from their hosts. Well, actually, unlike suckerfish they repay the favor by driving traffic.

In fact, I owe most of my readers to the low point in my blogging career, when after failing to submit my post about the Starbucks Siren to BoingBoing through their official black hole form, I begged Cory Doctorow to post it in a personal email. He did, I received tons of traffic and literally thousands of links from BB readers. Now that article shows up at the very top of Google search results for Starbucks logo.

Therein lies a problem: good content on the Internet does not always bubble up to the top on it's own. Blogosphere is a bit like the Black Sea, which has a layer of very active and vibrant biosphere at low depths. But it's very deep, and below 200 meters the depths are full of poisonous hydrogen sulfide, which luckily does not circulate very much (unless there's a particularly strong storm). Think about digg.com or StackOverflow.com- at the top stuff circulates, gets upvoted and downvoted. But below, there's a poisonous cesspool of Sturgeon's Law's 90 percent. And most of the time, new and worthwhile content starts not at the top, but at the bottom, or flutters briefly in above the mediocrity and the bad, does not get noticed and gets buried.

Speaking of StackOverflow, Joel Spolsky and Jeff Atwood recently touched on the topic of blogging success in their excellent podcast. They were discussing Steve Yegge's retirement from blogging, and tried to pinpoint what it meant to be a successful blogger. "Perhaps one metric of success is getting people you respect and admire to link to your writing in an organic, natural way (that is, without asking them to)." I am a miserable failure on this front. Sure, I have some high profile readers, but their link love is rare, while I'm not really below begging for links.

Jason Kottke, an A-list blogger and a primo toothpick sampler, was reflecting on the monetary success. He likened business blogging to shining shoes: there might be some individuals who can get rich by running a chain of shoe shining stores (Jason Calacanis, Nick Denton), and maybe even some individual outstanding shoeshiners (Dooce) who can make a decent living, but for the majority of shoeshiners it's not a very good career choice.

I've read somewhere about my hometown's "king of shoeshiners", a very colorful character. He was the best shoeshiner Odessa has ever seen, famous and loved by all, but he died poor and miserable. On his monument there was a short quote: "life is waksa" (waksa is a Russian word for shoe polish with a connotation of something pitch-black).

For me blogging takes a good deal of effort. In the immortal words of E.B. White "writing is never 'fun'". (White almost rejected an assignment to write an article that became the finest piece ever written about New York when an editor suggested that he might 'have fun'). What makes blogging less fun for me is looking at server statistics, number of comments, ad revenue, and thinking about payoff and success. And feeling like that I maybe should have done something else with my time.

My high school Economics teacher, Mr. Oster, taught me one very valuable concept: "opportunity cost". Whenever you make a decision do something, you almost always pay the opportunity cost - the difference in value you might have gotten by doing something better. Oh, there could be hundreds of things that have a better payoff than not very successful blogging.

I personally do not blog for money, and certainly don't blog professionally (the ads on my site cover my hosting expenses). Well, not yet, anyway - I am preparing stuff for a commercial venture that I'll soon announce. I blog in order to meet people (hanging out a Web 2.0 events and meetups would probably have been more productive), but mostly to get things out of my head. In that sense I'm a bit like Louise Bourgeois. I've recently seen an exhibition of her work, and I'm pretty sure that if she did not create all those sculptures and paintings, the inspiration for them (which must have been glipses of extra dimensions, cellular automata that drive our reality, and super disturbing things that can't even be described) would have made her a raving lunatic and not a lucid and sane 97 year old woman that she is.

I don't really intend on changing the format of deadprogrammer.com - the intricate, long, winding, interconnected posts about obscure topics. I probably would have had a lot more success if I just kept a photo blog about New York City. If I'd just stick to one popular topic and posted every day - I know I would have attracted a lot more readers. Instead, I'm going to start a new, for-profit blog. You'll hear about it soon. I think I should be able to make some shekels with my mad shoeshining skills. And while I agree with Mr. White about writing not being fun, the fund is in having written.