On the Importance of Lube

As a response to meaningless discussions about microformats, standards, widgets and other unimportant web gunk I wrote an article Deadprogrammer’s Hierarchy of Web Needs. The gist of that post is that what matters the most is text and images, and that the importance of everything else above it falls in geometric progression. Things high on the pyramid get too much consideration.

There is one modifier that does not fit on the pyramid: lubrication. You see, there’s a lot of friction associated with putting content online. It’s a major limiting factor to the growth of the internet. Those who focus on the base of the pyramid and apply enough lube succeed.

Twitter succeeded because it is the ultimate lube, the equivalent of a major dose of oil-based laxative. It lets you put little pooplets of thought at the speed of diarrhea. Text alone is enough – it’s the very base of the pyramid. Because of that people forgive Twitter the url shortening pandemic – the very thing that is poisoning the exchange of links, the terrible handling of images, and the procrustean shortening of the information that you can share.

My ideal twitter feed is kind of like now defunct memepool.com, but with inline images. I want good copy, I want good images and I want good links (and not the terrible shortened crap – this is not what hypertext is about).

Besides lube, there is stuff that seems like a good idea, but is actually adding friction. The days of black backgrounds and blinking text are behind us, but the new enemies of eyeballs are more subtle. Hashmarks in Twitter are terrible. I can’t read shit like “ugh, bad #weather in #hoboken #today #firstworldproblems”. Another thing that acts as sand in my eyes is “winerlinks“. They are little hashmarks that let you link to every paragraph in the story (which is a great idea), but at the same time they look like a bunch of bedbugs and scrape your eye with every saccade.

The year is 2011 and we are walking with supercomputers attached to digital cameras more powerful than the ones that went into space probes. Yet sharing an image is still a huge pain in the ass. It just takes too many steps. Iphone apps do it relatively well, even if too many people mangle their perfectly good pictures with a totally un-fun “a fun & quirky way to share your life with friends through a series of pictures” (whatever that means).

Here’s Vannevar Bush talking about “memex trails” in “As We May Think“:

“The owner of the memex, let us say, is interested in the origin and properties of the bow and arrow. Specifically he is studying why the short Turkish bow was apparently superior to the English long bow in the skirmishes of the Crusades. He has dozens of possibly pertinent books and articles in his memex. First he runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items. Occasionally he inserts a comment of his own, either linking it into the main trail or joining it by a side trail to a particular item. When it becomes evident that the elastic properties of available materials had a great deal to do with the bow, he branches off on a side trail which takes him through textbooks on elasticity and tables of physical constants. He inserts a page of longhand analysis of his own. Thus he builds a trail of his interest through the maze of materials available to him.

And his trails do not fade. Several years later, his talk with a friend turns to the queer ways in which a people resist innovations, even of vital interest. He has an example, in the fact that the outraged Europeans still failed to adopt the Turkish bow. In fact he has a trail on it. A touch brings up the code book. Tapping a few keys projects the head of the trail. A lever runs through it at will, stopping at interesting items, going off on side excursions. It is an interesting trail, pertinent to the discussion. So he sets a reproducer in action, photographs the whole trail out, and passes it to his friend for insertion in his own memex, there to be linked into the more general trail.”

Stinging together these trails is still too cumbersome. Have you ever tried to post a picture of two items (what JWZ calls “ exhibit A, exibit B” (this particular link leads to a collection that is totally worth your time)? What if there are no pictures of these items on the internet and you have to scan or photograph it, upload it, crop it, post it? And what if like Vannevar Bush’s bow and arrow researcher you’d like to add a comment in longhand, your own handwriting. Or how about a little hand-drawn diagram? This simple task will likely take at least half an hour.

But enough bellyaching. It’s 2011, and the flying cars are almost here. There’s Skitch and Evernote (Phil Libin seems to be making the dream of Memex a reality in a less lame way than anyone else). And as an alternative to Twitter there is Google+ – I can drag an image from Skitch into a text area and it automatically uploads! When they’ll open up the API doing A/B posts will become finally possible there. Please, please leave the suffocating, hashtag strewn stinkhole that Twitter became. Join Google+. I’ll be hanging out there.

8 Pieces of Architectural Advice for CMS

I have some advice for those in the business of building large websites with content management systems.

1) Do not implement search yourself.

Your CMS sucks at search, and so do you. I see this again and again and again. Everyone is implementing search on large websites instead of using Google. Developers are afraid of looking unprofessional. Managers are answer yes to the question “do you want advanced/faceted search” (the correct answer is no – user’s don’t like it and don’t use it). As a result a lot of resources (both server and developer) go into implementing something that Google is awesome at. Even some very smart people, like Jeff Atwood roll their own search, and their users end up going to google.com and typing “foo site:stackoverflow.com”.

Users are very happy with Google CSE, and don’t mind the text ads. Those text ads – well, that’s revenue that you would otherwise would not have, however small this is. If you absolutely can’t do Google CSE – buy their search appliance. If you can’t do that either – well, you better be using Solr.

2) Do not implement comments yourself (unless comments are what you do for a living).

It is extremely difficult to get comments right. Users absolutely abhor comments. Spammers – well, they love it. Luckily, you can just go and get DISQUS to do all the heavy lifting for you. The time saved on using DISQUS can be used on building something else, meanwhile users absolutely love leaving comments through it, while spammers hate it.

3) Physically separate your admin interface from the stuff that is going to be used by your users.

Maciej Ceglowski has some words of advice about not having your blog hacked: cache your output in flat files and hide the admin interface. The benefits of this are tremendous: cached files are fast and secure. You will need to do some fancy footwork to serve up parts that change a lot, but you can do it the same way DISQUS and Google CSE do it – through the magic of AJAX.

4) Sanity check: calculate the amount of RAM in the home computers of all of your interns. Compare that to the amount of RAM in your server farm. Who wins?

5) Use a CDN and/or caching proxy, don’t be cheap. These things will save your butt when Yahoo and Digg will come a-knocking at the same time. I’m not even going to mention Memcached – you can’t get big without it at all.

6) Fight WYSIWYG editors. These things are the worst. They are the Devil. They are a security hole. You never get what you see. People paste from Word. Do I need to go on?

The best middle of the road solution is something like Markdown.

Do not underestimate the user’s ability to learn a few simple rules. When I worked at TV Guide there was this movie database application. Very non-technical editors were using a very scary-looking Unix-based interface at an amazing speed. When I rewrote it as a web interface, it became more “user-friendly”, but they could not enter stuff as fast as before.

7) Make sure you have good backups

8) I know you won’t be able to follow my advice, I know I can’t either. Life is a constant compromise.

Memex is Here

I think I finally found a piece of software I was searching for all of these years, the Memex that Dr. Vannevar Bush predicted. Too bad that the  good ol’ leader of the Majestic 12 is not around to see it.

Evernote is almost everything that I ever wanted in a Memex. It now even has a web component which will let me use it on Linux. The text recognition actually works and is useful, unlike what the retards at Riya were trying to do.

Evernote seems to be rapidly improving, in leaps and bounds. It was around for a while, but without web storage and access I wasn’t interested enough.

This is best symbolized by what I presume is an old logo, which is pretty lame:

The new logo, with an elephant (they never forget), a dog ear ear – now that’s recursive, and an overall look that would not be out of place on an early 20th century pencil box just simply rocks. There’s more hidden imagery in the elephant’s head. In any case, looks like a real pro made this logo.

Where’s My Flying Car Part I : KABOOM!

“Celebrating Gertsen, we clearly see three generations,
three classes acting in the Russian Revolution. First –
noblemen and landowners, Decembrists and Herzen.
Horribly distant from the people. But their work was not in wain.
Decembrists woke Herzen. Herzen began revolutionary agitation.”
V.I. Lenin

Computers have existed like for 200,000 years in Internet time, yet the innovation in computer technology seems to be a little slow. Brick and mortar slow. Let me present to you an approximate timeline:

In 1945 Dr. Vannevar Bush wrote an article As We May Think about a device called the Memex.

In 1960 Theodor Holm Nelson, inspired by Bush, coined the term “hypertext” and started on Project Xanadu, a vaporware Superinternet.

In 1968 Dr. Douglas Engelbart delivered the MOAD, demonstrating videoconferencing, email, hypertext, copy and paste, as well as some novel input devices including a mouse.

Bush, Nelson and Engelbart show a progression from a dream into reality. Bush was a pure dreamer – he never intended to actually try and build the Memex. Nelson at least tried to build Xanadu, although he failed miserably. He could not even get to the demo stage. Engelbart actually built enough stuff to make very impressive demos, although never to build actual successful products except the mouse. These guys suffered from the RAND Corporation syndrome–the common joke went that RAND stood for Reasearch And No Development.

The problem with these three was that they could not focus on individual problems. Luckily for us, next came Xerox PARC. Xerox corporation had money coming out of its wazoo, decided to invest in a world class R&D center. They used the same approach that Google is using today: spend the extra money on hiring the brightest technologists around and let them run free and wild.

Bush, Nelson and Engelbart were a lot like a character named Manilov in Gogol’s Dead Souls. Manilov was an owner of a large rundown estate. He spent his days dreaming about improving it. Wouldn’t it be nice to build a bridge over the river and on it build little merchant booths so that the peasants could buy stuff there. Of course, none of his projects ever went anywhere, and if they did, they were quickly abandoned.

PARC engineers were men of action. Each concentrated on a particular aspect, and they’ve built working models of many things that we enjoy today: personal computer with GUI interfaces, Ethernet, WYSIWYG text editor, laser printer, and even a computer animation system amongst other things. Sadly, Xerox was able to capitalize mostly on the laser printer, which actually probably paid for all of PARC’s expenses. PARC indirectly influenced Apple and Microsoft in the development of GUI OS. Also Charles Simonyi left PARC to develop Word and Excel for Microsoft, thus creating an enormous amount of wealth. Bob Metcalfe and David Boggs also left PARC, took Ethernet and turned it into 3COM. John Warnock and Charles Geschke left PARC, took PostScript and created a little company called Adobe Systems. Well, you get the picture.

To give you another analogy, the technological revolution of the 60s, 70s and 80s was like a hydrogen bomb. A hydrogen bomb is made of three bombs: a conventional explosive that ignites a fission explosive that in turn ignites a fusion explosion. Semiconductor industry created by William Shockley and the Traitorous Eight was the fuel, Bush and Company–the conventional explosion, PARC–fission, what came after–fusion. KABOOM!

The Russian Tea Room Syndrome


“Man told me,” He said, “that these here elevators was Mayan architecture. I never knew that till today. An I says to him, ‘What’s that make me– mayonnaise?’ Yes, yes! And while he was thinking that over, I hit him with a question that straightened him up and made him think twice as hard! Yes, yes!”

“Could we please go down, Mr. Knowles?” begged Miss Faust.

“I said to him,” said Knowles, ” ‘This here’s a research laboratory. Re-search means look again, don’t it? Means they’re looking for something they found once and it got away somehow, and now they got to re-search for it? How come they got to build a building like this, with mayonnaise elevators and all, and fill it with all these crazy people? What is it they’re trying to find again? Who lost what?’ Yes, yes!”

“That’s very interesting,” sighed Miss Faust. “Now, could we go down?”

Kurt Vonnegut, “Cat’s Cradle

The Russian Tea Room, once a popular restaurant created by ballerinas and danseurs (aka male ballerinas) of the Russian Imperial Ballet for themselves and their friends. Later it became an expensive restaurant for the Manhattan high society. In 1996 the new owners closed it down for 4 year and $36 million renovations. In 2002 the restaurant closed, and the owners were bankrupt. In the aftermath, one of the chefs, M.D. Rahman, can be found on 6th avenue and 45th street selling some of the tastiest street food in Manhattan. I bet he’s making more than he did back at the Russian Tea Room now with his little cart.

In the parlance of the Internet this is known as a “redesign” or a “relaunch.” If you are making a living out of web development, like I do, chances are that you participated in a vicious cycle of web site redesigns. They usually happen like this: managers decide to do it and get funding, a lot of meetings follow, specifications are written (or not), arbitrary deadlines are set, designers create graphical mock-ups, then coders swarm and engage in what’s referred to as “death-march.” Managers change their minds about the look and feel a few times during the death-march for an extra morale boost. Finally, a redesigned website launches. Managers start planning the next redesign right away.

In the olden times the CEO’s nephew often got the web design job. Well, these days the nephew grew up, he has a consulting agency. “This is old and busted, let me redesign this mess and you’ll get new hotness” – he says. Pointy-haired bosses everywhere nod and say – “yes, yes, new hotness”, and the cycle keeps on going, redesign after a redesign.

There are a few different types of redesigns. Firs of all, there’s changing the look. In the simplest and best form, this is a very quick deal, especially if the site is properly architected for quick changes. It’s like taking your plain vanilla cellphone, buying a snazzy faceplate, one click – instant new hotness. I have nothing against this sort of redesigns.

The only thing you have to look out for here is what I call the “Felicity effect.” A television show Felicity had a famous redesign failure – the actress Keri Russell cut her trademark long hair. One might argue that she is hot no matter what, but the show suffered a huge drop in ratings. You have to keep in mind that a new look rarely attracts new customers, but often upsets the old ones. For instance, I like Keri’s new look, but I would not start watching that show.

The second type of a redesign involves changing the underlying technology of the site. One might change the content management engine, database engine, rewrite the site in a different language, make it run on a different web server, different operating system, etc. These usually turn out to be the most disastrous and costly of redesigns.

Joel Spolsky wrote about “… the single worst strategic mistake that any software company can make: … rewrit[ing] the code from scratch.” In the web publishing world these kinds of rewrites cause a lot of grief and devastation. A huge technology change always requires a lot of debugging and fixing afterwards, and as soon as most of the bugs are fixed, a new redesign comes around, because, see, ASP.NET 2.0 C# is “old and busted” and Vista Cruiser Mega Platform D## is “new hotness.”

I am not talking here about replacing a technology simply because it does not work or is dangerous. But redesigns are rarely aimed at fixing things – they are done in search of hot technologies and hot looks. By the way, amongst pointy-haired web execs fixing things is less glamorous than perusing new technologies, and that is less glamorous than changing the looks.

A building superintendent I know was in a middle of a huge project – repairing three old and unsafe elevators as well as fixing the crumbling facade of the building. Although the repairs were crucial, they did not earn him the love of the tenants that the old superintendent enjoyed. The old super, instead of fixing broken things, engaged in an almost constant painting projects, changing the color of the paint every time just a little bit. And when he wasn’t repainting, he would leave out the paint bucket and a brush on some rugs in the lobby.

The web execs often go for the best of both worlds – equivalent to changing the foundation of the building (and not the old one was sagging), as well as painting it a new color at the same time. The full Monty web redesign is what the pointy-haired want.

Let’s take a look at the sense that such redesigns make from a capitalist point of view in an area that I know well — web publishing. Web publishing businesses work just like any other. You take some money (aka capital), you spend that money to produce something and you hope that that something makes you even more money one way or another. In economics this is known as Marx’s general formula for capital: Money-Commodity-Money.

Another thing that I faintly remember from my economics class is a rather disturbing concept called “opportunity cost“. See, when you invest money in something you instantly incur this cost. Why? because you can’t invest your money twice, and there always seems to be something you could have invested in that would give you a better return. Let’s say it’s 1995 and you are an editor in, oh, Random House or HarperCollins. You have a budget to publish some children’s books and there’s a pile of proposals on your table. You pick a few. They make money, win awards, etc. Yet, the opportunity cost on every one of those books is about a kajillion dollars, as in that pile there was a certain book by a woman named Joanne Rowling.

In theory, any web executive’s first objective should be to make, and not lose money. Also they should look to minimize the opportunity cost whenever possible. This is of course not the case for many of them. They are thinking: hey I have this fat budget – I can do a big redesign, or …. hmm, what else can I do with that money so it will make me more money?

So how would one go about increasing profits? In the web publishing today content is once again king because of the maturing web advertising, vast improvements in hosting costs and google-inspired web indexing and searching. This was not the case in the earlier days of the web, but now you can directly convert “eyeballs” into profits. The process is rather simple: you create web pages, users visit them, you show users ads (for which you are paid). The relationship is linear – more users = more ad impressions = more money.

So, first of all, you might produce more pages. With search engines like Google, even pages that are hidden in archives of your website will still produce pageviews. The more pages you add, the more revenue you’ll get. In fact, pages with useful information, once placed online become something very dear to a capitalist’s heart – an income generating asset, the very thing that the author of Rich Dad, Poor Dad is so excited about. They are like the geese that lay golden eggs.

The cost of producing more pages comes from three sources: the cost of content – you need to pay someone to write, take pictures, etc; the cost of placing it online – “web producers”, the people who write html, create hyperlinks and optimize images draw a salary; and the cost of hosting/bandwidth – if you are hosting huge videos you costs might be more than what you can get from advertising, but if it’s just text and pictures you are golden. As you surely don’t expect the Spanish Inquisition, there’s the fourth cost: the opportunity cost of showing this content for free, instead of asking for subscription money. The main thing to remember, once the content/feature is created, the costs to keep it online and generating money is trivial.

Besides producing more content, there are other ways of making more money. One might improve the relevance of ads on your pages. If you have a third party ad system, you are pretty much can’t do that. But if you have your own, you might create mechanisms for serving super-relevant ads. Sometimes you might add e-commerce capability to your content website. For instance, if you have a gadget review site, injecting opportunities to easily and cheaply buy the gadgets that you are writing about will likely bring in more more money than machine generated dumb ads.

One might create content that is more valuable to advertisers. For instance, keywords such as “mesothelioma lawyers”, “what is mesothelioma” and “peritoneal mesothelioma” generate ridiculous costs per click on Google’s AdSense. If creating content about “form of cancer that is almost always caused by previous exposure to asbestos” that is so popular with lawyers is not your piece of cake, you can create content about loans, mortgages, registering domain names, etc.

Then we enter the murky waters of web marketing, and especially “SEO” – search engine optimization. In short, if you get other websites to link to your pages, you will get more vistits, partially from those links, and even more importantly, because search engines will place your pages higher in their results. The hard, but honest way to do this is to produce unique, interesting and timely content. No body’s interested in that. Encouraging the readers to link by providing urls that never change and even “link to us” buttons is not in vogue: most web execs prefer non-linkable flash pages. Another way is to pay for links – in the best case for straight up advertising, in the worst case – to unscrupulous “link farm” owners that sell PageRank. Then comes the deep SEO voodoo – changing the file names, adding meta tags, creating your own link farms and hidden keyword pages. At the worst, there’s straight up link and comment spamming. Unethical methods of promoting your business work: Vardan Kushnir who spammed the entire world to promote his “Center for American English” had enough money for booze and hookers, but not many people shed a tear for him when he was brutally murdered (maybe even for spamming). In corporate world the equivalent is the PageRank ban from Google.

So, you could spend your money on all of these things that I described, and hopefully make more money. On the other hand, redesigning a website from top to bottom to make it “look good” or “more usable” will not bring in more “eyeballs”. A redesign of a large site takes several months for the entire web staff. The possible positive aspects of the redesign are these:

1) Faster loading pages
2) Easier to read text
3) More straightforward navigation
4) Cleaner look
6) Bug fixes
7) Switching from a more expensive software and hardware to cheaper

Existing users will probably like you better, but will new ones all of a sudden descend onto the redesigned site? Not likely. In fact, some think that the ugliness of MySpace design is an asset rather than a drawback. People want something from websites. Be it news, funny links, videos, naked pictures, savings coupons or product reviews, design does not matter too much to them. If they can click it, read it and (for the valuable geeks with blogs and websites) link to it – users are generally satisfied.

Here’s an example of a well executed major redesign of a high profile website, the New York Times. NYT always had a well designed website, and the new one is pretty nice too. But is there a lot of new traffic? Here’s an Alexa graph.

At the worst redesigns bring:

1) Broken links (sometimes every single url changes and all links from outside break)
2) Heavier graphics, proliferation of Macromedia Flash
3) Slower loading pages
4) Loss of features and content
5) New bugs
6) New software and licensing costs, more expensive servers

Often this is all that they bring. Broken links hurt the search engine positioning. New software costs money. It takes a long while to work out the bugs.

Here’s an Alexa graph of another major redesign on a website, which name I’d like to omit. Just as the traffic recovered after a big redesign in 2000, a new one hit in 2003. It seems to be recovering again.

The thing is, many businesses are very robust and the disastrous effects of web redesigns do not kill them. Pointy-haired bosses make their buddies rich, while getting kudos for the redesigns. Everyone stays busy, and software companies get to sell a lot of server software.


Listen, people. Let me do a little follow up, and then I’ll shut up about Starbucks and logos for a little while. Honest.

Firstly, earlier I wrote an article about the progression of the NASA logo – the Meatball, the Worm and the Vector.

Secondly, MTA logo is nicknamed the “Pacman” because it looks like the 25 year old video game character. Waka-waka-waka, watch out for the ghosts.

Thirdly, I wrote about the Roslyn Bank, the Blibbet and Starbucks logos too. All I can add to that are these two fine logos of the Microsoft products of the day gone by:

Funny enough this Microsoft product allows to bring Microsoft BOB back to life. Melinda Getes’ legacy endures beyound Clippy!

Fourthly, Amazon is selling this:

Incredibly, they also have Women of Wal-Mart and Women of Enron.

Holy Relevancy, Batman!

I put in Google Adsense ads ( aka Ads by Goooooogle) hoping that maybe they will compensate me for my almost daily Fair and Balanced newspaper, or maybe even cover the cost of my hosting. I am stoked – practically for the first time ever I’ve seen an ad for something that I could use. More than that, it’s an ad for something that I was desperately searching for and could not find:

NYC Subway Mosaic To Buy
Buy restoration mosaic like you see on the walls of the NYC Subway.

It is way better than I expected:

“Subway Mosaic Handmade Tile
This ceramic mosaic comes from a source that has supplied mosaic to a great deal of subway stations in the NYC Subway System. This is the real deal. The clay body is a frost-resistant stoneware body. The glazes are a combination of matte and glossy. You can have a copy of any station in the NYC Subway System made to any size you want. You can even customize the color, if you do not like the original glazes.”

And the prices are way reasonable – 18 x 28 (from what I understand this is a standard small IRT medallion size) is $285.00! I think my bathroom is going to get one of those.

The moral of the story – Google ads are worth paying attention to.

New And Improved Deadprogrammer.

Now with 20% more rantolioum!

No, I am not on vacation. And no, I am not reading the Harry Potter book. And of course I haven’t run out of ideas. I was just a bit busy. In fact, I am still busy. But I will steal time from sleep, reading and Tivo to bring you more good stuff.

Coming soon to a browser near you:

  • Deadprogrammer’s favorite skyscraper (this is going to be a good one)
  • Really awesome artifacts of Art Deco era
  • Pen computing, mind mapping, memex and friends (continuation of the rant)
  • NYC subway mosaics
  • Rants about Tablet PC
  • Some more artsy fartsy pictures, why not
  • Sci-fi book that I want to plug
  • Livejournals you should read
  • All I Really Want for Christmas is a Memex

    I am finishing up “Bootstrapping: Douglas Engelbart, Coevolution, and the Origins of Personal Computing “. Next up are “The Dream Machine: J.C.R. Licklider and the Revolution That Made Computing Personal” and “Endless Frontier: Vannevar Bush, Engineer of the American Century”. I’ve read a good chunk of Ted Nelson’s “Literary Machines”. It’s difficult. Just like Nelson’s personality. I’ll write about all of that in a little while.

    I can’t fricking believe how expensive “From Memex to Hypertext: Vannevar Bush and the Mind’s Machine” is. You know, overall, used books at Amazon got so expensive. I used to be able to find almost anything for a few bucks, but now people have snapped up all the cheap copies. The only advantage of buying used books there is that they will be shipped faster. This sucks. Where is Xanadu? Where is my Memex? Where is my flying car?

    Speaking of expensive. I’ve finally broke down and purchased a Tablet PC. It’s an Acer TMC102Ti . I’ve got it for about $1700 at ecost.com, and it comes with a $100 rebate. I finally own a laptop! I’ll post a review soon.