On the Importance of Lube

As a response to meaningless discussions about microformats, standards, widgets and other unimportant web gunk I wrote an article Deadprogrammer’s Hierarchy of Web Needs. The gist of that post is that what matters the most is text and images, and that the importance of everything else above it falls in geometric progression. Things high on the pyramid get too much consideration.

There is one modifier that does not fit on the pyramid: lubrication. You see, there’s a lot of friction associated with putting content online. It’s a major limiting factor to the growth of the internet. Those who focus on the base of the pyramid and apply enough lube succeed.

Twitter succeeded because it is the ultimate lube, the equivalent of a major dose of oil-based laxative. It lets you put little pooplets of thought at the speed of diarrhea. Text alone is enough – it’s the very base of the pyramid. Because of that people forgive Twitter the url shortening pandemic – the very thing that is poisoning the exchange of links, the terrible handling of images, and the procrustean shortening of the information that you can share.

My ideal twitter feed is kind of like now defunct memepool.com, but with inline images. I want good copy, I want good images and I want good links (and not the terrible shortened crap – this is not what hypertext is about).

Besides lube, there is stuff that seems like a good idea, but is actually adding friction. The days of black backgrounds and blinking text are behind us, but the new enemies of eyeballs are more subtle. Hashmarks in Twitter are terrible. I can’t read shit like “ugh, bad #weather in #hoboken #today #firstworldproblems”. Another thing that acts as sand in my eyes is “winerlinks“. They are little hashmarks that let you link to every paragraph in the story (which is a great idea), but at the same time they look like a bunch of bedbugs and scrape your eye with every saccade.

The year is 2011 and we are walking with supercomputers attached to digital cameras more powerful than the ones that went into space probes. Yet sharing an image is still a huge pain in the ass. It just takes too many steps. Iphone apps do it relatively well, even if too many people mangle their perfectly good pictures with a totally un-fun “a fun & quirky way to share your life with friends through a series of pictures” (whatever that means).

Here’s Vannevar Bush talking about “memex trails” in “As We May Think“:

“The owner of the memex, let us say, is interested in the origin and properties of the bow and arrow. Specifically he is studying why the short Turkish bow was apparently superior to the English long bow in the skirmishes of the Crusades. He has dozens of possibly pertinent books and articles in his memex. First he runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items. Occasionally he inserts a comment of his own, either linking it into the main trail or joining it by a side trail to a particular item. When it becomes evident that the elastic properties of available materials had a great deal to do with the bow, he branches off on a side trail which takes him through textbooks on elasticity and tables of physical constants. He inserts a page of longhand analysis of his own. Thus he builds a trail of his interest through the maze of materials available to him.

And his trails do not fade. Several years later, his talk with a friend turns to the queer ways in which a people resist innovations, even of vital interest. He has an example, in the fact that the outraged Europeans still failed to adopt the Turkish bow. In fact he has a trail on it. A touch brings up the code book. Tapping a few keys projects the head of the trail. A lever runs through it at will, stopping at interesting items, going off on side excursions. It is an interesting trail, pertinent to the discussion. So he sets a reproducer in action, photographs the whole trail out, and passes it to his friend for insertion in his own memex, there to be linked into the more general trail.”

Stinging together these trails is still too cumbersome. Have you ever tried to post a picture of two items (what JWZ calls “ exhibit A, exibit B” (this particular link leads to a collection that is totally worth your time)? What if there are no pictures of these items on the internet and you have to scan or photograph it, upload it, crop it, post it? And what if like Vannevar Bush’s bow and arrow researcher you’d like to add a comment in longhand, your own handwriting. Or how about a little hand-drawn diagram? This simple task will likely take at least half an hour.

But enough bellyaching. It’s 2011, and the flying cars are almost here. There’s Skitch and Evernote (Phil Libin seems to be making the dream of Memex a reality in a less lame way than anyone else). And as an alternative to Twitter there is Google+ – I can drag an image from Skitch into a text area and it automatically uploads! When they’ll open up the API doing A/B posts will become finally possible there. Please, please leave the suffocating, hashtag strewn stinkhole that Twitter became. Join Google+. I’ll be hanging out there.

I, for one, welcome our new social overlords

Disclamer: I thought that Google Wave was an excellent idea, so you can safely disregard my blathering here.

Here’s what I’m picturing in my head: Google has approached Facebook and Twitter on the playground. Twitter stole a piece of Facebooks lunch, but can’t really hold onto it. After a few threats and a bit of running around and a few ineptly thrown punches Google got itself into position to really clean Facebook’s clock and take its lunch. Foursquare and Groupon which earlier evaded Google’s punches in the most ebarracing for Google way possible are likely to be lunchless later. It is rather strange that Google does not go after scrawny TV Guide and White Pages – it looks like their lunches are not that tasty.

Yes, it’s just another social network. Yes, Google has a track record of failing fast and frequently (which if I remember correctly is a “good thing”). But remember, a bunch of incompetent coders received such an applause, press coverage and a whole evem some money to build a Facebook alternative. And finally mighty Skynet is doing the same thing. I think the company behind the mighty Skynet and the future parent of our robotic overlords has a chance against a bunch of compiled spaghetty PHP.

P.S. Zuckerberg and his approach to privacy creeps me out, so I have deleted my Facebook account and turned it into a blank account used only for work (writing Facebook apps, testing and such). I’m completely fed up with the character limit on Twitter – it’s nothing more than a feed from my blog. But I do want to share photos, and I do want to post shorter, non-blog-worthy thoughts. I’m really rooting for Skynet here.

The New Blogging Manifesto – Or a 3 Ways (4 Ways) To Make Blogging Easier

I noticed that twitter sapped much of my blogging mojo, and I am not happy about that. Wasting a perfectly good photo from my iphone on a twitpic feels painful: it takes an extra effort to view it, and a triple effort to annotate. Here’s my message to Twitter: images should be seen but link urls should not. It’s the other way around, you wildly successful jerks.

Castrating my thoughts with a character limit is unpleasant as well. How much information do I need to sacrifice for the ease of posting? Twitter is like Procrustes, a Greek mythological dude who would chop off the legs of his guests to fit the length of his bed. Twitter’s procrustean limits mess with my procrastination. See, a painful pun like this is impossible on Twitter.

Facebook has much saner character limit and link/image handling, but I really don’t want to place my junk in the “walled garden” of “a host of a party who goes through the pockets of the coats his guest hang up” (I don’t remember the source of the second metaphor, but I like it a lot). I got tired of twiddling settings every time Zuck’s army decided to opt me into yet another privacy nightmare. I dumped my old account and created a new one that I only use for work-related testing and development.

So, over the weekend I redesigned deadprogrammer.com. Here are my new rules for blogging:

1) The blog post input form goes on the front page. I’m basically aping WordPress’ P2 theme. Having a post form staring you in the face instead of being a few clicks away is amazing. It changed the way WordPress developers blog, and I’m hoping it will do the same for me (it seems to be working).

2) Big images. I’m tired of small images. The screens are big, the bandwidth is cheap, almost everybody has a fast connection, my camera takes amazing pictures that lose much of their life when squeezed into 600 pixel width. Then New standard width is 1000 pixels.

3) The P2-style post form is the first step on removing friction out of posting. But that’s a topic for another post – I need to keep my missives manageable. I’ll break things up: there will be pithy posts, and medium length ones, and then there will be long David Foster Wallacian ones (I just need to figure out the best way to do footnotes).

4) Facebook and Twitter will get posts from my RSS feed. That’s all they are good for.

eBay and The Michigan Deposit Scam

eBay is such a horrible hassle these days. I tried selling a few things recently, and between the horrible UI, all the hassles with payments, answering questions and shipping it turned out to be a huge waste of time.

I am sitting on a small fortune of items I would like to get rid of, but I don’t want to deal with strangers on Craigslist or going through the eBay rigomarole. An ideal solution would have been an eBay drop-off shop, but it seems that these went the way of the Dodo.

eBay drop-off store is an idea that many have tried, but it turned out mostly like Seinfeld’s Michigan deposit scam.

In one episode Newman keeps trying to find a way to make a scheme that would bring New York cans and bottles to Michigan, which has a 10 cent deposit instead of New York’s 5 cent one. Kramer keeps telling him that it would not work due to the transportation overhead, but finally Newman figures out a way to get a postal truck for free.

It seems that the time overhead is so high on running an eBay store is so high, that most of the bigger ones that tried it went out of business.

In reality the Michigan deposit scam is against the law, but it actully costs the state 14 million a year in lost revenues. It’s doable.

eBay is showing Twitter-like incompetence in serving its customers. While Google gives its customers huge amounts of storage, email, and software for free, eBay can’t seem to provide free image galleries and other useful services, selling out its customers to an unsavory bunch of third party providers. Image storage is not a very difficult technical problem, and neither is url shortening, but eBay and Twitter are still in the dark about it.

Instead of making selling on eBay easy, developing drop off stores, and making its service better eBay seems to be focused on buying and selling unrelated busenesses for billions of dollars (and losing money on it).

wi.nr

A couple of my friends created a new url shortener. Wait, stop booing. There’s a twist – it has the coolest url ever – http://wi.nr. And you can win something or other by just using it. And you get statistics. Well, I guess it’s about it. Did I mention these are my friends that are doing that?

Why should you care how short your url is? Well, it’s basically because the retards at Twitter don’t allow for inline urls (if they will one day, url shorteners would die like they deserve to), and if you want your stuff retweeted, you need to leave a couple of characters for RT and the username. Of course url shorteners are evil in general, and people at Twitter are incompetent technologically, but very lucky. And being lucky is more important than being competent.

The funny thing is, I absolutely legitemately won their first $5 Amazon gift certificate.

Cognitive Filtering and Bayesian RSS

I hope one thing from the future will become popular in 2009: cognitive filtering. If the Internet was Dr. Dorian from the hit tv show “Scrubs”, I would be Dr. Cox with his list of things he cares very little about.

I got this idea from a science fiction book. In John C. Wright’s Golden Age Trilogy the singularity happened and people can upgrade and back up their wetware in any way they can afford. They still had the same problem that Henry Kuttner described in his short story “Year Day” – an overbearing amount of very innovative ads that masquerade as information and other spam. The trick in Golden Age was cognitive filtering: configurable software that removed any manifestations of anything an owner considered unpleasant: ads, sounds, pictures, symbols, and even people.

I like Twitter, and I like Robert Scoble. But I am tired of Robert’s relentless posts about friendfeed (sometimes I’m not even sure if he works with me at Fast Company or at friendfeed). Filtering this out would not be too hard – I could just ignore any post that has “friendfeed” in it. In fact, a Bayesian filter for Google reader, Facebook, and Twitter after a bit of training could do this automatically: I’d just flag posts that annoy me and the filter would analyze the words in the post, figure out which ones occur together more frequently in the posts that annoy me and hide future annoying posts based on that.

To take this a bit further, I would also like a Bayesian filter that would find me good posts from the firehydrant rss flow based on the ones I already like. There seem to be a few of these out there, but I find it hard leaving Google Reader.

7 Things You Can (Mostly) Do Without in Your Web Business

I’ve spent a lot of time in meetings about websites. Not as much as I’ve spent building websites, but a sizable chunk of my career. I mostly spent that time listening and not being listened to. But now that I’m older, have “Sr” in my title (it stands for Senor), a beard, those cool designer glasses, and have a lot more weight in meetings. Mostly due to the fact that I got pretty fat.

Previously I wrote about the evils of redesigns in The Russian Tea Room Syndrome, and about how web developers are like cooks and prison inmates. Restaurants are a notoriously difficult businesses to run, mostly because there are a lot of amateurs who do not understand what is not important. It’s not what’s important. Everything is important. It’s knowing what can be cut, especially in the beginning, that makes some restaurants succeed when others fail.

Here’s my list of 7 things that seem like they are important in websites, but really aren’t. These are not deal breakers. These are the things to think about last.

1) Looks. It’s nice to have a clean and beautiful design. But making a site pretty is not going to make you more money. Just look at plentyoffish.com – probably the ugliest dating website in existence. It does not stop its maker from raking in 10 mil a year without any hard work whatsoever.

2) SEO. SEO is the alchemy of the web business. I’ve seen more sites get sandboxed by Google than gain pagerank from SEO efforts. Most big url rewriting efforts create broken links, which are bad no matter how you look at it. Don’t break urls, if you can – make them descriptive, and try to make your site linkable (i.e. GET instead of POST search forms), but that’s about all that might help you. Spending a lot of money on SEO is just plain stupid.

3) Performance. Everybody hates slow and crashing websites. But unless this lasts for years, it’s not a deal breaker. Twitter suffers from worst imaginable performance trouble. Livejournal went through a long stretch of bad performance. Even the big dogs like eBay and Amazon have a slow spell or outage or two. MS Windows became the most popular OS in the world not because of its stability. Of course it’s currently losing market share to Apple, but this precess took decades. If anything, it looks like Twitter outages make its users miss the service so much, that when they get back in the twitter their brains out after bitching about the outage for a bit.

4) Good branding. A good name, url, and logo are not going to make you more money. They are just not that important. As long as it’s not too embarrassing, like therapist.com it’s going to be ok. If you look on Alexa, icanhascheezburger.com has almost as much traffic as tvguide.com.

5) Pure CSS markup and web standards compliance. I’m sick and tired of being told that “tableless” design is somehow important. It’s not, it’s not, it’s not. Go to google.com, amazon.com, ebay.com, nytimes.com and view the source. You will see tables galore. Wasting time eliminating tables is just plain stupid. And all-div completely web standards-compliant XHTML markup is not going to make you any more money. I refuse to feel bad about using tables. And perfectly validating XHTML is only going to help page scrapers.

6) Keeping the site ad-free. Site users are ok with ads. They really, really are. If you have what they want they will suffer through the biggest ads you can throw at them. “Half Page Godzillas”, “Skyscrapers”, “Page Killas”, “Shrieking Flash Sound Diddlers” – whatever you call your most annoying ad – despite heated assurances from the users, it’s not going to make most of them leave. Some will and more will follow, but it’s not as drastic as you might think. If you have something unique. I’m not advocating horrible Flash ads. “Flash Sound Diddlers” are not more effective in selling stuff than tasteful Adsense ads which will not have anybody at all leave. You can use ad money to buy more servers, more content, ads of your own. This will bring in more users.

7) Widgets. If your entire web strategy is based on building widgets, well, you are in trouble. You are entering an frenzied and very crowded market. Widgets are the bastard child of old school web “badges” and “push technology.” Widgets sometimes work great for increasing pagerank, just like the “web awards” that were given out by some sites in Web 1.0 times. They might get people to link to you, especially if these people are Myspacers that are constantly looking for shiny things to line their pages with. But in the big scheme of things widgets are not a great way to spend ttime and money.

CRUD Ain’t Hard

And now for a little exercise in armchair software architecture — the most despicable coder’s pastime. Dear non-coding readers: despite its name, this blog is still mostly not about programming. Just skip this post or something. Dear coders, many of you will probably disagree with me. I am not a very good or accomplished coder myself, and you probably should not be taking your advice from me. But then again, I could be right, so keep your mind open.

You might have been aware of the very popular, but uptime-challenged social networking tool called Twitter. They have one of the best problems to have: too many very active users. The site is so popular that it constantly goes down and displays and “over capacity” screen that the users have nicknamed The Fail Whale.

Rapidly writing and displaying short chunks of text with high concurrency on the web is not one of them unsolvable problems in programming. It’s not easy, but with right people and tools Twitter could be rewritten inside a month. Twitter founders should do some soul searching. Meanwhile the critical mass has already been reached, the niche for bloggers who want to SMS instead of blogging is big, and even horrible uptime can’t this service. I use it myself.

There is a lot of speculation in the blogocube about whether the reason behind the Fail Whale is the wrong choice of technology — the highly hyped and sexy Ruby on Rails and if it can “scale”. Or is it just simple incompetence?

To me Ruby on Rails falls into a class of technologies that are affected by what I call “the VRML syndrome.” Basically, if I wait long enough the hype will go away, the recruiters will stop posting job listings requiring 4 years of experience in a 4 month old technology, books as fat as my two fists will stop being published, and I will not have to learn it.

What’s the problem with Ruby on Rails? Well, it’s the same problem that slightly affects the content management system that I am currently working with (Drupal), and is the reason why I completely gave up using Microsoft web technologies which are saturated with this shit. See, software craptitechts all of a sudden decided that writing CRUD applications is too difficult for regular developers, and complicated GUI tools and frameworks need to be created to help the poor things. CRUD stands for “Create, Read, Update, Delete” and is just a funny way to say “a browser-based application chock-full’o forms”.

The default way to build these is to rather simple. You hand-code the html forms, then you write functions or classes to deal with the form input — validators and SQL queries for creating, updating and deleting. Then you write some code that will query the database and display the saved data in various ways: as pages, xml feeds, etc. None of this is difficult or non-trivial. Bad coders don’t do a good job of validation and input sanitizing resulting in the Little Bobby Tables-type situation, but these things are not very hard to learn and there are great libraries for this.

Ruby on Rails makes it very easy to create CRUD apps without hand-coding forms or writing SQL. RoR goes to great lengths to abstract out SQL, not trusting the developers to do it right. SQL is more functional than procedural, and thus a difficult thing for many programmers to grasp, but it’s not that hard. Really. SQL is located far enough levels from the machine that abstracting it out becomes a horrible thing due to the Law of Leaky Abstractions. Even when you have full control of SQL queries optimizing them is sometimes hard. When they are hidden by another layer it becomes next to impossible.

In short, RoR makes something that is easy (building CRUD apps) trivial, and something that’s hard – optimizing the database layer next to impossible.

In Drupal there are two modules, CCK and Views that allow you to create CRUD entirely through web interfaces. This is a feature that exist in just about every major CMS, it’s just that in Drupal it’s a little buggier and overcomplicated than necessary. These are fine for small websites and are really useful to amateurs. The problem arises when these are used for high traffic websites.

I think that a lot of people will agree with me that writing HTML and SQL queries using GUI tools is amateur hour. You just can’t make a good website with Microsoft Front Page. You can’t, you can’t, you can’t. But in Drupalland it’s all of a sudden fine to use Views to build queries for high traffic sites. Well, it’s not. Dealing with Views and Views Fast Search has been an ongoing nightmare for me. Hell is not even other people’s code in this case. It’s other people’s Views.

RoR, Views, CCK are one level of abstraction higher than you want to be when building a high performance application. The only way the can be an “Enterprise” tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it’s an app for the HR department of a company with 12 employees – knock yourself out. If you are building a public website for millions of people – forget about it.

Your, Deadprogrammer.

P.S. Yes, I know, you can abstract just about everything and reduce your software application to a single button labled “GENERATE MONEY”. You have to be a very smart LISP developer for that.