And now for a little exercise in armchair software architecture -- the most despicable coder's pastime. Dear non-coding readers: despite its name, this blog is still mostly not about programming. Just skip this post or something. Dear coders, many of you will probably disagree with me. I am not a very good or accomplished coder myself, and you probably should not be taking your advice from me. But then again, I could be right, so keep your mind open.

You might have been aware of the very popular, but uptime-challenged social networking tool called Twitter. They have one of the best problems to have: too many very active users. The site is so popular that it constantly goes down and displays and "over capacity" screen that the users have nicknamed The Fail Whale.

Rapidly writing and displaying short chunks of text with high concurrency on the web is not one of them unsolvable problems in programming. It's not easy, but with right people and tools Twitter could be rewritten inside a month. Twitter founders should do some soul searching. Meanwhile the critical mass has already been reached, the niche for bloggers who want to SMS instead of blogging is big, and even horrible uptime can't this service. I use it myself.

There is a lot of speculation in the blogocube about whether the reason behind the Fail Whale is the wrong choice of technology -- the highly hyped and sexy Ruby on Rails and if it can "scale". Or is it just simple incompetence?

To me Ruby on Rails falls into a class of technologies that are affected by what I call "the VRML syndrome." Basically, if I wait long enough the hype will go away, the recruiters will stop posting job listings requiring 4 years of experience in a 4 month old technology, books as fat as my two fists will stop being published, and I will not have to learn it.

What's the problem with Ruby on Rails? Well, it's the same problem that slightly affects the content management system that I am currently working with (Drupal), and is the reason why I completely gave up using Microsoft web technologies which are saturated with this shit. See, software craptitechts all of a sudden decided that writing CRUD applications is too difficult for regular developers, and complicated GUI tools and frameworks need to be created to help the poor things. CRUD stands for "Create, Read, Update, Delete" and is just a funny way to say "a browser-based application chock-full'o forms".

The default way to build these is to rather simple. You hand-code the html forms, then you write functions or classes to deal with the form input -- validators and SQL queries for creating, updating and deleting. Then you write some code that will query the database and display the saved data in various ways: as pages, xml feeds, etc. None of this is difficult or non-trivial. Bad coders don't do a good job of validation and input sanitizing resulting in the Little Bobby Tables-type situation, but these things are not very hard to learn and there are great libraries for this.

Ruby on Rails makes it very easy to create CRUD apps without hand-coding forms or writing SQL. RoR goes to great lengths to abstract out SQL, not trusting the developers to do it right. SQL is more functional than procedural, and thus a difficult thing for many programmers to grasp, but it's not that hard. Really. SQL is located far enough levels from the machine that abstracting it out becomes a horrible thing due to the Law of Leaky Abstractions. Even when you have full control of SQL queries optimizing them is sometimes hard. When they are hidden by another layer it becomes next to impossible.

In short, RoR makes something that is easy (building CRUD apps) trivial, and something that's hard - optimizing the database layer next to impossible.

In Drupal there are two modules, CCK and Views that allow you to create CRUD entirely through web interfaces. This is a feature that exist in just about every major CMS, it's just that in Drupal it's a little buggier and overcomplicated than necessary. These are fine for small websites and are really useful to amateurs. The problem arises when these are used for high traffic websites.

I think that a lot of people will agree with me that writing HTML and SQL queries using GUI tools is amateur hour. You just can't make a good website with Microsoft Front Page. You can't, you can't, you can't. But in Drupalland it's all of a sudden fine to use Views to build queries for high traffic sites. Well, it's not. Dealing with Views and Views Fast Search has been an ongoing nightmare for me. Hell is not even other people's code in this case. It's other people's Views.

RoR, Views, CCK are one level of abstraction higher than you want to be when building a high performance application. The only way the can be an "Enterprise" tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it's an app for the HR department of a company with 12 employees - knock yourself out. If you are building a public website for millions of people - forget about it.

Your, Deadprogrammer.

P.S. Yes, I know, you can abstract just about everything and reduce your software application to a single button labled "GENERATE MONEY". You have to be a very smart LISP developer for that.

Blue Sun Corporation is and important, but not very noticeable part of the the brilliant, but so very canceled TV series Firefly. Their logo is everywhere you look, but they are oh so very evil. They conveniently provide all sorts of goods and services, but at the same time they run sinister human experiments, employ vicious killers and wallow in their crapulence in every imaginable way an evil corporation could.

You can buy your very own Blue Sun t-shirt at Think Geek.

In Manhattan there are two corporations that very much remind me of Blue Sun: Verizon and Chase. Every time I deal with them I feel that I am forced to do things that I don't want to do and that I am getting a bad deal. The only reason everybody's dealing with Chase and Verizon is because they are everywhere you look. In Manhattan you can't swing a dead cat without hitting a Chase branch, and Verizon cellular signal reaches underground into some subway stations.

Chase advertises its omnipresence with this sinister ad that could just as well be from an alien infection film.

This kind of ubiquity allows these corporations to charge above market prices and have bad customer service.

Why do I hate Chase? Well, they keep thinking of ways to make depositing money more difficult. First they changed their deposit slips. Am I the only one inconvenienced by that? No. Here somebody altered the little poster announcing the change.

Now they started using cash machines that do not take envelopes, but scan your check. As you, me, and the people who plowed money into Riya, you can't rely on computers to non-trivial optical recognition. I tried depositing 3 checks several times. The machine ate one of the checks (not giving me a receipt) and rejected the other two. I wasted a lot of time and cell phone minutes trying to report the issue (they did not even provide a courtesy customer service phone). I still haven't seen the money from that check.

Lying commission-driven customer service is another big problem. At Chase they constantly trying to sell you something. Once a customer rep tried to sell me a historically market out-performing mutual funds. He had this awesome "prospectus" with charts carefully selected to show crazy returns, but refused to give me a copy so I could research it.

Verizon reps will routinely forget to tell you about contract extension that comes with any service change, even if you don't have get a new phone. Then they will refuse to change anything in your contract. They will add expensive features you don't ask for. Good luck trying to have your defective phone repaired - it's an ordeal.

Both Chase and Verizon are a bad value, but great convenience. I suspect that part of their penchant for name changing is not so much because they keep buying up competition, but because their customers don't think very well of them at all. I was their customer when they were Chemical Bank and Bell Atlantic. They sucked back then too.

The worst part of dealing with banks and communications companies is that they heavily penalize you for your mistakes, but there's not much you can do to charge them for theirs.

Chase stopped sending me Amazon credit card rewards for about a year. An hour of customer service phone calls and a month later I got my Amazon gift certificates. It's free for them to mess with you: you have to do a lot of work to make sure that what you get from them actually comes through. Instead of digitally depositing the certificates, they send them on paper slips containing long strings of letters that you have to type in. It's cheaper to splurge on the cost of printing and mailing in the hope that it will get lost. And if they stop sending them and you forget? Bonus. Also, there's something called "float."

On the other hand, send your credit card payment late and you get a huge fee.

Use a bit more minutes than are in your Verizon plan, and you'll get a bill that will make your teeth grind. But on the other hand, they overcharge you and then sheepishly return the money (which just now happened to me), you don't get to charge them a fine.

I think there was this guy who charged his bank a fine for every mistake that they've made, but I can't find a link.

Anyway, to make the long story short, Verizon and Chase make me want to vomit in terror. I've been with them for years, but it's time for a change.

It's interesting to note that I've worked for both Chase (briefly as a consultant) and for Newscorp. What's interesting about it? Well, Newscorp owns New York Post which was founded by Alexander Hamilton. The "Manhattan" part of Chase Manhattan Bank (as Chase used to be known) comes from The Manhattan company, founded by none other than Aaron Burr. Because I currently work at the World Trade Center, I frequently walk past Hamilton's grave in Trinity churchyard.