CRUD Ain’t Hard

And now for a little exercise in armchair software architecture — the most despicable coder’s pastime. Dear non-coding readers: despite its name, this blog is still mostly not about programming. Just skip this post or something. Dear coders, many of you will probably disagree with me. I am not a very good or accomplished coder myself, and you probably should not be taking your advice from me. But then again, I could be right, so keep your mind open.

You might have been aware of the very popular, but uptime-challenged social networking tool called Twitter. They have one of the best problems to have: too many very active users. The site is so popular that it constantly goes down and displays and “over capacity” screen that the users have nicknamed The Fail Whale.

Rapidly writing and displaying short chunks of text with high concurrency on the web is not one of them unsolvable problems in programming. It’s not easy, but with right people and tools Twitter could be rewritten inside a month. Twitter founders should do some soul searching. Meanwhile the critical mass has already been reached, the niche for bloggers who want to SMS instead of blogging is big, and even horrible uptime can’t this service. I use it myself.

There is a lot of speculation in the blogocube about whether the reason behind the Fail Whale is the wrong choice of technology — the highly hyped and sexy Ruby on Rails and if it can “scale”. Or is it just simple incompetence?

To me Ruby on Rails falls into a class of technologies that are affected by what I call “the VRML syndrome.” Basically, if I wait long enough the hype will go away, the recruiters will stop posting job listings requiring 4 years of experience in a 4 month old technology, books as fat as my two fists will stop being published, and I will not have to learn it.

What’s the problem with Ruby on Rails? Well, it’s the same problem that slightly affects the content management system that I am currently working with (Drupal), and is the reason why I completely gave up using Microsoft web technologies which are saturated with this shit. See, software craptitechts all of a sudden decided that writing CRUD applications is too difficult for regular developers, and complicated GUI tools and frameworks need to be created to help the poor things. CRUD stands for “Create, Read, Update, Delete” and is just a funny way to say “a browser-based application chock-full’o forms”.

The default way to build these is to rather simple. You hand-code the html forms, then you write functions or classes to deal with the form input — validators and SQL queries for creating, updating and deleting. Then you write some code that will query the database and display the saved data in various ways: as pages, xml feeds, etc. None of this is difficult or non-trivial. Bad coders don’t do a good job of validation and input sanitizing resulting in the Little Bobby Tables-type situation, but these things are not very hard to learn and there are great libraries for this.

Ruby on Rails makes it very easy to create CRUD apps without hand-coding forms or writing SQL. RoR goes to great lengths to abstract out SQL, not trusting the developers to do it right. SQL is more functional than procedural, and thus a difficult thing for many programmers to grasp, but it’s not that hard. Really. SQL is located far enough levels from the machine that abstracting it out becomes a horrible thing due to the Law of Leaky Abstractions. Even when you have full control of SQL queries optimizing them is sometimes hard. When they are hidden by another layer it becomes next to impossible.

In short, RoR makes something that is easy (building CRUD apps) trivial, and something that’s hard – optimizing the database layer next to impossible.

In Drupal there are two modules, CCK and Views that allow you to create CRUD entirely through web interfaces. This is a feature that exist in just about every major CMS, it’s just that in Drupal it’s a little buggier and overcomplicated than necessary. These are fine for small websites and are really useful to amateurs. The problem arises when these are used for high traffic websites.

I think that a lot of people will agree with me that writing HTML and SQL queries using GUI tools is amateur hour. You just can’t make a good website with Microsoft Front Page. You can’t, you can’t, you can’t. But in Drupalland it’s all of a sudden fine to use Views to build queries for high traffic sites. Well, it’s not. Dealing with Views and Views Fast Search has been an ongoing nightmare for me. Hell is not even other people’s code in this case. It’s other people’s Views.

RoR, Views, CCK are one level of abstraction higher than you want to be when building a high performance application. The only way the can be an “Enterprise” tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it’s an app for the HR department of a company with 12 employees – knock yourself out. If you are building a public website for millions of people – forget about it.

Your, Deadprogrammer.

P.S. Yes, I know, you can abstract just about everything and reduce your software application to a single button labled “GENERATE MONEY”. You have to be a very smart LISP developer for that.

gPhone

So, Google announced what we are going to get instead of the gPhone. This is a bit like getting $1000 towards college education instead of that hot new toy for your 12th birthday.

This is excellent news, of course. I really hope this will force the evil cell phone companies in the US to either change for the better or go out of business.

I spent a week in the Ukraine, and experienced what the cell phone experience is like in the rest of world.  I purchased a very nice new Nokia phone for about $60, activated a SIM card that came with it and immediately  received a phone number. It came with enough credits for 100 minutes of non-time-of-day restricted conversation. Later I was able to purchase cards with scratch-off code on just about any street corner that refilled my minutes at very reasonable prices.  The competition is fierce and prices are good because you can change phones and SIM cards at will.  Phone calls and SMS messages in the Ukraine were very cheap, and even calls to the US were only about 25 cents per minute.

On the other hand, Verizon, my provider of choice, increased the length of my contract just because I added a single handset, added extra data “services” to my plan without checking with me just because my phone supports them, made using activation of a third party handset a 4 hour rigmarole, not even counting all the time that I have to spend on the phone with them just to make sure that they are not overcharging me. I hate Verizon so frickin’ much, but at least they have enough towers in the city to actually allow to use my phone to, you know, conduct whatchumacallit — phone conversations. Ironically, the usually more reliable SMS messages are dropped or delivered days late with them.