Varnished WordPress

As this blog comes closer to the 10th year anniversary, my blogging software choices continue to change. So far the path has been: Livejournal -> Movable Type -> WordPress -> Drupal 5 -> Pressflow Drupal 6. Since I had some time on my hands lately, as an exercise, I decided to upgrade to Drupal 7, but after a few hours gave up in complete disgust. Drupal community is very proud for keeping its technical debt very low, but they rarely talk about who pays it.

It turned out to be quicker to export everything to the latest version of WordPress. I still like Drupal, but latest WordPress has some really nice features. I also took the time to install Varnish and trick out my sites with reasonably advanced caching via Memcached and APC. I also switched to Percona MySQL just for kicks.

Webmaster Tools Crawl stats

I’m still working on theming the blog, and tweaking, so things will break from time to time.

Gmail and tracking numbers

Hunting around for UPS, Fedex,USPS, and Japan Post tracking numbers in Gmail is no fun. I really wish there was a way to aggregate all the shipping numbers in a single Gmail plugin which would at a glance tell me where all the crap that I ordered is at any given moment. Google already knows how to tell a tracking number from all other strings, and there are apps for iOS that aggregate tracking (unfortunately you have to manually type in all the tracking numbers). A Gmail plugin that would keep track of tracking numbers would be great – maybe anyone with a bunch of mythical %20 percent time at Google will implement this…

Pro Drupal Development

Pro Drupal Development is strongly recommended for any PHP programmer who wants a truly in-depth look at how Drupal works and how to make the most of it.

— Michael J. Ross, Web developer/Slashdot contributor

Drupal is one of the most popular content management systems in use today. With it, you can create a variety of community-driven sites, including blogs, forums, wiki-style sites, and much more. Pro Drupal Development was written to arm you with knowledge to customize your Drupal installation however you see fit. The book assumes that you already possess the knowledge to install and bring a standard installation online. Then authors John VanDyk and Matt Westgate delve into Drupal internals, showing you how to truly take advantage of its powerful architecture.

Youll learn how to create your own modules, develop your own themes, and produce your own filters. You’ll learn the inner workings of each key part of Drupal, including user management, sessions, the node system, caching, and the various APIs available to you. Of course, your Drupal-powered site isnt effective until you can efficiently serve pages to your visitors. As such, the authors have included the information you need to optimize your Drupal installation to perform well under high-load situations. Also featured is information on Drupal security and best practices, as well as integration of Ajax and the internationalization of your Drupal web site. Simply put, if you are working with Drupal at all, then you need this book.

  • This book is written by Drupal core developers.
  • Drupal architecture and behavior are mapped out visually.
  • Common pitfalls are identified and addressed.
  • Chapters provide regular discussion and reference to why things work they way they do, not just how.
  • The front matter features a foreword by Dries Buytaert, Drupal founder.

CRUD Ain’t Hard

And now for a little exercise in armchair software architecture — the most despicable coder’s pastime. Dear non-coding readers: despite its name, this blog is still mostly not about programming. Just skip this post or something. Dear coders, many of you will probably disagree with me. I am not a very good or accomplished coder myself, and you probably should not be taking your advice from me. But then again, I could be right, so keep your mind open.

You might have been aware of the very popular, but uptime-challenged social networking tool called Twitter. They have one of the best problems to have: too many very active users. The site is so popular that it constantly goes down and displays and “over capacity” screen that the users have nicknamed The Fail Whale.

Rapidly writing and displaying short chunks of text with high concurrency on the web is not one of them unsolvable problems in programming. It’s not easy, but with right people and tools Twitter could be rewritten inside a month. Twitter founders should do some soul searching. Meanwhile the critical mass has already been reached, the niche for bloggers who want to SMS instead of blogging is big, and even horrible uptime can’t this service. I use it myself.

There is a lot of speculation in the blogocube about whether the reason behind the Fail Whale is the wrong choice of technology — the highly hyped and sexy Ruby on Rails and if it can “scale”. Or is it just simple incompetence?

To me Ruby on Rails falls into a class of technologies that are affected by what I call “the VRML syndrome.” Basically, if I wait long enough the hype will go away, the recruiters will stop posting job listings requiring 4 years of experience in a 4 month old technology, books as fat as my two fists will stop being published, and I will not have to learn it.

What’s the problem with Ruby on Rails? Well, it’s the same problem that slightly affects the content management system that I am currently working with (Drupal), and is the reason why I completely gave up using Microsoft web technologies which are saturated with this shit. See, software craptitechts all of a sudden decided that writing CRUD applications is too difficult for regular developers, and complicated GUI tools and frameworks need to be created to help the poor things. CRUD stands for “Create, Read, Update, Delete” and is just a funny way to say “a browser-based application chock-full’o forms”.

The default way to build these is to rather simple. You hand-code the html forms, then you write functions or classes to deal with the form input — validators and SQL queries for creating, updating and deleting. Then you write some code that will query the database and display the saved data in various ways: as pages, xml feeds, etc. None of this is difficult or non-trivial. Bad coders don’t do a good job of validation and input sanitizing resulting in the Little Bobby Tables-type situation, but these things are not very hard to learn and there are great libraries for this.

Ruby on Rails makes it very easy to create CRUD apps without hand-coding forms or writing SQL. RoR goes to great lengths to abstract out SQL, not trusting the developers to do it right. SQL is more functional than procedural, and thus a difficult thing for many programmers to grasp, but it’s not that hard. Really. SQL is located far enough levels from the machine that abstracting it out becomes a horrible thing due to the Law of Leaky Abstractions. Even when you have full control of SQL queries optimizing them is sometimes hard. When they are hidden by another layer it becomes next to impossible.

In short, RoR makes something that is easy (building CRUD apps) trivial, and something that’s hard – optimizing the database layer next to impossible.

In Drupal there are two modules, CCK and Views that allow you to create CRUD entirely through web interfaces. This is a feature that exist in just about every major CMS, it’s just that in Drupal it’s a little buggier and overcomplicated than necessary. These are fine for small websites and are really useful to amateurs. The problem arises when these are used for high traffic websites.

I think that a lot of people will agree with me that writing HTML and SQL queries using GUI tools is amateur hour. You just can’t make a good website with Microsoft Front Page. You can’t, you can’t, you can’t. But in Drupalland it’s all of a sudden fine to use Views to build queries for high traffic sites. Well, it’s not. Dealing with Views and Views Fast Search has been an ongoing nightmare for me. Hell is not even other people’s code in this case. It’s other people’s Views.

RoR, Views, CCK are one level of abstraction higher than you want to be when building a high performance application. The only way the can be an “Enterprise” tool if your enterprise is a) run by a morons that require 100 changes a day AND b) has very few users. In short, if it’s an app for the HR department of a company with 12 employees – knock yourself out. If you are building a public website for millions of people – forget about it.

Your, Deadprogrammer.

P.S. Yes, I know, you can abstract just about everything and reduce your software application to a single button labled “GENERATE MONEY”. You have to be a very smart LISP developer for that.

My Return to Blogging

I am finally free of lousy Dreamhost. I also switched from WordPress to Drupal. There probably will be some glitches and missing urls, and the design will stay in the current stock “Garland” theme, but I am back.

I also apologize in advance for rss feed glitches that might happen – I am still tweaking the site.

TT : Misc

Wow, who knew. According to Wikipedia, the order of the Teutonic Knights exists today as a charitable organization (they don’t seem to have a website though). It even has a grandmaster – one Bruno Platter. And I thought that Alexander Nevsky got rid of them completely. Or at least the Soviet era cartoon that I’ve seen made think so, because my education in that period of Russian History never progressed much further than that.

I noticed that at home I mostly use Outlook as a very slow and crappy spell checker for my blog. I only keep it around because of the spellchecker and because Gmail does not have an import feature. I tried importing with Gmail Loader, with the whole crappy export to Firebird thing, but that messes up most Russian emails and does not set the dates correctly. Why Google does now provide an import utility is beyond me – it would have completed lock-in for so many users. Also, I wonder, what’s the best software spell checker that money can buy?

Proudly Serving My Corporate Masters presents : Microsoft’s famous (mentioned in Microserfs and the Joel’s rant ) Ship-It award throughout the ages. I wonder if I could buy one of these on eBay :)


I really hate email these days. Gmail might have solved (at least for me) the storage problem and mostly solved the spam problem (the filter is very efficient), but there is soooo much crappieness in email.

Email servers and clients are just out of whack lately. Even Gmail checks zip files for executables somehow (neat trick) and refuses to add them. It works ok if you change the extension to or something like that. But this at least is a decent way of dealing with the problem of people sending virus laden executables – warn that you are not sending it and let through people who are smart enough to rename the extension.

On the other hand I’ve encountered every type of nastiness – from silently dropping emails to stripping the attachments (again, silently) to bouncing the email back with absolutely unintelligible error messages.

Filter stupidity similar to what excellent Joe Grossberg is describing here is also rampant.

Oh, and trying to send out an email in Russian. Fugedaboudit! The extra bits in Unicode or KOI-8 get chewed off every which way rendering my laboriously typed and spelling error infested emails unreadable half the time. If there is a way to reliably send Russian encoded emails without using attachments – I was not able to find it yet.

Worst of all, you sit there waiting for a replies wondering – are people just ingnoring me? Did the message get silently dropped, swallawed or chewed up on the way? Did it get lost amongst spam about Ciagra and Vialis? (As a side note, my co-workers were joking this morning about how I should write on my cubicle dweller’s box that contains vitamins, painkillers, antiacid and caffeine pills “V1A8RA” in marker). Did the person mean to answer me but forgot lately? Did something happen to him or her?

But you know what I hate even more than email? Public comments in blogs. Letting my own often illiterate and/or stupid comments spill out onto the Information Superhighway and having them fester and petrify there for future generations is not a good idea. From now on my policy is not leaving any comments whatsoever. I’ll use exclusively email from now on. If you want to leave me a public comment in Livejournal – go ahead, but I’ll probably answer via email. I do try to answer most comments.

Also a part of this policy is not reading or writing any private posts in Livejournal. Nothing good ever comes out of them.

In other news, I am thinking about leaving a little note at the bottom explaining obscure puns in my topics. For instance this one is based on the Sopranos Episode 204 title – “Commendatori” (Knights). Babelfish tells me that “commentatore” means “commentator”.