Dead Programmer's Cafe

Blog

  • Semi-literate Programming

    I recently finished “Coders at Work“, a series of interviews with famous programmers.

    On one hand, reading a book like this is a downer: it’s very clear to me that I occupy a place that is very close to the median of the bell curve, and the skill level of programmers is a very steep non-linear curve in itself. I’ll never be as good as JWZ or Brad Fitzpatrick. But I knew that before, and I am ok with it. On the other hand, this book inspired me to read more code.

    The programmers in the book disagree on many points, but they mostly agree on the importance of writing readable code and educating yourself by reading other people’s code. I make my living writing in scripting languages, and I haven’t written a line of C or C++ since college. But there’s nothing preventing me from downloading and taking a look at the source of Apache, PHP, MySQL.

    It’s important for me to understand “how the sausage is made” in the PHP stack, and as it turns out, what happens between Apache PHP and MySQL in term of requests and timeouts is not as simple as one might think. I asked at StackOverflow about this, but all the diagrams that people pointed me at were of the very rudimentary type: “look, here’s a happy cow, it goes to Bovine University, look – it’s all shrink wrapped on the supermarket shelf” instead of “sausage farm/slaughterhouse/truck/factory tour, starting with cow insemenation”.

    When I downloaded the source code of mod_rewrite, arguably the most useful Apache module in the world, I was amazed to find out that it’s only 5000 lines of C with comments.

    The book ends with the interview of Donald Knuth, and another two major questions that the interviewer is asking everyone is – “have you read Knuth’s books and have you tried literate programming”. It was interesting to find out that most of the famous programmers use Knuth’s the same way that I do. The books sit on my bookshelf, I look at them, I sometimes try to read them, I skip most of the math. They serve as a constant reminder to me that I suck at computer science even more than I suck at programming, and luckily there are people out there who know all of this stuff who are not idiots like me.

    Here’s a photo of my cubicle at TV Guide circa 2002, Knuth’s books are holding a place of honor next to the mini fridge. By the way, taking pictures of the places where you work and live is something that you should not forget to do: years from now nobody will care about those pictures of flowers, shadows, and sunsets, but

    I’ve read the book about Literate Programming at the time, and was rather inspired by it. Ok, maybe I didn’t read it and more like skimmed it. I don’t think I understood what real literate programming is.

    The way I understand it, Literate Programming is a way to write programs as a narrative that is readable to computers and humans. My father, in his former career a site supervisor (a type of a contractor) is very fond of giving very detailed instructions to me, the same way he used to give instructions to construction workers. His instructions usually are exaustive algorithms, with error handling. I think that his instructions, expressed as a flow of conciousness, would work not only on me and construction workers, but on computers as well, and are similar to what Donald Knuth has in mind. All you really have to do is to build a layer of abstraction between these instructions and a computer language. Also, since computers don’t forget things, he would only need to repeat his instructions once.

    These days my dad is a COBOL programmer. Everybody dumps on COBOL, but in my mind it’s a language worth of a lot of respect. It has a syntax that is very English-like, something that makes reading COBOL code easy. Well, maybe it’s like reading some old-timer’s newsgroup post written in all caps, but it’s still much closer to English than most other computer languages.

    At the time I was reading “Literate Programming” I was using ASP 3.0, IIS, and SQL Server 97. My task was to write a system that would account for booked and pending business. This is something that had to be done since the age of Mad Men. You see, the dealings of clients, account executives (like Pete Cambell), their bosses, account coordinators, creative department, etc are rather convoluted. But in the end, to get paid, you have to have a system that will track who brought in what business, who handled what, and how the commissions need to be split.

    This is normally the realm of something called EAS (Enterprise Application Software). Back at the turn of the century, this area was still dominated by a company called SAP, but there were a few smaller players, like Salesforce.com that tried to package these applications. Any sane IT manager looks to see if an EAS solution can be purchased first. It turned out that TV Guide’s buseness logic was impossible to shoehorn into any existing solution. SAP folks said – yeah, no problem, we’ll build you what you want, but our prices start at $1M, and then there are consultant fees. ERM world is a crazy place, you can read about some true craziness in “Cube Farm”, an account of one hapless developer’s adventures at Lawson Software. It’s a truly riveting book, and I fell that every developer out there should read it. It’s literally Lovecraftian in nature, that book.

    In any case, it fell to me to develop the application from scratch. Inspired by Knuth, I decided to write some semi-literate code. Me and a project manager, Brad, went to the clients and interviewed them at length, documenting their existing process (aka the most complicated set of spreadsheets you’ve ever seen). In the past, before cheap computers, all you needed was a Joan Holloway, but I believe they stopped making them.

    Brad went on to go back and forth with a very terse document about 5 pages in length that described how the new system would work. He would sit down with the clients and go through the narrative, step by step, confirming that this is what they wanted. Meanwhile I created an object oriented library that made dealing with the database, creating forms and navigation elements much easier. This is similar to to what you might find in a CMS like Drupal, only a little cruder.

    When the document shaped up, I created the database schema, and then I took a big chunk of the document and pasted it into one huge comment block. I proceeded to break off chunks of that block and writing the code around it. Interestingly enough, as time went on, the project manager started helping me to write the code: enough of scary database abstration was hidden by simple classes and method, and there were tons of self-evident examples all around to copy and paste. I switched to writing reports that involved cubes, rollups and other fancy stuff. Stored procedures that did the reports also received comments from the document that described the reports.

    This wasn’t a monolythic system – I was writing it for 2 years or so, releasing a chunk after chunk. In the end it was handed off to another developer, the whole transfer took only a couple of hours. There weren’t any major bugs, maintanence issues (I believe I received only one phone call about it after several years of continuous use). All in all I was pretty pleased with this approach and can absolutely recommend it.

    I believe this is the reason why so many English majors become excellent programmers: if you can write for people, you can write for computers. Sometimes there are reasons why you can’t do both at the same time, but there’s no reason not to find some middle ground.

  • The Running Man In Rockefeller Center Is Gone

    AOL’s logo, “The Running Man” (not to be confused with the book of the same name, which in turn is not to be confused with the movie of the same name) is gone from the window in Rockefeller Center.

  • The Capacitor Plague

    I woke up from a nap to a loud pop and a smell of burning plastic. The source turned out to be one of the most precious and important to me digital devices: a ReadyNAS NV+, a small silver box with over a terabyte of hard drives that store my backups, music, and photos.

    Network attached storage (NAS) is an engineering compromise. It’s a storage solution that lets you keep a bunch of drives in a self-contained device. It’s redundant: you can lose a drive (which is a statistical certainty) and not lose your data. There are also handy usb ports that let you connect usb drives and a button to run backup jobs onto these drives. It also serves as a print server, and in theory it can be used as a streaming media server. On the other hand it’s slow (gigabit networks are not fast enough when you need gigs of data fast), a complete nightmare to use with photo managers like Picasa, and an even worse nightmare if you want to use it as a Time Capsule.

    I’ve spent a lot of time babysitting my ReadyNAS NV+: changing the defective RAM that it shipped with, updating the buggy firmware, finding the right drives for it (some don’t have the right temperature sensors). Don’t get me started on what it took to make it work with Mac’s Time Machine.

    And after all that, the one box that was supposed to keep my precious digital archives safe was smoking. This was preceeded by a few days of weird performance issues and a couple of hangs. The power supply finally died a horrible death, and I realized that once again I was falling victim (or “mugu” as Nigerians say) to faulty capacitors.

    According to Wikipedia, the name of this phenomenon is “Capacitor Plague“. There is an epidemic of failure in electrolytic capacitors from certain shady manufacturers. Electrolytic capacitors are usually found in power supplies. They are little aluminum cylinders filled with special film and electrolytic liquid or gel. Power supplies get very hot, and the liquid part of the capacitors, the electrolyte, always wants to either dry up or explode. The formula for the electrolyte is very hard to get right.

    The rumor is that one or a few companies resorted to industrial espionage to steal electrolyte formulations. They weren’t entirely successful – they either got an incomplete formula or just plain Brawndo.

    Spectrum Online did some digging:

    “According to the source, a scientist stole the formula for an electrolyte from his employer in Japan and began using it himself at the Chinese branch of a Taiwanese electrolyte manufacturer. He or his colleagues then sold the formula to an electrolyte maker in Taiwan, which began producing it for Taiwanese and possibly other capacitor firms. Unfortunately, the formula as sold was incomplete.
    “It didn’t have the right additives,” says Dennis Zogbi, publisher of Passive Component Industry magazine (Cary, N.C.), which broke the story last fall. According to Zogbi’s sources, the capacitors made from the formula become unstable when charged, generating hydrogen gas, bursting, and letting the electrolyte leak onto the circuit board. Zogbi cites tests by Japanese manufacturers that indicate the capacitor’s lifetimes are half or less of the 4000 hours of continuous ripple current they are rated for.”

    Wastefulness of today’s society masks the problem: most people don’t perform autopsies on their dead $70 DVD players or $500 computers, they just use that as an excuse to buy the new hottness. The techies with (or without) spare time and soldering skills do the following: fill bulleten boards with tales of saving their devices by soldering in new capacitors; search for instructions on how to solder and purchase capacitors; and curse creatively after doing it for the 5th time.

    The unique thing about the capacitor plague is how easy it is to identify: the capacitors literally blow their tops, venting electrolyte through the special stress relief indentations. It’s also unique in that anybody with a soldering iron has a very good chance of fixing it: the caps are easy to locate and solder. In the age when most electronic components are of the “surface mount” type (the size of a sesame seed) or chips with dozens legs as fine as silk, soldering in a two legged capacitor is very refreshing.

    Here’s a nest of capacitors from my busted power supply: two in the left corner are clearly popped, the one on the right is probably ok:

    In the last couple of years the following devices that I own fell prey to faulty caps: a cheap off-brand dvd player, a speed control on my Dodge Caravan’s air conditioner, a Netgear network hub, a huge and expensive Air King window fan, and now, my ReadyNAS. The interesting thing is that the problem exists in both high end and low end products, as well as in high tech and low tech ones (I did not know there were electronic components in the window fan).

    I am out of warranty on my ReadyNAS because I bought it in May of 07. The following passage leads me to believe that the shitty capacitors are a problem that they are aware of and (maybe) fixed in newer releases of the hardware (they could not offer a 5 year warranty if they used the same capacitors – they’d just go broke).

    “Please be aware that ReadyNAS purchased prior to August 21, 2007 carries a one-year limited warranty. Extended warranty purchased for these ReadyNAS will be honored by NETGEAR. ReadyNAS NV+ and 1100 purchased August 21, 2007 and later have a 5-year limited warranty, and the ReadyNAS Duo has a 3-year warranty.”

    The brand name of the popped capacitors reads “Fuhjyyu”. It lead me to the an urban dictionary entry that says that Fuhjyyu is either

    “1) Chinese word for feces.

    or

    (2) Brand name of abysmal quality capacitors that are installed on logic boards, switching power supplies and various other electronic components.”

    There’s also a post from a guy who implores ReadyNas to stop using those capacitors.

    Then there’s badcaps.net – a global capacitor gripefest that is too depressing to read.

    You can see a nice gallery of busted caps over here

    There are broader implications of this: coupled with the fragile lead free solder, leaky capacitors don’t only cause kajillions of dollars of damage, but will also make electronics of our era impossible to use in the near future. The aluminum in burnable cds and dvds are rotting too, destroying the record of our time.

  • The Russian Profession

    At my current job there are four employees who speak Russian. Three are younger programmers, one is an older janitor.

    Related: my old post, The Unrussian Profession.

  • Cinematic New York

    When you live and work in New York, you spend a huge amount of time on tv and movie sets. Most of the time the sets are abandoned by the shooting crews, but very frequently tv or movie magic is happening as you are walking by.

    Why is New York so overrepresented on screen? Part of it is because it’s New York. But it’s also because the city government is also very friendly to the moving picture industry.

    When I worked on a website for Kenneth Cole, I learned an interesting factoid: the real name of this fashion powerhouse is Kenneth Cole Productions. It turns out that in the early days they abused a perk that the city gives to movie people: ability to park their huge trailers in places where normally only city services vehicles can linger. Cole applied for a permit to shoot a movie called “The Birth of a Shoe Company”, parked a huge truck in front of a hotel where a major shoe show was taking place, and proceeded to sell enough shoes while cameras were rolling (sometimes even with film) to start a company.

    While watching a movie or a show set in New York I get a lot of “oh, hey it’s” and a lot of “hmm, where’s that?” moments. Sometimes a movie or a show becomes more memorable just because its locations are so familiar to me.

    Let me give you some examples about how cinematically impregnated my environs are. Take, for instance 30 Rock. I spent 7 years working in two buildings that are behind 30 Rock, and every little thing in, under and over Rockefeller plaza is seared in my brain. Also, I have the same last name of one of the actors (is Jane Krakowski a relative? Probably not).

    The 47-50th Street/Rockefeller Center subway station that I got out at almost every day for those 7 years (unless I missed a few stops while reading or sleeping) is the one featured in a key scene in Darren Aranofsky’s “Pi”. The Brighton Beach bus stop in “Requiem for a Dream” – one of my first American jobs was right there, handing out fliers for a gypsy psychic. One of the buildings where I worked, 1211 Avenue of the Americas was very subtly featured as Sideshow Bob’s prisoner number in a Simpson’s episode.

    Sterling Cooper corporate headquarters are famously located at a non-existing 405 Madison Avenue. On the other hand 415 Madison Avenue is a very real building where my wife used to work.

    When I go to and from work now, I pass a grating which John McClane ripped off in one of the Die Hard movies to jump on the top of a moving train. The building where I work? Well, it doubles as the Massive Dynamic headquarters on “Fringe”. They do a lot of shooting at the floor where I work. You can see our big conference room called “Jail” in a number of commercials. You know, Doctor House, he’s supposed to stay in New Jersey, but one time he slept on “my” couch at the office after shooting a commercial there. The butterflies of doom from Fringe also live in “Jail”.

    Ironically, the only famous person who went to my hight school is Larry David, the co-creator of a certain show about nothing set in New York, but shot in LA.

  • Alien ESB

    ESB in this photo reminds me of “District 9” aliens.

    #49

  • Treyf

    I find Jewish humor to be one of the best ways to explain certain situations in programming. Here are two that I find particularly funny and useful.

    The first is a true story told me by a friend. I use it when I’m told that good web developers don’t use tables. It goes like this: My friend’s aunt met her religious relatives for the first time after coming to America from the Soviet Union. Horrified at being served pork sausage, they told her: “But auntie, Jews don’t eat pork!”. She replied — “Nonsense, I eat it all the time.”

    The second is an old and racist Soviet-era joke. A Chukcha serves in the Soviet Army, and is an exemplary soldier in border patrol. There’s only one problem — he tends to eat patrol dogs, considering them a delicacy (this untrue ethnic detail must have been created to make the joke setup work). An army psychologist offers to correct this. He sits the soldier down, takes out his watch, and hypnotizes him with the words “you are not a Chukcha, you are a Jew. You don’t like to eat dogs, you like to eat gefilte fish.” The patrol dogs continue to vanish even after the hypnosis seems to have worked. Authorities send another soldier to follow the hypnotized Chukcha around. This soldier reports that the Chukcha sits the dogs down, takes out his watch and hypnotizes them with the words “You are not a dog, you are gefilte fish.” I tend to tell it when I’m told that the act of turning a hack into a Drupal module somehow makes it “gefilte fish.”

  • To the Moon, Alice

    I recently visited my alma mater, Brooklyn College. Some things changed for the better, like the gorgeous new library addition, some for the worse, like the Campus Sugar Bowl restaurant replaced by Starbucks.

    On the other hand, the science classrooms and offices in the old and new Ingersall building seem to have been frozen in time, down to the wall niches. You see, most floors have these glassed in niches which the various departments fill. Compsci displays books written by professors, Geology shows off a collection of minerals and fossils (a fancy one at that), Biology has a series of stands with pickled and dried specimens that I think dates to the 1940s, like something out of a Hellboy comic.

    The Physics department has a very old, dusty and ironic display, seemingly not opened since the 80s:

  • Homer Simpson’s Toothpick Method of Blogging

    There’s something that has been bothering me for a while, something that I call “Homer Simpson’s toothpick school of blogging”. In one of the Simpsons episodes Homer is marauding a grocery store at brunch, making a meal out of free samples. He proceeds to eat a few non-sample items by proclaming that “if it has a toothpick in it, it’s free” and sticking his toothpic into a variety of items. He even drinks a beer, piercing it with a toothpick. The most successful blogs are basically like that: they either paraphrase or directly quote juiciest pieces of online articles. There might be a little bit of commentary (the snarkier – the better), but the meat of these blogs is in the quotes.

    This is known as “curating” – the successful toothpickers have excellent taste in content. The people they quote and take images from are very glad to receive traffic from these A-listers. BoingBoing.net, kottke.org, daringfireball.net are like that: short, high volume (once you get the hang of it, it does not take much to turn that interesting site in your firefox tab into a pithy little wrapper around a juicy quote), very enjoyable. More so than mechanized versions of the same thing like digg.com and stumbleupon.com. For one, submitters don’t do a very good job of quoting or paraphrasing, and you find yourself clicking on links more. Very successful blogs stick their toothpics into so much content that you don’t really need to click through to the originals much: I can read BoingBoing, Gothamist or Lifehacker without clicking too much – the juiciest stuff is already there. In fact Gothamist seems to be almost completely pulled from from New York Times and New York Post headlines. It’s a bit like a segment on some NY TV news stations where they read the latest headlines from local papers.

    Now, there isn’t anything unethical about quoting and paraphrasing – it’s all squarely in the realm of fair use. These blogs are a bit like suckerfish that attach themselves to whales or sharks in that they benefit immensely from their hosts. Well, actually, unlike suckerfish they repay the favor by driving traffic.

    In fact, I owe most of my readers to the low point in my blogging career, when after failing to submit my post about the Starbucks Siren to BoingBoing through their official black hole form, I begged Cory Doctorow to post it in a personal email. He did, I received tons of traffic and literally thousands of links from BB readers. Now that article shows up at the very top of Google search results for Starbucks logo.

    Therein lies a problem: good content on the Internet does not always bubble up to the top on it’s own. Blogosphere is a bit like the Black Sea, which has a layer of very active and vibrant biosphere at low depths. But it’s very deep, and below 200 meters the depths are full of poisonous hydrogen sulfide, which luckily does not circulate very much (unless there’s a particularly strong storm). Think about digg.com or StackOverflow.com– at the top stuff circulates, gets upvoted and downvoted. But below, there’s a poisonous cesspool of Sturgeon’s Law’s 90 percent. And most of the time, new and worthwhile content starts not at the top, but at the bottom, or flutters briefly in above the mediocrity and the bad, does not get noticed and gets buried.

    Speaking of StackOverflow, Joel Spolsky and Jeff Atwood recently touched on the topic of blogging success in their excellent podcast. They were discussing Steve Yegge’s retirement from blogging, and tried to pinpoint what it meant to be a successful blogger. “Perhaps one metric of success is getting people you respect and admire to link to your writing in an organic, natural way (that is, without asking them to).” I am a miserable failure on this front. Sure, I have some high profile readers, but their link love is rare, while I’m not really below begging for links.

    Jason Kottke, an A-list blogger and a primo toothpick sampler, was reflecting on the monetary success. He likened business blogging to shining shoes: there might be some individuals who can get rich by running a chain of shoe shining stores (Jason Calacanis, Nick Denton), and maybe even some individual outstanding shoeshiners (Dooce) who can make a decent living, but for the majority of shoeshiners it’s not a very good career choice.

    I’ve read somewhere about my hometown’s “king of shoeshiners”, a very colorful character. He was the best shoeshiner Odessa has ever seen, famous and loved by all, but he died poor and miserable. On his monument there was a short quote: “life is waksa” (waksa is a Russian word for shoe polish with a connotation of something pitch-black).

    For me blogging takes a good deal of effort. In the immortal words of E.B. White “writing is never ‘fun’”. (White almost rejected an assignment to write an article that became the finest piece ever written about New York when an editor suggested that he might ‘have fun’). What makes blogging less fun for me is looking at server statistics, number of comments, ad revenue, and thinking about payoff and success. And feeling like that I maybe should have done something else with my time.

    My high school Economics teacher, Mr. Oster, taught me one very valuable concept: “opportunity cost“. Whenever you make a decision do something, you almost always pay the opportunity cost – the difference in value you might have gotten by doing something better. Oh, there could be hundreds of things that have a better payoff than not very successful blogging.

    I personally do not blog for money, and certainly don’t blog professionally (the ads on my site cover my hosting expenses). Well, not yet, anyway – I am preparing stuff for a commercial venture that I’ll soon announce. I blog in order to meet people (hanging out a Web 2.0 events and meetups would probably have been more productive), but mostly to get things out of my head. In that sense I’m a bit like Louise Bourgeois. I’ve recently seen an exhibition of her work, and I’m pretty sure that if she did not create all those sculptures and paintings, the inspiration for them (which must have been glipses of extra dimensions, cellular automata that drive our reality, and super disturbing things that can’t even be described) would have made her a raving lunatic and not a lucid and sane 97 year old woman that she is.

    I don’t really intend on changing the format of deadprogrammer.com – the intricate, long, winding, interconnected posts about obscure topics. I probably would have had a lot more success if I just kept a photo blog about New York City. If I’d just stick to one popular topic and posted every day – I know I would have attracted a lot more readers. Instead, I’m going to start a new, for-profit blog. You’ll hear about it soon. I think I should be able to make some shekels with my mad shoeshining skills. And while I agree with Mr. White about writing not being fun, the fund is in having written.

  • Three Firsts

    I always thought that the quote went “I’ll try anything once” and it was Andy Warhol who said it. Apparently the quote is “I’ll try anything once, twice if I like it, three times to make sure” and it belongs to Mae West.

    Living in New York, amongst other things, pushes you to try something for the first time ever almost every single day. Here are three things I recently tried for the first time, mostly under pressure from New York City.

    Religious

    Probably every New Yorker that looks even remotely Semitic in appearence has been repetidly asked “Are you Jewish?” by the Hasidim. If you answer yes, you’ll get a Billy Mays-worthy pitch to pray/light Shabbat candles/put on at Teffilin. I will admit to occasionally denying my membership in the tribe when in a hurry, but most of the time my answer is “a little bit”, followed by a firm sticking to plain cowardly agnosticism.

    Ever since I wrote a long and rambling post about Tefillins, I meant to put one on. So this one time, after being approached by a young Hasid in the Atlantic station passageway, and customarily declining his pamphlet, I accepted his halfhearted offer to help me lay Teffilin. He was particularly surprised – I don’t really think he gets to help a lot of people perform this particular mitzvah a lot.

    He produced a Tefillin set from a black shopping bag and a loaner kipah from a pocket, helped me put it on and say the necessary prayers right there on the BMT’s Atlantic Avenue platform, amongst the hustle and bustle of people and trains. It felt strange, yet somehow very comforting – performing this ritual in one of the most familiar places to me.

    He also gave me a pamphlet in which the Lubavitcher Rebbe explains to a computer science professor that Tefillin is a symbolic representation of computers. He was very glad to be able to accomplish such an epic mitzvah.

    Culinary

    I was walking through Union Square farmers’ market, already having sampled and bought a package of organic bacon hawked by an upstate hippie (I’m not a very observant Jew as you might have already noticed in this post). I was passing by a little stall providing free samples of wine made by hippies somewhere upstate. That bit of hippie bacon called for some wine, but I did not want to fight the mob of greedy Manhattan housewifes for a tiny sip, but then I heard a magical phrase – “We also have dandelion wine”.

    I never really finished reading “Dandelion Wine” by Ray Bradbury, and always thought that there was no such thing – how can you make wine out of bitter yellow flowers? Apparently this is how. I bought a bottle. All I have to say is that dandelion wine tastes just like they say it should: like summer and childhood. I also bought some salad corn shoots that I’ve read about in New York Times. Those tasted like raw corn kernels.

    Automotive

    You know that nobody really drives in New York because there are too many cars there. I’ve spent many happy car-less years here, but the arrival of a baby forced me to buy a car (a minivan, in fact). Parking is a very sore topic around these parts. I was never willing to splurge on a garage, and had to subject myself to the indignities of alternate side parking regulations.

    There’s a whole book about parking in NYC – Calvin Trillin’s brilliant Tepper Isn’t Going Out. I bought it only because I have a friend named Tepper, but ended up immensely enjoying it. Which is what I can’t say about parking in the street.

    Well, recently, I finally broke down and shelled out $200 for a spot in a garage. The feeling on the “alternate” days is rather novel – hey, I don’t need to move a car! Also new – not worrying about what those loud teenagers are probably doing to my poor car, or if there’s a used car window repair place that just received a shipment of my car’s specific windows (did you notice how they always have a used window for your car ready, no matter how obscure, when you go to a nearest car window shop for a mysteriously shattered one?). It’s a new and pleasant feeling.

    What about you?