Google – Dead Programmer's Cafe

May 21, 2012June 3, 2012

On Facebook Advertising

Two things happened recently: Facebook’s IPO fizzled and there was a slew of articles and blog posts about how Facebook ads are ineffective. The majority of the articles follow the same song and dance: we spent $250, got some likes from random people and nothing changed. The one standout it GM dropping Facebook ads. GM’s budget was 10 million dollars, and the way I understand it, this is their equivalent of $250 for a small business.

While I personally dislike Facebook tremendously and want it to fail, I think that the FB ad-bashing articles are off base. Facebook ads are a great deal. I am not an advertising professional, but I drank with a lot of them and I think picked up a reasonable amount of knowledge about the industry. I also stayed at … hmm, can’t remember the name of the hotel, but this is a reference to the TV ads where people say “well, I’m not a [professional that has to be very smart and skilled] but I stayed at [name of the hotel]”.

There are many, many bad deals in advertising. There are big-name websites that have “respectability” plus a huge saleseforce. These guys can charge $15, $20, $30, sometimes even $50 per 1000 banner impressions. cost for 1000 impressions is called CPM: cost per M where M is the Roman numeral 100. At the higher price you get gigantic custom ads known by a variety of names: browser crashers, godzillas, superskyscrapers, page fuckers, etc. There are also “sponsorships” where dumb companies buy little badges on new sites with next to no traffic.

How do these things get sold? Well, the agency people who control how the ad budget is spent are often young and get a lot of free drinks. Also, ad sales people are very good at what they do.

Then there are cut rate ad networks that have 5-20 cents CPM. The traffic for these comes from all kind of low quality sites: lyrics websites, guitar tab sites, all kinds of web farms, dating sites, porn and near-porn sites and the like. Big sites sometimes buy traffic from these sources when they don’t have enough “inventory”, but they’ve already sold a lot of high CPM ads. These are a pretty bad deal, and I suspect much of the traffic is simulated by bots. It is cheap and there’s a lot of it though. Targeting options are pretty weak with the exception of plentyoffish.com: POF allows for very detailed targeting. If you want to target only Canadian redheads between the ages of 18-22 – you can.

Google ads are a better deal: you are targeting people that are searching for something. Google has pretty sophisticated algorithms for detecting fraud, but I suspect click fraud is still rampant. Google’s predominant model is charging per click instead of per 1K impressions, and you have to compete with other people for hot search keywords. For instance, ambulance chasers pay ridiculous money for “mesothelioma” keyword. Mesothelioma is a type of cancer caused by asbestos, and layers apparently can make crazy money suing on behalf of people who have it. A click on an ad can fetch as much as $10 or more. There are many other expensive keywords where even a single click is worth paying actual humans to click on them from time to time. There are also pity clicks and punitive clicks: sometimes people click ads to support sites that they like and to punish ones advertisers they don’t like.

Facebook ads by my estimation cost about 1/10th of Google’s, almost in the low quality network territory. The targeting is amazing though: Facebook knows a lot about what people are interested in and lets you have a sniper-like precision of putting your ad in front of them. Do you want New Yorkers who are into planted aquariums, fishing and knife sharpening? If there are 10 people like that you can reach them on Facebook for a few bucks. Unlike Google users who are searching for something, Facebookers are there for ogling hot people and playing Farmville. They tend to ignore the ads, but you can get an even better deal by buying CPC ads. Massive click fraud is difficult on Facebook, most of these clicks are real.

There are many things about advertising in general that are mysterious: starting with the “if I almost never click on ads then who does” to “do tv and magazine ads actually make me buy anything”. Are funny ads where you remember the joke but not what was advertised worth the money? Are people swayed by car commercials? How about those Coca Cola billboards? How about those ads in New York taxi cabs and ads that annoy in general – are they effective? QR codes ( http://wtfqrcodes.com/ ) – are they the advertising equivalent of “free public wifi” zombies? I have a hunch, but I don’t really know.

What I do know is that compared to other options FB advertising is a pretty good deal at current pricing.

September 13, 2011September 13, 2011

On the Importance of Lube

As a response to meaningless discussions about microformats, standards, widgets and other unimportant web gunk I wrote an article Deadprogrammer’s Hierarchy of Web Needs. The gist of that post is that what matters the most is text and images, and that the importance of everything else above it falls in geometric progression. Things high on the pyramid get too much consideration.

There is one modifier that does not fit on the pyramid: lubrication. You see, there’s a lot of friction associated with putting content online. It’s a major limiting factor to the growth of the internet. Those who focus on the base of the pyramid and apply enough lube succeed.

Twitter succeeded because it is the ultimate lube, the equivalent of a major dose of oil-based laxative. It lets you put little pooplets of thought at the speed of diarrhea. Text alone is enough – it’s the very base of the pyramid. Because of that people forgive Twitter the url shortening pandemic – the very thing that is poisoning the exchange of links, the terrible handling of images, and the procrustean shortening of the information that you can share.

My ideal twitter feed is kind of like now defunct memepool.com, but with inline images. I want good copy, I want good images and I want good links (and not the terrible shortened crap – this is not what hypertext is about).

Besides lube, there is stuff that seems like a good idea, but is actually adding friction. The days of black backgrounds and blinking text are behind us, but the new enemies of eyeballs are more subtle. Hashmarks in Twitter are terrible. I can’t read shit like “ugh, bad #weather in #hoboken #today #firstworldproblems”. Another thing that acts as sand in my eyes is “winerlinks“. They are little hashmarks that let you link to every paragraph in the story (which is a great idea), but at the same time they look like a bunch of bedbugs and scrape your eye with every saccade.

The year is 2011 and we are walking with supercomputers attached to digital cameras more powerful than the ones that went into space probes. Yet sharing an image is still a huge pain in the ass. It just takes too many steps. Iphone apps do it relatively well, even if too many people mangle their perfectly good pictures with a totally un-fun “a fun & quirky way to share your life with friends through a series of pictures” (whatever that means).

Here’s Vannevar Bush talking about “memex trails” in “As We May Think“:

“The owner of the memex, let us say, is interested in the origin and properties of the bow and arrow. Specifically he is studying why the short Turkish bow was apparently superior to the English long bow in the skirmishes of the Crusades. He has dozens of possibly pertinent books and articles in his memex. First he runs through an encyclopedia, finds an interesting but sketchy article, leaves it projected. Next, in a history, he finds another pertinent item, and ties the two together. Thus he goes, building a trail of many items. Occasionally he inserts a comment of his own, either linking it into the main trail or joining it by a side trail to a particular item. When it becomes evident that the elastic properties of available materials had a great deal to do with the bow, he branches off on a side trail which takes him through textbooks on elasticity and tables of physical constants. He inserts a page of longhand analysis of his own. Thus he builds a trail of his interest through the maze of materials available to him.

And his trails do not fade. Several years later, his talk with a friend turns to the queer ways in which a people resist innovations, even of vital interest. He has an example, in the fact that the outraged Europeans still failed to adopt the Turkish bow. In fact he has a trail on it. A touch brings up the code book. Tapping a few keys projects the head of the trail. A lever runs through it at will, stopping at interesting items, going off on side excursions. It is an interesting trail, pertinent to the discussion. So he sets a reproducer in action, photographs the whole trail out, and passes it to his friend for insertion in his own memex, there to be linked into the more general trail.”

Stinging together these trails is still too cumbersome. Have you ever tried to post a picture of two items (what JWZ calls “ exhibit A, exibit B” (this particular link leads to a collection that is totally worth your time)? What if there are no pictures of these items on the internet and you have to scan or photograph it, upload it, crop it, post it? And what if like Vannevar Bush’s bow and arrow researcher you’d like to add a comment in longhand, your own handwriting. Or how about a little hand-drawn diagram? This simple task will likely take at least half an hour.

But enough bellyaching. It’s 2011, and the flying cars are almost here. There’s Skitch and Evernote (Phil Libin seems to be making the dream of Memex a reality in a less lame way than anyone else). And as an alternative to Twitter there is Google+ – I can drag an image from Skitch into a text area and it automatically uploads! When they’ll open up the API doing A/B posts will become finally possible there. Please, please leave the suffocating, hashtag strewn stinkhole that Twitter became. Join Google+. I’ll be hanging out there.

June 30, 2011June 30, 2011

I, for one, welcome our new social overlords

Disclamer: I thought that Google Wave was an excellent idea, so you can safely disregard my blathering here.

Here’s what I’m picturing in my head: Google has approached Facebook and Twitter on the playground. Twitter stole a piece of Facebooks lunch, but can’t really hold onto it. After a few threats and a bit of running around and a few ineptly thrown punches Google got itself into position to really clean Facebook’s clock and take its lunch. Foursquare and Groupon which earlier evaded Google’s punches in the most ebarracing for Google way possible are likely to be lunchless later. It is rather strange that Google does not go after scrawny TV Guide and White Pages – it looks like their lunches are not that tasty.

Yes, it’s just another social network. Yes, Google has a track record of failing fast and frequently (which if I remember correctly is a “good thing”). But remember, a bunch of incompetent coders received such an applause, press coverage and a whole evem some money to build a Facebook alternative. And finally mighty Skynet is doing the same thing. I think the company behind the mighty Skynet and the future parent of our robotic overlords has a chance against a bunch of compiled spaghetty PHP.

P.S. Zuckerberg and his approach to privacy creeps me out, so I have deleted my Facebook account and turned it into a blank account used only for work (writing Facebook apps, testing and such). I’m completely fed up with the character limit on Twitter – it’s nothing more than a feed from my blog. But I do want to share photos, and I do want to post shorter, non-blog-worthy thoughts. I’m really rooting for Skynet here.

June 2, 2011

Gmail and tracking numbers

Hunting around for UPS, Fedex,USPS, and Japan Post tracking numbers in Gmail is no fun. I really wish there was a way to aggregate all the shipping numbers in a single Gmail plugin which would at a glance tell me where all the crap that I ordered is at any given moment. Google already knows how to tell a tracking number from all other strings, and there are apps for iOS that aggregate tracking (unfortunately you have to manually type in all the tracking numbers). A Gmail plugin that would keep track of tracking numbers would be great – maybe anyone with a bunch of mythical %20 percent time at Google will implement this…

May 19, 2011

Arbitrage

It boggles my mind to think that I grew up in a country where most private enterpreneurship was a criminal offence, a felony. It was like this: create a business, be scorned by your customers at best, and at worst get caught and go to a labor camp.

There were of course people engaged in small business that escaped persecution. One particular example stuck in my memory: my father once pointed out a disheveled man rooting around in books at our favorite second hand book store. The store accepted books on comission, with the book owner setting the price. The disheveled man, my father explained, did not work anywhere. He made his living from his encyclopedic knowledge of the Soviet book market. He picked underpriced books and relisted them at market prices. I did not know it back then, but this is a very common tactic called “arbitrage”. In the US it is employed by multitudes of people, from library sale scroungers as disheveled as that man, but armed with handheld computers and laser scanners hooked up to Amazon.com, to venture capitalists buying bad software companies from badly run companies and selling them to even worse run software companies at billions in profit.

In the US “Rich Dad, Poort Dad” author is making millions explaining the benefits of enterpreneurship over salaried proffesionalism, and I am in fact workin for not one, but two business magazines: Fast Company and Inc. I spent almost five years here, but it’s almost time for me to go. I did not line up the next job yet, but months ago I told my boss that I was leaving so that he could hire a replacement. My replacement is here, and I’m close to finishing knowledge transfer.

I have a few startup ideas, but what scares me is not the Soviet Militia, but the lack of affordable healthcare and the lack of a trusted technical co-founder. I am mulling taking another corporate job, and luckily Google and its ilk hoovered up web professionals, so the market looks promising.

For months I would tell myself that I would leave when the Freedom Tower would eclipse WTC 7 where I work. I’d say that time is near.

June 16, 2010June 16, 2010

Hunting Heads and Developing Back Ends

I’d like to share with you a few thoughts about tech recruiting. This is not a post about how to write resumes, read resumes, ask or answer tech interview questions. Enough is written on the subject by people who are better at all of this than me. No, it’s about that email that arrives in almost every developer’s inbox about “a full-time, permanent position with a very competitive salary” for “a very prestigious company located in mid/down/up-town” which is looking for “an alphabet soup of technologies”. Half the time it’s a phone call. These happen during the times of boom and bust in the tech sector, although with varying frequency.

Now, people who are not programmers would probably think that this is spammers or scammers who will ask for a fee. Well, there is a fee involved, but this is not scammers: every time there’s a reasonably well paying job involved. A huge number of programming positions gets filled this way.

There’s always a shortage of decent software developers, project managers, system administrators and allied tradespeople. This was true back when I was starting my career, in 1997, but it only got worse. First of all the cost of running a startup came down from hundreds of thousands of dollars to thousands, and many talented programmers with ideas, tolerance of Ramen-heavy diet, slumming, and a little risk exited the corporate workforce to build url shorteners and travel sites for hipsters. Top notch people with kids and/or love of brightly colored furniture and fridges stocked with fancy sodas were vacuumed up by Google, Microsoft, Facebook, Yahoo and other modern Xerox Parc-wannabees. The hedge funds sucked in all those who are good at implementing complicated formulas and algorithms in code that can make a Kessel run in 12 parsecs or less.

The biggies look for a healthy mix of specialists, but they can easily afford to hire jack-of-all trades hackers. Everybody else usually looks for a person who has proven experience with a certain “stack” – an alphabet soup of languages, operating systems, and servers. Sometimes stacks are ageless like COBOL, CICS, DB2 – even today typing it into dice.com brings up 155 results. Some come out of favor: LISP – only 16 results though. Somer are old standbys: Php MySQL Apache – 370 results: C# SQL Server IIS – 439 results. Some, like RoR are up and coming – 93 results.

If you are relatively young(cheaper that way) and have done some work with a fashionable stack your phone will be ringing weekly with recruiter calls in the dreariest days of a tech bubble burst. If you are a specialist who deals with a particularly gnarly piece of hot tech – it will ring a few times a week.

Now here we come to a somewhat interesting phenomenon. Most of the time the person calling is a “headhunter”. Here’s what’s happening:

Web department in company Webco Enterprises has an opening. It could be caused by a number of things, but most commonly

1) after gettig tired of incessant complaining of the VP of Web Things, the Bean Counter finally agreed to budget in Y dollars for another headcount.

2) an experienced headcount who was making Y dollars accepted an offer of Y + $30,000 + an office with a door from company iWeb 2.0 and left.

So here is VP of Web Things with a budget of Y dollars. She’s swamped with work. In the next team meeting she tells her nerds to ask all of their friends if they know somebody and offers a referral bonus. Next she puts together a typical job ad complete with technological alphabet soup and takes it to the Master of HR. The Master of HR posts it on the company intranet, on the corporate site and on monster.com, dice.com, etc. Next VPoWT goes to the same sites and looks for candidates who posted their resumes there. It’s mostly drek, and the one person who might be ok wastes a lot of her time because he has another offer for Y + $10,000 from another company.

Now more desperate, she starts using “headhunters”. They send a couple of somewhat crappy candidates, followed by one barely ok one, followed by one who’s good enough. After a little song and dance the good one accepts Y dollars per year minus – 15% that silently go to the recruiter. The next few years he’ll be getting modest raises and maybe even a bonus or two that will not break the budget: the Bean Counter approved Y dollars, and it stays in the budget like that, and the headhunter only gets paid once.

Now, here’s what was happening on the other end: headhunters start pounding searches into their computers. LinkedIn, all recruiting sites, internal databases, etc. They might not understand what those letters in the alphabet soup mean, but they sure can try to match them up with what comes back from searches. Then they start shooting in the dark – it’s easier to get people who are currently looking for jobs, but it’s the shallow end of the pool that VP of Web Things already explored somewhat. They start pinging people who are working at the moment – this is where the good heads are.

Finally someone hunts up the the right head for VPoWT’s headcount and pockets the 15% of a yearly salary of that head.

Now, here comes the interesting part: most of these headhunters who are calling are trying to line up publicly available job listings with people on LinkedIn. For instance, I recently received two emails about a rather awkwardly named position “Director of Back End Development” (second recruiter spelled it “Backend”).

I looked it up on one of the recruiting sites (I think dice.com), and it was an old and infamous pre-dot-com-boom company that chose such a unique job title for what I do for a living. It was literally the only one with that creative title. I could have applied directly, bypassing the 15% headhunter’s fee. This is the reason why headhunters rarely name the company name.

In the past, through my random and numerous connections I’ve heard stories about what the setup is like at that place, and it’s a doozy. I’m pretty sure that the person who will take that job will take a lot of development up his or her back end.

The moral of the story is this: it’s easy enough to bypass recruiters – they are just engaged in arbitrage: the hiring manager is busy, the potential headcount is not even looking. That 15% is OPM (other people’s money) to the hiring managers, but not so much to the headcounts.

February 11, 2010February 11, 2010

Deadsticking

Let me tell you about web development and fishing, my two great passions. Here’s a fishing story. When I was a kid, i fished off the long piers in the Black Sea. I did not catch all that much, and I mostly thought that was because of my lack of skill and resources: I though if I could cast further, have a better fishing rod, or be able to go out on a boat, maybe I could catch more. Then I noticed that one fisherman was catching huge quantities of fish.

He had an interesting technique. Instead of using a single rod and switching from a place to a place, he’s bring ten. Each one was cheap and simple bamboo rod. He’d bait them, and drop the hook in shallow water in clear water, where the sea floor was covered with concrete blocks with holes used to stabilize the sand. I tried fishing near those holes before, but never caught anything. He’d set up his ten rods, and then just wait. An interesting thing happened: after about an hour the fish started biting, and were mostly just catching themselves: all he had to do was walk from rod to a rod and take off the fish. Sometimes just a single hole would be producing, then he would take that rod and catch fish after fish from the same place.

This technique is called deadsticking: you leave the bait motionless, and thus exposed to the fish for much longer periods of time. Most fish grab the bait and run: you don’t even need to set the hook, the fish catches itself. When on the boat the same technique often works. Having a number of rods fishing all the time gives you two benefits: it shows you the hot spots and exposes your hooks to more fish.

I see this again and again: a company redesigns a website, changes the core technology used to build it, spends a lot of money, and then the traffic and search rankings fall, and thus revenues fall.

I am pretty sure I know the cause of this: broken links. Any redesign of a website of just about any complexity, especially when technology changes breaks a lot of links. Search engines are like fish: they do not like things moving from a place to a place in an unnatural manner. A fisherman once told me: hey, do you think a Tautog (a kind of fish) ever seen a dead fiddler crab jump three feet up and down? Fish do like movement, jiggling the bait often entices them to bite. But the important thing is, the jiggling can’t be too vigorous and take the bait out of the view! Google likes to see changing content, but if the location of the content darts around – you betcha boots you are going to see your Pagerank take a hit.

The best thing to do when faced with with less traffic from Google is not to redesign the site again, but to dead stick: fix all the broken links, keep the site stable, and better yet, bring in more rods – build more sites.

In my time I’ve seen a large number of websites and careers that were set back by CMS switches and redesigns.

Further reading: The Russian Tea Room Syndrome and Deadprogrammer’s Hierarchy of Web Needs.

January 27, 2010January 27, 2010

eBay and The Michigan Deposit Scam

eBay is such a horrible hassle these days. I tried selling a few things recently, and between the horrible UI, all the hassles with payments, answering questions and shipping it turned out to be a huge waste of time.

I am sitting on a small fortune of items I would like to get rid of, but I don’t want to deal with strangers on Craigslist or going through the eBay rigomarole. An ideal solution would have been an eBay drop-off shop, but it seems that these went the way of the Dodo.

eBay drop-off store is an idea that many have tried, but it turned out mostly like Seinfeld’s Michigan deposit scam.

In one episode Newman keeps trying to find a way to make a scheme that would bring New York cans and bottles to Michigan, which has a 10 cent deposit instead of New York’s 5 cent one. Kramer keeps telling him that it would not work due to the transportation overhead, but finally Newman figures out a way to get a postal truck for free.

It seems that the time overhead is so high on running an eBay store is so high, that most of the bigger ones that tried it went out of business.

In reality the Michigan deposit scam is against the law, but it actully costs the state 14 million a year in lost revenues. It’s doable.

eBay is showing Twitter-like incompetence in serving its customers. While Google gives its customers huge amounts of storage, email, and software for free, eBay can’t seem to provide free image galleries and other useful services, selling out its customers to an unsavory bunch of third party providers. Image storage is not a very difficult technical problem, and neither is url shortening, but eBay and Twitter are still in the dark about it.

Instead of making selling on eBay easy, developing drop off stores, and making its service better eBay seems to be focused on buying and selling unrelated busenesses for billions of dollars (and losing money on it).

November 17, 2009

8 Pieces of Architectural Advice for CMS

I have some advice for those in the business of building large websites with content management systems.

1) Do not implement search yourself.

Your CMS sucks at search, and so do you. I see this again and again and again. Everyone is implementing search on large websites instead of using Google. Developers are afraid of looking unprofessional. Managers are answer yes to the question “do you want advanced/faceted search” (the correct answer is no – user’s don’t like it and don’t use it). As a result a lot of resources (both server and developer) go into implementing something that Google is awesome at. Even some very smart people, like Jeff Atwood roll their own search, and their users end up going to google.com and typing “foo site:stackoverflow.com”.

Users are very happy with Google CSE, and don’t mind the text ads. Those text ads – well, that’s revenue that you would otherwise would not have, however small this is. If you absolutely can’t do Google CSE – buy their search appliance. If you can’t do that either – well, you better be using Solr.

2) Do not implement comments yourself (unless comments are what you do for a living).

It is extremely difficult to get comments right. Users absolutely abhor comments. Spammers – well, they love it. Luckily, you can just go and get DISQUS to do all the heavy lifting for you. The time saved on using DISQUS can be used on building something else, meanwhile users absolutely love leaving comments through it, while spammers hate it.

3) Physically separate your admin interface from the stuff that is going to be used by your users.

Maciej Ceglowski has some words of advice about not having your blog hacked: cache your output in flat files and hide the admin interface. The benefits of this are tremendous: cached files are fast and secure. You will need to do some fancy footwork to serve up parts that change a lot, but you can do it the same way DISQUS and Google CSE do it – through the magic of AJAX.

4) Sanity check: calculate the amount of RAM in the home computers of all of your interns. Compare that to the amount of RAM in your server farm. Who wins?

5) Use a CDN and/or caching proxy, don’t be cheap. These things will save your butt when Yahoo and Digg will come a-knocking at the same time. I’m not even going to mention Memcached – you can’t get big without it at all.

6) Fight WYSIWYG editors. These things are the worst. They are the Devil. They are a security hole. You never get what you see. People paste from Word. Do I need to go on?

The best middle of the road solution is something like Markdown.

Do not underestimate the user’s ability to learn a few simple rules. When I worked at TV Guide there was this movie database application. Very non-technical editors were using a very scary-looking Unix-based interface at an amazing speed. When I rewrote it as a web interface, it became more “user-friendly”, but they could not enter stuff as fast as before.

7) Make sure you have good backups

8) I know you won’t be able to follow my advice, I know I can’t either. Life is a constant compromise.