The Capacitor Plague

I woke up from a nap to a loud pop and a smell of burning plastic. The source turned out to be one of the most precious and important to me digital devices: a ReadyNAS NV+, a small silver box with over a terabyte of hard drives that store my backups, music, and photos.

Network attached storage (NAS) is an engineering compromise. It’s a storage solution that lets you keep a bunch of drives in a self-contained device. It’s redundant: you can lose a drive (which is a statistical certainty) and not lose your data. There are also handy usb ports that let you connect usb drives and a button to run backup jobs onto these drives. It also serves as a print server, and in theory it can be used as a streaming media server. On the other hand it’s slow (gigabit networks are not fast enough when you need gigs of data fast), a complete nightmare to use with photo managers like Picasa, and an even worse nightmare if you want to use it as a Time Capsule.

I’ve spent a lot of time babysitting my ReadyNAS NV+: changing the defective RAM that it shipped with, updating the buggy firmware, finding the right drives for it (some don’t have the right temperature sensors). Don’t get me started on what it took to make it work with Mac’s Time Machine.

And after all that, the one box that was supposed to keep my precious digital archives safe was smoking. This was preceeded by a few days of weird performance issues and a couple of hangs. The power supply finally died a horrible death, and I realized that once again I was falling victim (or “mugu” as Nigerians say) to faulty capacitors.

According to Wikipedia, the name of this phenomenon is “Capacitor Plague“. There is an epidemic of failure in electrolytic capacitors from certain shady manufacturers. Electrolytic capacitors are usually found in power supplies. They are little aluminum cylinders filled with special film and electrolytic liquid or gel. Power supplies get very hot, and the liquid part of the capacitors, the electrolyte, always wants to either dry up or explode. The formula for the electrolyte is very hard to get right.

The rumor is that one or a few companies resorted to industrial espionage to steal electrolyte formulations. They weren’t entirely successful – they either got an incomplete formula or just plain Brawndo.

Spectrum Online did some digging:

“According to the source, a scientist stole the formula for an electrolyte from his employer in Japan and began using it himself at the Chinese branch of a Taiwanese electrolyte manufacturer. He or his colleagues then sold the formula to an electrolyte maker in Taiwan, which began producing it for Taiwanese and possibly other capacitor firms. Unfortunately, the formula as sold was incomplete.
“It didn’t have the right additives,” says Dennis Zogbi, publisher of Passive Component Industry magazine (Cary, N.C.), which broke the story last fall. According to Zogbi’s sources, the capacitors made from the formula become unstable when charged, generating hydrogen gas, bursting, and letting the electrolyte leak onto the circuit board. Zogbi cites tests by Japanese manufacturers that indicate the capacitor’s lifetimes are half or less of the 4000 hours of continuous ripple current they are rated for.”

Wastefulness of today’s society masks the problem: most people don’t perform autopsies on their dead $70 DVD players or $500 computers, they just use that as an excuse to buy the new hottness. The techies with (or without) spare time and soldering skills do the following: fill bulleten boards with tales of saving their devices by soldering in new capacitors; search for instructions on how to solder and purchase capacitors; and curse creatively after doing it for the 5th time.

The unique thing about the capacitor plague is how easy it is to identify: the capacitors literally blow their tops, venting electrolyte through the special stress relief indentations. It’s also unique in that anybody with a soldering iron has a very good chance of fixing it: the caps are easy to locate and solder. In the age when most electronic components are of the “surface mount” type (the size of a sesame seed) or chips with dozens legs as fine as silk, soldering in a two legged capacitor is very refreshing.

Here’s a nest of capacitors from my busted power supply: two in the left corner are clearly popped, the one on the right is probably ok:

In the last couple of years the following devices that I own fell prey to faulty caps: a cheap off-brand dvd player, a speed control on my Dodge Caravan’s air conditioner, a Netgear network hub, a huge and expensive Air King window fan, and now, my ReadyNAS. The interesting thing is that the problem exists in both high end and low end products, as well as in high tech and low tech ones (I did not know there were electronic components in the window fan).

I am out of warranty on my ReadyNAS because I bought it in May of 07. The following passage leads me to believe that the shitty capacitors are a problem that they are aware of and (maybe) fixed in newer releases of the hardware (they could not offer a 5 year warranty if they used the same capacitors – they’d just go broke).

“Please be aware that ReadyNAS purchased prior to August 21, 2007 carries a one-year limited warranty. Extended warranty purchased for these ReadyNAS will be honored by NETGEAR. ReadyNAS NV+ and 1100 purchased August 21, 2007 and later have a 5-year limited warranty, and the ReadyNAS Duo has a 3-year warranty.”

The brand name of the popped capacitors reads “Fuhjyyu”. It lead me to the an urban dictionary entry that says that Fuhjyyu is either

“1) Chinese word for feces.

or

(2) Brand name of abysmal quality capacitors that are installed on logic boards, switching power supplies and various other electronic components.”

There’s also a post from a guy who implores ReadyNas to stop using those capacitors.

Then there’s badcaps.net – a global capacitor gripefest that is too depressing to read.

You can see a nice gallery of busted caps over here

There are broader implications of this: coupled with the fragile lead free solder, leaky capacitors don’t only cause kajillions of dollars of damage, but will also make electronics of our era impossible to use in the near future. The aluminum in burnable cds and dvds are rotting too, destroying the record of our time.

Optimus Mini Three Full Review

I once read a book called “The Mouse Driver Chronicles: The True-Life Adventures of Two First-Time Entrepreneurs” about two guys who started their own company with an unorthodox business plan: making a real, physical product and selling it. This happened during the dot com era, when everyone was making money hand over fist with immaterial products: websites, content, synergy and such. At the very most, bits and bytes would overlap with physical world in online stores – you could order something online and get it delivered, but there were very few companies that produced an actual product. Well, there was one place that would send a real dog turd to a recipient of your choice, but they went under and I can’t even find their website. “Brick and Mortar,” a metaphor for physical world (as opposed to online) stores became a pejorative.

So, these two guys embarked on creating a company with a single product: a mouse that is shaped like golf driver and selling it to big novelty stores and catalogs. Very not dot-com. Designing the product was easy: take a golf driver head, slap mouse buttons on it – there it is. Dealing with the manufacturer was a lot more difficult. These days most of electronic products are made in China, and flying there is pretty expensive. You have to deal with the language barrier, timezone shift, and cultural differences, while collaborating on making a physical product. A single miscommunication and a whole batch worth tens of thousands of dollars might be ruined.

This is the reason why so many great product ideas go unrealized. A great example of that is SiliconFilm: a film roll-sized device that would convert your regular SLR into a digital one. Many people would want to buy one, but it’s been a stady winner of Wired’s vaporware awards.

Mousedriver was a simple product: a stock mouse in a slightly different housing. When I’ve heard that Art. Lebedev studio was actually planning to make one an OLED custom input device, Optimus Mini Three, I had my doubts that it would ever become real, but plunked down 100 bucks (a special pre-order price, it’s $160 or so now) and was prepared to get my money back in a year or two. Instead, in less than half a year I got a parcel from Taiwan. Inside was a working Optimus Mini keyboard. A was dumbstruck.

Now, Art. Lebedev Studio is a slightly more serious outfit than the mousedriver guys. It’s a large (about 150 people) design firm lead by a brilliant designer Artemy Lebedev. This guy:

Artemy (aka Tema aka Art.) Lebedev is so notorious in Russian web design scene, that he goes by the moniker “Youknowwho.” The Studio is based in Moscow, with satellite offices in Latvia and Ukraine. Web design is their bread and butter, but lately they’ve been branching out into industrial design. Starting with a funky coffee mug called ColorShift Atmark, they’ve been steadily building their portfolio of actually manufactured objects.

There are only two other design companies that excite me as much, IDEO and Frog Design. Lebedev Studio in my mind is destined to be as great as IDEO. One day I found out that my favorite toothpaste tube(Crest Neat Squeeze) and toothbrush (the “fat” Oral-B one) were both designed by IDEO, as well as many other wonderful things. Good design is very important to me, and Art. Lebedev Studio is finally starting to come out with things that I can buy.

The concept design for a keyboard with buttons containing little OLED screens called Optimus recieved a lot press coverage. Lebedev would be crazy to attempt manufacturing something complicated like that, just as it would have been stupid to attempt to create Apollo spacecraft without building a Redstone rocket first. So Youknowwho decided to do a proof of concept – a three button OLED “keyboard” and called it Optimus Mini Three. As I mentioned before, mine arrived from Taiwan a short time ago.

I opened the box, plugged in the USB cable, installed the software and was up and running in about a minute. USB devices are supposed to be plug and play, but can be very finicky – refuse to be recognized, fail to install drivers, etc. Wasn’t the case with OM3 – the usb communication code is rock solid.

One thing you should know about the organic led screens is that they are extremely hard to photograph. They behave kind of like the old tv screens and computer monitors, having some sort of a refresh scan. Ideally, I would take pictures in a professional light tent with a camera on a tripod taking a lot of pictures with a slow shutter speed. I ended up taking a lot of pictures hand holding the camera with the exposure of 1/13th of a second. My lens has an anti-shake feature and I have very steady hands, but the pictures could be a touch sharper if I used a tripod. There are also problems with moire pattern that shows up in pictures, but is of course not visible to the human eye.

Here I loaded some sample images from the web. They look very crisp live, but even with all the camera artifacts, they look passable photographed. The contrast and resolution is very impressive. Also, the plastic that covers the screens gives off very little glare and does not hold fingerprints well. The only thing that shows up is light-colored specks of dust. I did not even bother wiping down the buttons after pressing them for the most part.

Here’s an example of the silly slot machine game that comes with the software.

Here’s a closeup of a single button. There is a bit of an issue with the pixels right at the top and bottom: they are a bit off.

The memory and processor resource tracking application shows a \little artifact: a small stripe of dead pixels. This seems to be a software issue though: these pixels work in other applets and images, and actually show up in the software preview. In resource monitor mode the software itself takes about 10% of processor time for itself. This can probably improved, but this is not much more than I would expect from any resource monitoring application. Overall the “configurator” software is in a pretty solid beta. It does not crash, but certain features need some work: the Windows Media Player widget keeps scrolling “Winamp” messages, time and weather widget does not change the weather and is hard to read, etc.

I could not resist opening the this thing up. I pried up the non-skid rubber and found two screws. Mr. Lebedev has a habit of leaving funny notes in code comments of most of his websites. I was looking for a message, or the design team’s signature on the inside of the case (like with the original Apple Macintosh), but did not find anything.

The keyboard is not completely silent – when in operation it generates a very faint buzzing sound and the buttons are slightly warm to the touch. I thought that there might a little fan inside, but did not find one.

The keyboard looks like it’s made out of metal, but it’s actually very high quality plastic. To make it more hefty, the designers used two strips of metal as wedges that hold the pcb in place.

I did not take apart the screen assembly, but unlike most electronic devices these days, the mini keyboard seems to have been designed with future service in mind. Replacing the screens is easily within the abilities of do-it-yourselfers. As you can see, the key mechanism is the soft, rather than clicky one. This is a matter of preference, of course, but a click, as a feedback mechanism would be nice.

A lot of message board nerds cry – “unlike the full keyboard it’s useless!” The usefulness of mini three is in the software, of course. Even in it’s present state with one minute of spare time I came up with at least one cool use for it – a three webcam viewer. The picture is worse than the original, but the idea is that the left button shows Mount Fuji (at the moment under the veil of the night), the middle one – the Empire State Building and the right one – a live seahore webcam that lets me know what the weather is at my favorite fishing hole. I haven’t played with the SDK kit that is available yet, but it seems to be super nice. “What can I do with this?” – whine some on digg and reddit message boards. A hacker’s reaction would be different – “what can’t I do with this!”

160 bucks seems like a lot of money for a three button keyboard. I’ll admit, it’s not for everyone. The thing is, people spend a lot more on stupid case modifications and other blinkenlghts. And this well-designed gem of an accessory would be the coolest thing as far as the eye can see in your personal hell of a cubefarm. It’s a little hackable art piece. Practical? Not very. Cool? You bet your squeedly-spootch. Is it a good deal? Well, a much lamer lcd display goes for 80 bucks. A very neat laser keyboard goes for 180. Shiny is expensive.

I have one gripe with the design. The keyboard is made so that it will lie flat. I’d like to have a little wedge, so I’d be able to see it at an angle (I think I’ll make one myself). What I do like, is that the screens are easily rotatable in software, so you can just lay it vertically.

Of course, it’s a bad idea to program Optimus Mini Three to trigger self destruct, as it’s not cat-proof. But I think I’ll program a cat toy sequence in it though. So far the plastic stood up very well to cat claws.

For sticking with this lengthy and rambling post, I’ll let you take a little glimpse in my geeky life. Here’s a snapshot of a part of my table, including my fancy input accessories, for which, as you might have noticed, I have a bit of a weakness.