Someone at AMEE pointed out to me that there’s been a flurry of activity around so-called “Web Hooks” when I referred to the concept. This is quite heartening as I thought of this a couple of years ago and implemented this in Smartmessages early last year! I call them callbacks, but the idea is the same – it’s essentially a distributed observer pattern. I couldn’t figure out why nobody seemed to understand what I was on about… When I get some interesting event (e.g. a message open, mailshot completion, clickthrough etc), I ping a user-supplied URL with the appropriate event data, pretty much the one-liner that Jeff alludes to. The reason we do it is that sync with external systems (usually CRM) is something that were always running into, and there seems to be no sensible, generic way of dealing with it other than this, so I’m surprised it has not been discussed in this context before.
There’s one downside as far as I can see – it is highly dependent on the receiver to be able to handle the event in a timely fashion. This isn’t an issue if you’re connecting say, Yahoo! to Google, but it could be a big problem if you connect Google to your WordPress blog… My experience of CRM systems is that they are simply too slow to cope with the high rates of traffic that we are likely to generate, for example, if we point a stream of ~200 events per second at a CRM system, it will probably just bog down and fail (I’m thinking of the SalesForce API here which typically takes 1-2 sec to deal with a single SOAP API call). Retrying will only make this worse. I have two solutions for this: limit events to those that don’t happen so often (kind of lame!), or alternatively, use an outbound message queue to rate-limit the sending (Amazon SQS and Memcacheq spring to mind). Queueing works, but you lose some of the real-time aspect. Ideally clients would implement their own incoming queue in order to allow them to process events at their leisure, but this is mostly beyond the vast majority of web authors (or at least those that host the CRM systems that we hear from!).
Anyway, it’s nice to know that I’m not completely barking…
Scalable irony
This article on highscalability.com is a really excellent rundown of some of the options available for scaling a site to the heights of Digg. Ironically enough, at the time of writing the highscalabilitycom web server shows this error:
Unable to connect to database server
The MySQL error was: User highscal_admin already has more than ‘max_user_connections’ active connections.
I can point them at this really good article on how to avoid problems like this… oh, wait…
Hard Drive Story
So my MacBook was just sitting on a desk, idle. It was showing a screen saver or something. I woke it up and was greeted with a spinning beachball of death. This seemed to be an unusually persistent hang, so I killed the power and rebooted. I get a flashing system folder, no OS found. I reboot from a Leopard DVD and run disk utility. My HD does not even appear on the SATA bus. Around this time I notice a very quiet repeated clicking coming from the HD and I get a bad feeling about the dreaded “click of death“. I power off and reboot from an external drive, but still no sign of the internal one. It seems the drive is dead.
Now is the time to contemplate my backups. I have some recent enough Time Machine backups on an external drive, so I restore one of those onto the other external. That all works, but when I reboot into it and try to log in, I get an odd error about not being able to log in due to a FileVault problem, which I was indeed using. A bit of googling reveals the sad truth: Time Machine and FileVault do not play nicely together. Or more to the point, Time Machine just ignores everything protected by ‘FailVault’, that is to say if backs up everything except the bits that you actually want backed up. It turns out that it will back stuff up, but only if you log out. Hm. Logging out is for people that want to live without the instant-on we’ve come to love from Apple, and don’t mind waiting 5 minutes to for their 10 apps to launch on login, i.e. those that don’t have laptops.
So I extract the internal HD and ship it off to a data recovery company. After a day or so I get the bad news:
We regret to inform you that we have tried our very best to recover data from the faulty Hard Disk Drive you sent to us, unfortunately we were unable to recover any data.
Please accept our apologies for not bringing our services to your satisfaction, despite the fact that from a technical point of view, the damaged inflicted on HDD is beyond repair and it is absolutely impossible to recover any data as the HDD had suffered from severe internal mechanical failure accompanied by media damage, therefore the extreme nature of the damage made it impossible to recover any data.
Despite the usage of different components to get the HDD to spin, the internals were too damaged to read any data from the HDD. The effect of the media damage is immediate on the magnetic information stored on the drive, jeopardising the stored data files and the logical structures.
You can view the scratched area of the hard drive platter in the attached picture, it is the thick dark ring you see running around the inner part of the platter.
I’ve attached that very picture and you can see there is some fairly obvious mechanical damage to the drive – a great (for want of a better word) example of a head crash. The chances of a 50Gb encrypted volume file surviving that intact are pretty slim, as they say.
I’m pretty surprised that a modern, state-of-the-art hard drive can suffer spontaneous catastrophic damage like this without having experienced any physical shock. For those that want to know (so you know what to avoid), the drive was a 160Gb Seagate Momentus 5400.3, a bit under 2 years old. It’s still under guarantee, but a fat lot of use that is now.
I need to reconsider my backup and encryption options…
eBay’s new page design
I took a look at eBay’s new page design, which they are currently previewing. Verdict: needs lots more work. Continue reading “eBay’s new page design”