Pet Pixels

Basic Linux-HA heartbeat and pound config

marcus@bointon.com (Marcus Bointon) — Wed, 02 Dec 2009 17:05:36 +0000

I've had trouble in the past setting up Linux-HA a.k.a heartbeat to create a redundant front-end for a web service. The HA documentation is quite thorough and detailed, but it's lacking realistic examples to get a working system up and running. This is a quick guide to creating a 2-way cluster with one or more floating IP addresses and automatic failover on Ubuntu 9.04. I also include a quick config for pound, a reverse proxy / load balancer.

Continue reading "Basic Linux-HA heartbeat and pound config"

MySQL backups with Percona's XtraBackup

marcus@bointon.com (Marcus Bointon) — Fri, 11 Sep 2009 12:18:00 +0100

MySQL backup is sometimes very hard to do effectively. MySQL provides various options for backup, but many of them are simply unsuitable for large systems, particularly if they need to remain active during backups. Percona's XtraBackup is an open-source clone of InnoBase's InnoDB Hot Backup utility. So what makes XtraBackup a better solution, and how does it work?

Update: on December 10th 2009, Percona released Xtrabackup 1.0.

Continue reading "MySQL backups with Percona's XtraBackup"

Google Charts API Simple and Extended Encoders in PHP

marcus@bointon.com (Marcus Bointon) — Fri, 14 Aug 2009 09:34:27 +0100

Google's charting API has been around for quite a while now, but I've only just needed to actually look at it. It became immediately obvious that I needed a PHP encoding function, so off to google I went. Though I found several implementations, they were all incomplete or deficient in one way or another (and it didn't help that there was an error in google's extended encoding docs), so I've written my own based on several different ones. Both simple and extended encoders support automatic scaling, inflated maximum and lower-bound truncation, so you can pretty much stuff whatever data you like in, with no particular regard for pre-scaling and you'll get a usable result out. They have an identical interface, so you can use either encoding interchangeably according to the output resolution you need (contrary to popular belief, the encoding to use has very little to do with the range of values you need to graph). By default, the full range of possible values is used as it just seems silly not to. I deliberately omit the 's:' and 'e:' prefixes so that you can call these functions for multiple data series, and I include a function that does just that. You still need to generate your own URLs and other formatting, but that's a different problem. Read on for the code...
Continue reading "Google Charts API Simple and Extended Encoders in PHP"

Why isn't UK voting anonymous?

marcus@bointon.com (Marcus Bointon) — Thu, 04 Jun 2009 10:59:27 +0100

I voted in the European election today. I was pre-registered and had received my voting card. I went to the polling station where they looked at my card, checked it against my name on the electroal roll, crossed it out, and noted its number on a list next to the unique ID of the ballot paper. I made my choice and put the slip in the ballot box. Much of this process is necessary in order to combat fraud and ballot rigging, however, it's clearly not anonymous - a simple reference back from the paper's ID to my voting card ID means that the government knows what I voted. Knowing what I voted gives no anti-fraud advantage I can see, and means that the collected data is way more sensitive than it needs to be. The simple fact that this knowledge exists is likely to distort voting patterns - people might be afraid of their political leanings being exposed. I'm not particularly worried about that personally, but those with more extreme views (like those that vote for the BNP) might not want this known, and it undermines the absolute right to freedom of association. Political affiliation is regarded as the highest form of 'personal data' under the data protection act, along with ethnicity, sexual preference, medical records etc, and the best way to avoid problems with this kind of data is not to keep it in the first place, as the numerous leaks/losses/exposures of government information have highlighted. So why is it done? Aren't elections meant to be done by secret ballot? Did I miss something? Why aren't more people disturbed by this?

Subversion 1.5 repository upgrade on SourceForge

marcus@bointon.com (Marcus Bointon) — Tue, 05 May 2009 11:19:04 +0100

I've just had a slightly tricky time upgrading a subversion repository on sourceforge. They have recently added support for subversion 1.5 at the server end. 1.5 brings major new features for merging, but as it's not backward compatible with older subversion clients, the upgrade is not done automatically. SF have also done a major rearrangement of their documentation while transferring everything to Trac, and it's not always easy to get the right info. Normally to upgrade a subversion repo, you just run the 'svnadmin upgrade /path/to/repo', however, it's not quite so simple on sourceforge as you don't have direct access to the repo, and the instructions they give are slightly wrong at the time of writing. You're likely to get an error like this (it's not obvious that this is a fatal error) when you reload a dump file:

svnadmin: File already exists: filesystem '/svnroot/projectname/db', transaction '443-0', path 'tags' \* adding path : tags ...

This is because load is intended to add files to an existing repo, not to replace those that are already there, so you need to wipe the repo and start from scratch. So, here is a working command sequence that needs to be run from a project login shell on sourceforge (it applies to the project you're logged in through, substitute your project's name for projectname):

adminrepo --checkout svn svnadmin dump /svnroot/projectname > svn.dump rm -rf /svnroot/projectname/\* svnadmin create /svnroot/projectname svnadmin load /svnroot/projectname < svn.dump adminrepo --save svn

Yes, you do need to delete the whole thing and re-import it, but it's quick and easy, and you have a backup in the dump file you take at the start. After the upgrade, make sure you get a new checkout of your project to ensure that you're using 1.5 all the way through. Now you'll find that commands like 'svn merge --reintegrate' work.

Speaker & room calibration

marcus@bointon.com (Marcus Bointon) — Fri, 20 Mar 2009 18:45:54 +0000

I was lucky enough to pick up a Behringer Ultracurve Pro DSP8024 for a mere £50 on eBay recently. It turned out to have a buggy OS version (1.2), and Behringer very kindly sent me a replacement EPROM with a new 1.3 OS on it, which works just fine. I now have it installed between my Soundcraft mixer and my Wharfedale active monitors. I used its "Auto-Q" calibration routine and put up with some quite loud pink noise to calculate a room correction curve. Because it knows the spectrum it's generating, it assumes that what it gets back has been altered by the combination of speakers, room and microphone, so it can calculate an eq map to compensate for it. It's quite fun to watch as it has a nice big LCD screen to display the 31 1/3 octave bands - the initial spectrum is fairly peaky, but as it iteratively applies corrections you see (and hear) it flattening out. It's also very obvious that my monitors don't put out much below 50Hz (it analyses down to 20Hz), but that's to be expected from a moderately sized box with a 6.5" driver. The results are really pretty good, sounds lovely and smooth, but the real surprise is when you've been listening to it for a while and you switch out the EQ - it's really quite a shock to hear the uncorrected version. Lots of purists don't like room correction by EQ, saying it's better to fix the room in the first place, and also that EQ calculated like this is highly dependent on the listening position (which it is). I have a lots of bookshelves facing my speakers; they make fantastic diffusers, and I have some Universal Acoustics absorber tiles on the sloping ceiling above my listening position. The longest room mode will be fairly undamped (I'm not about to start hanging duvets around the walls!), but the resulting EQ is below 6db in either direction across the whole range - I've heard of rooms with 24db peaks! Anyway, after all that, it sounds lovely, and I'm happy!

The web developer's holy vhost trinity

marcus@bointon.com (Marcus Bointon) — Fri, 20 Mar 2009 12:11:12 +0000

When you're developing web stuff, working with projects in path names (i.e. not at the top level of a domain) can be difficult (gets in the way of absolute links, rewrite rules etc), so you often need to set up a local apache virtual host, stick an entry in DNS and create an SSL certificate before you can get on with the serious business of doing some real work. This can get to be a drag when you do it a lot, but there is an extremely elegant solution that means you'll never have to do it again...
Continue reading "The web developer's holy vhost trinity"

Ginger and Honey Snaps

marcus@bointon.com (Marcus Bointon) — Mon, 09 Mar 2009 13:50:15 +0000

I had this peculiarly specific desire to make some biscuits that were crunchy rather than chewy. Nigella didn't seem to have anything that fitted the bill, but I found this recipe in Rachel Allen's "Bake" book that I got for Christmas, and have had good results with so far. When rolling out the biscuits (definitely the best part) I thought they looked oddly like small potatoes. Next time I won't squash them quite so much so that they stay a bit rounder. While I was baking there was an enormous rainstorm, really quite spectacular and unexpected as it had been quite sunny all day. They came out brilliantly (if I say so myself), a lovely crisp mixture of sweet and spicy, exactly what I was after. As usual, the main problem with making biscuits is that I eat them, which isn't terribly good for me. I did send some round to Sandeep and Pam though, and they are proving handy for bribing Zoë.

Changing MacBook Bezel

marcus@bointon.com (Marcus Bointon) — Mon, 09 Mar 2009 13:15:11 +0000

I recently had cause to change the bezel on my rev A 13" MacBook. If you follow the guide on MacFixit, it is a long and complicated procedure, involving dismantling nearly everything. Luckily, most of it is unnecessary - it's really quite easy to simply remove the bezel without dismantling anything; just jump straight to step 35 and carry on from there.

Web Hooks, Callbacks and Distributed Observers

marcus@bointon.com (Marcus Bointon) — Wed, 04 Mar 2009 14:44:37 +0000

Someone at AMEE pointed out to me that there's been a flurry of activity around so-called "Web Hooks" when I referred to the concept. This is quite heartening as I thought of this a couple of years ago and implemented this in Smartmessages early last year! I call them callbacks, but the idea is the same - it's essentially a distributed observer pattern. I couldn't figure out why nobody seemed to understand what I was on about... When I get some interesting event (e.g. a message open, mailshot completion, clickthrough etc), I ping a user-supplied URL with the appropriate event data, pretty much the one-liner that Jeff alludes to. The reason we do it is that sync with external systems (usually CRM) is something that were always running into, and there seems to be no sensible, generic way of dealing with it other than this, so I'm surprised it has not been discussed in this context before. There's one downside as far as I can see - it is highly dependent on the receiver to be able to handle the event in a timely fashion. This isn't an issue if you're connecting say, Yahoo! to Google, but it could be a big problem if you connect Google to your Wordpress blog... My experience of CRM systems is that they are simply too slow to cope with the high rates of traffic that we are likely to generate, for example, if we point a stream of ~200 events per second at a CRM system, it will probably just bog down and fail (I'm thinking of the SalesForce API here which typically takes 1-2 sec to deal with a single SOAP API call). Retrying will only make this worse. I have two solutions for this: limit events to those that don't happen so often (kind of lame!), or alternatively, use an outbound message queue to rate-limit the sending (Amazon SQS and Memcacheq spring to mind). Queueing works, but you lose some of the real-time aspect. Ideally clients would implement their own incoming queue in order to allow them to process events at their leisure, but this is mostly beyond the vast majority of web authors (or at least those that host the CRM systems that we hear from!). Anyway, it's nice to know that I'm not completely barking...

PHP London Conference 2009

marcus@bointon.com (Marcus Bointon) — Mon, 02 Mar 2009 15:58:36 +0000

From this truly excellent conference, I took away some good memories, some new ideas and a nasty bout of conference flu. There's nothing quite like being in close proximity to a few hundred people to really spread things around... Highlight for me was Aral Balkan's keynote. It's always nice to see someone showing plain enthusiasm, and I couldn't agree more with him about the "lost magic" of computing. Had a chat with him afterwards about AMEE and other things. He also seems to have put together some odd but dull things that I had noticed a need for - EU VAT codings and ISO language references as web services! I didn't really enjoy David Soria Parra's talk on sharding. It all came across as very negative and many of the ways of doing it and coping with the fallout were not really discussed. No mention of MySQL 5.1's partitioning (which is limited, but is at least a start), or more radical approaches like Sequoia. David Axmark's talk on Drizzle was more interesting than I expected, nice to see effort being put into this direction. Microsoft really does seem to be trying a bit harder these days - their CSS test suite for IE8 is very welcome, and the effort they are putting into PHP, apache and other projects benefits many people. It has to be said that while it's not a mainstream product, Surface is really pretty cool to play with. Chris Shiflett's talk was excellent too; his demos and examples were particularly good, and entertaining. The post-conference social was great fun, I met lots of nice new people. After our move to France I suspect it will be harder to get to events like this, so I should make the most of them while I can! I've had several ideas for talks that I'd like to do (I get sick of email sometimes!), so I guess I need to get a bit more proactive on actually submitting them to a call for papers.

Scalable irony

marcus@bointon.com (Marcus Bointon) — Mon, 02 Mar 2009 15:52:13 +0000

This article on highscalability.com is a really excellent rundown of some of the options available for scaling a site to the heights of Digg. Ironically enough, at the time of writing the highscalabilitycom web server shows this error:

Unable to connect to database server The MySQL error was: User highscal_admin already has more than 'max_user_connections' active connections.

I can point them at this really good article on how to avoid problems like this... oh, wait...

Mixers

marcus@bointon.com (Marcus Bointon) — Wed, 28 Jan 2009 16:07:37 +0000

I've had a little Behringer UB802 mixer for some time and found it very frustrating to use as a front-end to my computer system. There's nothing particular wrong with it (clean audio, good functions, simple, reliable), it's just not very well suited to the job, mainly because its routing is not flexible enough to be used for audio input at the same time as output. One big problem I had was that the mic I use for audio input (mainly for Skype) routed to my speakers too; a recipe for feedback and poor input level. In order to resolve all this, I'd been contemplating a Soundcraft Compact 10 as it seemed to have much better routing options. Last week I managed to pick up the smaller Compact 4 for a mere £30 on eBay (I figured I could live without the extra size and inputs for that price). What a difference! It achieves its wonders by having an additional "recording" mix buss. It also has separate routing for monitoring. Each channel has a button that when pressed, routes that input to the recording buss and removes it from the main mix. Similarly there is a monitor button that routes it to the monitor buss. if neither are pressed, it goes to the main mix, which you can conveniently route back to the monitor mix too. Multiple mix busses are par for the course on bigger mixers, but almost nobody does it for small mixers, yet there are tons of n:2 small mixers around that are used in this role, suggesting there are a lot of frustrated users that don't know there is a way out. As Soundcraft's manual says:

Why dont other manufacturers design consoles like this ? a) Because they are out of touch ? b) Because they are not very innovative ? c) Because they dont have the experience ? d) Because they dont listen to their users ? Who knows :-)

The upshot of all this is that I can route the microphone to the computer's audio inputs without having it also appearing on the mix bus going to the monitors. As far as I can tell, this routing flexibility makes the Soundcraft about the only small mixer that's actually designed for this role. Most seem to gloss over this routing problem, or not want to "confuse" users with the concept of an additional mix buss. As an added bonus, it has two headphone outputs that are independent of the mix output, so I can turn my speakers down without turning down my headphones. The Behringer tries to do this, but only by giving you main and monitor mix levels, but as far as I can see you never really need the main mix outputs, only the monitor mix. The only real workaround for simple n:2 mixers is to have a separate mixer for input and output, which is quite a reasonable proposition when you see the price of things like the Behringer Xenyx 502, but I'm much happier having it in one box. Behringer make bigger mixers that have more busses (I think the cheapest is the Xenyx 1222FX), but they are also bigger, more complex and more expensive - overkill for my application. It's also interesting to contrast the marketing. Behringer describes the 802 as having 8 inputs, which is technically true - 2 mono, 2 stereo, and a stereo return - but in reality that's only 5 independent inputs (total of 10 input channels). Soundcraft go the other way - the Compact 4 has 5 independent inputs at a push, but you can actually squeeze 8 channels into it in total, and you can actually get 16 channels into the Compact 10. British understatement at work? The Behringer is still a great little mixer, and I'll miss its diminutive size, the aux send, and a couple of extra inputs (which, now I'm firmly in software-synth land, isn't really a problem any more). Anyone want to buy my UB802?

Cookie frenzy

marcus@bointon.com (Marcus Bointon) — Wed, 10 Dec 2008 14:31:19 +0000

I've been baking some cookies for Christmas. Not your average tree decorations, I've been attempting to bake Alsatian "Bredele", which might be called "petit-fours" in French. These have traditionally been baked at Christmas since the mid 15th century in Alsace. I'm working from a French recipe book called "Bredele et Gâteaux de Noël" by Thierry Kappler. So far I've made (I'm using the Alsatian names as they are more interesting!):

Wiss un schwàrzi butter bredele (those chequered and spiral ones)
Vanille hernle (litttle vanilla crescents, no picture as they, um, got tasted a bit too much)
Spretz bredele (wiggly piped vanilla shapes)
Hernele (a.k.a. Fer à Cheval, i.e horseshoes)
Kokos Màkrone (coconut macaroons)
Spetzbuewe (home-made jammy dodgers!)

I've run into a few problems; the coconut macaroons were looking perfect until I added the coconut, which made the meringue separate horribly in the piping bag; the hernele dough was far too crumbly to roll, so I added an egg; the spretz bredele dough was far too stiff to pipe; I don't know if it's my oven, but nearly all the recipes turn out undercooked, so I'm adding on about 50% cooking time. I've still got a few more to do, but I'm pretty happy with how these are turning out; they certainly look nice!

Synths, synths, synths

marcus@bointon.com (Marcus Bointon) — Tue, 09 Dec 2008 00:05:41 +0000

Right now I'm inundated with synthesizers. It's a hobby of mine to buy up old synths, refurbish them as best I can, then sell them on. Just recently I got very lucky on a Gumtree ad and picked up 4 in various states of disrepair: a Korg DS-8, Korg DW-8000, Roland JV-80 and a Roland D-70.
Continue reading "Synths, synths, synths"