Geeks often want to release some of the code they write at work as open source. But often they don't really have any good idea why they should do it, other than that it feels like the right thing to do. Well, here's a damned good reason for you, and what's more, it's one that your boss will like.
Just under two weeks ago, I released some of my code. I didn't really expect anyone else to be particularly interested in it, but releasing it couldn't do any harm. But a few days later, someone submitted a bug report. What's more, he included a patch, and regression tests.
If I hadn't released my code, it would still have bugs in it, which would no doubt bite me in the arse later and figuring out what was going wrong would waste a lot of my time. By releasing my code, I got someone else to test and fix my code for free.
So now, when your boss asks you to justify releasing the code you've written for him, point him at this post, and tell him "open source bugs get fixed by other people, for free".
The most recent version of the code in question, including the bugfix, can be found here.
Posted at 20:01:13
by David Cantrell keywords: geeky | work
This optimisation might not matter for small values of n, but for n = 1000000 it really makes a big difference. This got me thinking whether there was a similar trick for calculating the product of all numbers from 1 to n - that is, n-factorial, or n!.
This matters even more for n! because it gets so large so quickly that computers are unable to accurately represent the intermediate values. In fact, 20! is the largest that can be respresented on a 64 bit machine. 21! can only be approximated, as you need to use a floating point number. This means that not only is calculating really big factorials time-consuming, it's not possible at all to do it accurately with native datatypes.
I don't even know if there is a simple re-statement of n! like there is for the sum above, but I'm gonna spend a few idle minutes working on it.
Posted at 20:10:32
by David Cantrell keywords: geeky | maths
I want per-user resolv.conf files. I've mentioned this in a couple of places and got asked why. It's so that I can have a consistent set of aliases everywhere I go (so I can, eg, type ssh sdf instead of ssh sverige.freeshell.org) and also so that the resolver will fill in my choice of domain on the end of hostnames, so I can be lazy and type ssh plough instead of ssh plough.barnyard.co.uk.
Some quick googling didn't turn up anything usefully cross-platform, but the implementation shouldn't be too hard - just set LD_LIBRARY_PATH or LD_PRELOAD and provide my own resolver library. Unfortunately, the resolver is part of libc so I need to figure out how to:
persuade arbitrary binaries to load an extra library;
have that library effectively replace bits of libc ...
... even when the functions I'm replacing are called by other functions in libc
Damn, I might have to read about how the linker actually works if I'm ever to do this.
Posted at 19:10:58
by David Cantrell keywords: geeky | unix
I've decided that using different sized text for the tag cloud works better than using different colours, but it has the drawback of eating a lot of screen space. I need to find an algorithm to pack the text in more efficiently so that it doesn't waste so much vertical space.
Actually, I already have an algorithm to do it in my head. Unfortunately it's, umm, rather inefficient. In fact I think it's O(N!) which would be fine if I only had 10 tags, but I have 52 so far, and am still occasionally adding tags.
52! is roughly 8e69. That's 8 followed by 69 zeroes. 69, dude!
Of course, this is a variant on the rectangular packing problem, which is itself a variant of the knapsack problem, which is NP-complete, so I'm going to have to come up with a heuristic that will return a reasonable (but not optimal) solution quickly.
I've decided that the best heuristic is to ask for pointers to code that other people have written that will do the job for me :-)
My constraints are that I need to fit an arbitrary number of rectangles of arbitrary size into a rectangle of fixed width but whose height can vary as necessary, with minimum wasted space. And I'd prefer a perl or javascript solution.
Use both sides of the paper. No conferring. You may begin now.
Posted at 23:44:09
by David Cantrell keywords: geeky | meta
I'd like to express my warm thanks to the lovely people at Yahoo and in particular to their bot-herders. Until quite recently, their web-crawling bots had most irritatingly obeyed robot exclusion rules in the robots.txt file that I have on CPANdeps. But in the last couple of weeks they've got rid of that niggling little exclusion so now they're indexing all of the CPAN's dependencies through my site! And for the benefit of their important customers, they're doing it nice and quickly - a request every few seconds instead of the pedestrian once every few minutes that gentler bots use.
Unfortunately, because generating a dependency tree takes more time than they were allowing between requests, they were filling up my process table, and all my memory, and eating all the CPU, and the only way to get back into the machine was by power-cycling it. So it is with the deepest of regrets that I have had to exclude them.
Whenever someone grumbles online that they've lost some data, I always smugly suggest that they restore from their backups. After all, all of my stuff is backed up. I even test my backups because I also smugly tell people that "if you've not tested them you don't have backups, you have hopes".
Well, I just found out the hard way that I'd forgotten to add one of my remote shell accounts to my daily backups. Oops. You may now all point and laugh.
Posted at 01:20:39
by David Cantrell keywords: geeky | rsnapshot
Today I got a copy of "Haskell: the Craft of Functional Programming" in the post, which, while it's been on my wish list for ages, I'd not got round to actually buying. So - many thanks to whoever got it for me, it is most appreciated.
Posted at 13:59:25
by David Cantrell keywords: geeky | meta
I am most grateful to Geeklawyer for volunteering to beta-test the new version of OS X for me. Evri fule knoe that the first release of a piece of software is a steaming pile of shit, so *I* will wait at least another few weeks to make sure there's no horror stories before I buy it myself.
Posted at 12:36:59
by David Cantrell keywords: geeky
According to a Windows programmer of my acquaintance who, to spare his blushes, will remain anonymous here:
" The recognised way of communicating with a POP3 server to retrieve mail is to create an FTP connection and use that connection to send text commands to the POP3 server to download responses and messages. "
The gentleman in question runs a company which "Develops stand alone or networked database applications, inventory databases, CRM systems, and help desk dall (sic) logging."
With developers like that, it's no wonder that the Windows platform has software of such low quality.
Posted at 18:20:16
by David Cantrell keywords: geeky | silly
My Lightning Talk on cpandeps went down really well, although as José pointed out, I need to fix it to take account of File::Copy being broken. I also need to talk to Domm after the conference is over to see if I can get dependency information from CPANTS as well as from META.yml files.
There were lots of other good lightning talks. Dmitri Karasik's regexes for doing OCR, Juerd Waalboer's Unicode::Semantics, and Renée Bäcker's Win32::GuiTest were especially noteworthy.
Richard Foley's brief intro to the perl debugger was also useful. Unfortunately Hakim Cassimally's talk was about debugging web applications, which I'd not noticed on the schedule, so I didn't stay for that.
And finally, Mark Fowler's grumble about why perl sucks (and what to do about it) had a few interesting little things in it. I am having vaguely sick ideas about mixing some of that up with an MJD-stylee parser.
At the auction I paid €250 to have the Danish organisers of next year's YAPC::Europe wear the Swedish flag on their foreheads. This, I should point out, was Greg's idea. I would never be so evil on my own.
Posted at 17:15:36
by David Cantrell keywords: geeky | perl | yapc
A day of not many talks, but lots of cool stuff. Damian was his usual crazy self, and MJD's talk on building parsers was really good. Although I probably won't use those techniques at work as functional programming seems to scare people.
The conference dinner at a Heuriger on the outskirts of Vienna was great. The orga-punks had hired a small fleet of buses to get us there and back, and one of the sponsors laid on a great buffet. The local wine was pretty damned fine too, and then the evening de-generated into Schnapps, with toasts to Her Majesty, to her splendid navy, and to The Village People.
It wasn't all debauchery in the evening though - on the bus, I had a very useful chat with Philippe about Net::Proxy, and re-designing it to make it easier to create new connectors for it.
Posted at 11:57:06
by David Cantrell keywords: geeky | perl | yapc
As is becoming normal, I used the times between talks to bugfix some of my modules - this time Tie::STDOUT and Data::Transactional. The former was failing on perl 5.6, the latter on 5.9.5. The former was a bug in perl (you can't localise tied filehandles and expect the tieing to go away in 5.6, so it now declares a dependency on 5.8), the latter was a bug in my code.
Philippe Bruhat's talk on Net::Proxy was great - you can tell it's great because I came away with ideas for at least four things that I need to write. First up will be a plugin for it to allow the user to specify minimum and maximum permitted data rates for proxied connections. This will permit bandwidth limits for maximum permitted rates, but will also help to defeat IDSes doing traffic analysis if you specify a minimum permitted data rate.
This will protect (eg) ssh sessions from being identified based on their very bursty traffic pattern, by "filling in the blanks" with junk data.
In the evening, the CPAN-testers BOF was productive.
Posted at 23:28:23
by David Cantrell keywords: geeky | perl | yapc
As a service to module authors, here is a tool to show a module's pre-requisites and the test results from the CPAN testers. So before you rely on something working as a pre-requisite for your code, have a look to see how reliable it and its dependencies are.
Posted at 20:22:55
by David Cantrell keywords: geeky | hacking | perl
To make up for a disappointing gap in Palm's software for the Treo smartphone, I wrote a small perl script to parse the database that stores my call history. I then re-wrote it as a re-useable module which also figgers out whether the call was incoming or outgoing.
I got irritated at how hard it was to use Wikipedia on my Treo. There's so much rubbish splattered around their pages that it Just Doesn't Work on such a small screen. Given that no alternatives seemed to be available - at least, Google couldn't find any - I decided to write my own Wikipedia handheld proxy.
It strips away all the useless rubbish that normally surrounds Wikipedia pages, as well as things like the editing functions which are also hard to use on portable devices. Internally, it's implemented using perl, LWP, and mod_perl, and is hosted by Keyweb.de.
I'm going to Vienna by train for YAPC::Europe. If you want to join me you'll need to book in advance, and probably quite some way in advance as some of these trains apparently get fully booked.
arr
dep
date
Waterloo
1740
Fri 24 Aug
Paris Nord
2117
Paris Est
2245
Munich
0859
0928
Sat 25 Aug
Vienna
1335
The first two legs of that are second class, cos first wasn't available on Eurostar (being a Friday evening it's one of the commuter Eurostars and gets booked up months and months in advance) and was way too spendy on the sleeper to Munich. Upgrading to first class from Munich to Vienna is cheap, so I have.
Coming back it's first class all the way cos upgrading was nearly free ...
arr
dep
date
Vienna
0930
Fri 31 Aug
Zurich
1820
Zurich
1402
Sun 2 Sep
Paris Est
1834
Paris Nord
2013
Waterloo
2159
Don't even think about trying to book online or over the phone, or at the Eurostar ticket office at Waterloo. Your best bet is to go to the Rail Europe shop on Picadilly, opposite the Royal Academy and next to Fortnums.
I contributed part of a chapter to this book, and so I got a free copy. I was expecting to take it home, put it on the shelf, and never use it. Today, less than 48 hours after getting the book in the post, I had to use it. The thoughtful comments and excellent description of how dump / restore work prevented me from looking like a complete tit on a public mailing list. I therefore recommend this book.
More seriously, it does look jolly good, covering just about all the backupish stuff that I've heard of and lots that I haven't. But more importantly, it devotes lots of space to restoring your backups - complete with step-by-step instructions for "bare metal" recovery - and talks about things to do when your backups are broken.
And it covers things that lots of admins don't like to think about, like Exchange and MySQL (and other databases; judging from a quick skim of the Oracle section I expect the coverage to be good).
Buy a copy of this book for your friendly local sysadmin. He will love you for ever.
Another tiny new Bryar feature - the CGI front-end has sprouted an extra method 'is_mobile' which you can call to figger out whether the user is on a mobile device or not. It's available in your templates like this:
I'm looking for a combination of GPS hardware and software that I can use with my Treo 680. I'm sure that one of the many products I've looked at will do the job, but their websites are ever so unhelpful.
I need to be able to plan waypoints in advance, record waypoints as I go, and have my position displayed on a moving map. I will supply the map files myself as bitmaps in any reasonable format. When I am near the edge of a map, the software should merge image segments from up to four map files. Anything that requires Windows is defined as broken.
I do *not* need automatic route finding, although I suppose it would make sense to be able to add seperate software for that later.
Posted at 12:39:45
by David Cantrell keywords: geeky | gps | palm
There's a new release, version 1.58, of Number::Phone, my set of perl modules for picking information out of phone numbers. Changes from the previous release are that Mayotte, Reunion and Comoros can't decide which country is which, and there's the usual updates to the database of UK numbers, mostly to support the new 03 numbers.
Posted at 00:34:36
by David Cantrell keywords: geeky | perl | telecoms
I just added support for specifying keywords (which the cool kids call "tags") in your Bryar postings. Please see if you can break things while I test them on this journal.
Please note that I've not yet added keywords to my old postings. That will take quite a while to do, so please be patient. At the time of writing, keywords 'bryar' and 'whisky' should do vaguely useful things.
You can even subscribe to keyword RSS feeds, like this.
This evening, I wrote the following in response to some trouble someone was having with their mail on one of my mailing lists, one which is normally devoted to talk about electronic circuits. In the end I didn't post it, which is probably a good thing.
XXXX XXXXXXXXX wrote:
> So far the only person who has made any sense to me has been XXXX
> XXXXXX. The other postings have answered in, to me, computer
> gobblydegook. Definitely not 'admirable suggestions of how the
> individual issues can be resolved'.
In all other areas, people take the time to figure out how to do things properly, and learn the terminology needed so that they can discuss any problems they have with an expert without wasting that expert's precious time. It's just plain good manners for someone to learn what some of the standard openings are called before asking a chess expert for tips on how to improve their game, or the basics of what a resistor does before asking questions in this 'ere mailing list.
The same applies to computers and email. The "gobbledigook" is the basics that people would learn in any other field of expertise. Why people can't be bothered to learn it when a computer is involved is beyond me.
Yes, I know it takes time to learn. I know it's annoying to have to do it. But to the experts, your problem is as insignificant and uninteresting as my inept chess-playing is to my local club champion.
Anyway, the solution to your email problems is "consult your local system administrator or your ISP's help desk". If they can't help you, fire them, hire a better one and repeat the process until the problem is solved. No gobbledigook required.
Gosh, Evil Dave showing restraint. Whatever next?!!?
Posted at 22:04:13
by David Cantrell keywords: geeky | rant
My trusty old 6310i is giving up the ghost so I need a new phone. And I might as well combine it with a PDA. Being a long-term Palm user I'm inclined to go for the Treo 680 (or whatever is the studliest Palm OS version is - not Windows because I'm not stupid).
Specific questions I'd like answers to, and which I can't find on Palm's webshite, are:
Can the mail client talk imap(s) and smtp, or is it crippled by being MS-ware only?
Can the web browser do the Google Maps dance?
How zoomy is the interwebnet connection?
Is there a decent ssh client and terminal emulator?
What's battery life like when (and when not) using it as a phone?
And of course, is it any good and where should I get it from?
Your time starts now, please use both sides of the paper.
Posted at 19:09:34
by David Cantrell keywords: geeky | palm | phone
You're a retarded buggy undocumented piece of shit. Please fuck off and die slowly and painfully, impaled on a rusty shit-smeared spike.
The same goes for the utter fuckwits who designed you. I hope their families die too so they can't spread whatever defective genes spawned a moron like you.
Hugs n Glasgow kisses
Posted at 21:01:31
by David Cantrell keywords: geeky | rant
For the last couple of days, my wireless network has been rather unreliable and I've been trying to figger out why. It turns out that an Apple Airport base station which flidded out a couple of years ago, and which I never unplugged because it was buried amidst a tangle of other cables, has spontaneously risen from the grave and is now working.
It was breaking shit because it was on the same IP address and announcing the same SSID as the machine that had replaced it as my wireless router.
How odd.
(If you object to my use of 'flid' above, please read 'spaz' instead)
Posted at 23:28:33
by David Cantrell keywords: geeky | weird
Yesterday, my file server crashed badly. I hit the per-user process limit, so couldn't log in remotely, couldn't log in on the console - and so couldn't do a graceful shutdown. So I cycled the power. When it came back up, one of the filesystems was a bit buggered, as expected. But it was a bit more buggered than I first thought and couldn't be recovered. So I mkfsed it, adding journalling while I was at it, and it is now restoring from the last backup.
Recently, OFCOM asked for remarks about number portability. My response is here (PDF). The free software that I refer to in that page is my Number::Phone::UK module on the CPAN, which you can see in action here.
Posted at 20:49:02
by David Cantrell keywords: geeky | telecoms
I have released my XML::Tiny module. The parser at its core is less than twenty lines of code. Pretty easy to follow code too, I think, and that also includes error handling. One of my aims in writing it was to keep memory usage and code to the absolute minimum, so it doesn't handle all of XML. The documentation says that it supports "a useful subset of XML". Personally, I think it supports the useful subset. It's certainly enough to parse the data I get back from Amazon when I use their web services, and to parse an RSS feed.
Posted at 21:55:43
by David Cantrell keywords: geeky | hacking | perl
As I've been promising to do for ages (it's over a year since Simon said I could take over maintenance of it, and several months since I promised Bob that he could have all the shiny new features I had planned) I've finally got round to releasing a new version of Bryar, the software what provides the brains behind this 'ere august journal.
The most important change is that I released the stuff for filtering out comment spam.
In the digital age, accuracy and speedy calculations come from using powers of two. You also want to stick with integers to avoid rounding error. Therefore I propose that we measure speed in nano-furlongs per mega-fortnight. The conversion from metres per second is fairly simple - 1 m/s is approximately 264 nFurlong/Mfortnight.
This means that we can measure incredibly slow speeds with great precision - down to of the order of 10-19 m/s without having to enter the dangerous world of floating point. And of course we can easily measure all the way up to the fastest possible speed, that of light, which is only about 292 nF/MF. This provides ample room for further expansion of the measuring stick if Mr. Einstein turns out to have been wrong. A 128-bit value would let us measure speeds up to 1010 times that of light, in extremely small increments.
Posted at 21:10:18
by David Cantrell keywords: geeky | silly
You're a bunch of cunts. Not only do you provide no way for me to export my wishlist to XML or CSV or anything useful like that, you manage to break all the third-party applications that do it as well. I hate you and hope you die.
update: they had for no apparent reason disabled my account, although they were still sending me information about how to use it. It would have been nice if they'd told me. It would have been nice if I got helpful error messages back from the Net::Amazon module too.
Posted at 17:30:49
by David Cantrell keywords: amazon | geeky | rant
Oh dear. Read this and this. I think the Debian developers should:
grow up
realise that they are the only people in the whole world to care about the Debian Social Contract
I'll certainly not be installing their brand of madness again. These days, Ubuntu is sufficiently mature.
And while I'm venting my spleen (what on earth is a spleen, how do they vent, and do they taste nice?) Debian should sodding well fix their stupid release names. What the fuck is 'sid' or 'buttplug' or whatever they call it? What's wrong with a nice simple '5.7'?
Posted at 23:09:19
by David Cantrell keywords: geeky | linux | rant
Most of you will not be aware that I run a mirror at home of recent episodes of The Linux Link Tech Show. Whenever there's a new episode, or they get slashdotted, my DSL gets hammered which means that I get a really sucky slow connection for anything I want to do to the rest of the world.
So a few weeks ago I installed and configured mod_bandwidth. It didn't appear to work and I left it until later to fix it. I have now fixed it, by the simple expedient of turning it on by HUPping Apache.
Doh!
I've not imposed much of a restriction - the tllts mirror can eat up to about 97% of what's available, leaving just enough for me to do important things like ssh to this 'ere web sewer of mine and witter in my journal.
Posted at 10:24:26
by David Cantrell keywords: geeky | web
Oh dear. I just got email from Paypal (and yes, it really is from Paypal - I, unlike most people, know how to check it out properly) advising me about how to protect myself from being ripped off by fraudsters sending spam emails which merely claim to be from Paypal but which actually direct you off elsewhere so they can steal your Paypal username and password. This practice is commonly known as "phishing".
Trouble is, to an unsophisticated eye, phishers' emails look just like Paypal's real emails. What Paypal should do is simply never email their customers except in direct response to the customer doing something on their site, such as sending someone some money. That way, less technologically-literate customers can simply ignore all unexpected mails "from Paypal" and be safe.
That behaviour is good enough for my bank, so I wonder why Paypal don't do it.
And before anyone mutters about what would happen if someone sends me money (which I obviously want to know about) - the person sending it should tell me. And I'll probably make an exception for when I'm selling something through ebay too.
Posted at 22:15:27
by David Cantrell keywords: geeky | spam
Today I was at precisely 0°0'0". Greenwich is cool. I yelled very loudly "THE WORLD IS MEASURED FROM HERE". Aside from the museums, it is teeming with fine pubs, at which I got very precisely tipsy.
Look on the right! The list of all my archived posts has been broken down by year. By default you'll see archived posts from the current year, clicky-clicky on the numbers at the top to get previous years.
It's all done with CSS and Javascript, so I only needed to fiddle with the templates and not with the application that drives the journal. And of course it degrades gracefully and everything is still available in non-CSS non-Javascript browsers. I even tested it in lynx. Hooray!
I've also slimmed the page down considerably by using a named style for journal entry links instead of embedding style info in the page for every link, and for archived entries have reduced the length of their links considerably using the <BASE HREF> tag. That gave a weight reduction of something like 40%.
Posted at 02:20:36
by David Cantrell keywords: geeky | meta
There were quite a few interesting talks in the morning, especially Ivor's one on packaging perl applications. Oh, and mine about rsnapshot, of course, in which people laughed at the right places and I judged the length of it just right, finishing with a couple of minutes left for questions.
At the traditional end-of-YAPC auction, I avoided spending my usual stupid amounts of money on stupid things, which was nice. Obviously the hundred quid I put in to buying the hair style of next year's organisers wasn't stupid. Oh no. Definitely not.
In the last month, there have been well over 400 attempts at spamming this journal. All have failed. And yet the spammers still try. And I get email notifying me each time, because there's always a possibility that a legitimate comment might get classified as spam and need to be manually approved.
Ah well, I have the IP for each of those 400-odd spams, and using routeviews.org I can easily turn them into a considerably shorter list of netblocks. And then auto-create a shitload of Deny from rules. 104 of them, to be precise. It will be interesting to see if the spammers notice their lack of access and keep trying.
Posted at 00:21:56
by David Cantrell keywords: geeky | meta | spam
