Since this box has two hard disks, and not being one to back down from a challenge, I eventually reinstalled it over the weekend with nothing – no install media, no reinstall robot or intelligent hands – just a reliable internet connection and a healthy dose of courage. Here’s how.

Target: reinstalled machine with the same network settings, ssh host keys, and other minor configuration ported. The disk layout is to be RAID-1 containing LVM, with separate /var volume and separate /boot partition, also RAID-1.

1. One disk in the box contained old data, so I cleared that out and wiped it (including the MBR for good measure) and partitioned it.
2. I set up a degraded RAID-1 array for a small /boot partition, a large RAID-1 array for the LVM and a swap partition.
3. I mounted the new partitions in the correct layout in /mnt and used debootstrap(8) to get a very basic root set up. I also bind-mounted /sys, /proc, /dev and /dev/pts for now, they can be done properly when the root is a bit more mature.
4. Next, I copied into the new root /etc/apt/sources.list and chroot(8)ed into it. Now I could apt-get update and tasksel install standard to get an almost fully-functional base system. At this point it is also sensible to install locales, tzdata and console-data and dpkg-reconfigure them, followed by mdadm and lvm2 if required and openssh-server so you can get back in after rebooting. Some or all of these may already be installed by tasksel.
5. Time to install a kernel before leaving the chroot: apt-get install linux-image-2.6, followed by grub-pc which should detect both installations and set up menu entries for them.
6. Back in the old system, I copied in the network, hosts, resolv and hostname configuration files, and set up /etc/fstab to my liking.
7. Install grub to both hard disks if it isn’t already so (dpkg-reconfigure grub-pc) and again check that it detects both installations and creates the right menu entries. At this stage, booting from either hard disk will allow the loading of either the new or old installations, which is exactly what we want. It’s now time to umount the new installation.
8. Now I followed the excellent guide for remote kernel upgrades at http://ariekanarie.nl, except in this case we are using the same method to try booting the new system and fall back to the old one if it’s a disaster.
9. Reboot and hope!

At this point I rebooted to find myself back in the old kernel, which was disappointing – this means the new kernel has panicked and rebooted, and grub has fallen back to the old system (exactly as planned). It turned out there was nothing in /dev at boot time, and udev doesn’t start early enough to populate it before panic. That’s easily solved by mounting the installation again and using MAKEDEV as a seed.

10. With a bit of luck, you’re now in the new installation and can dpkg-reconfigure grub-pc again to install grub to both hard disks again. This isn’t strictly necessary, but it records this choice in debconf so future upgrades will automatically upgrade the bootloader everywhere it’s needed.
11. Now I could do some tidying up, mount the old installation and copy over all the data I wanted, and after careful checking wipe the first disk clean ready to be added into the RAID arrays.
12. Finally, add the old disk to the RAID arrays so they are fully redundant.

Sources:
http://www.michael-hammer.at/server_config/debootstrap/
http://d-i.alioth.debian.org/tmp/en.i386/apds03.html
http://www.debian.org/releases/stable/amd64/apds03.html.en
https://wiki.archlinux.org/index.php/Convert_a_single_drive_system_to_RAID
http://ariekanarie.nl/archives/211/remote-kernel-upgrade-with-debianubuntu-and-grub2

In fact I’ve been using StartSSL certificates for about three years now, but I get them issued to Level 2 verification which incurs a fee. (It’s more expensive now than when I was first validated, but still good value.)

StartCom are the only issuer I’ve ever dealt with who work like this. They validate the individual, using:

• two forms of government ID
• third-party background checks
• telephone verification at a number of their choosing, based on the checks

This makes me trust them far more than other issuers, who don’t bother with any meaningful validation at all. Their approach is to establish identity, then allow you to:

• validate domains and issue as many certificates as you wish, valid for two years, including SAN and wildcard certificates
• validate email addresses and issue X.509 certificates in your name
• issue code signing and XMPP certificates
• undertake stringent Organisation Validation, and then issue certificates in a company name as well as an individual
• validate other individuals with a web-of-trust arrangement, like CACert
• undergo Extended Validation and issue EV certificates
• if you have an unspecified amount of money, become a private CA yourself

Although this doesn’t make up for trust (the presence of an SSL certificate doesn’t guarantee the data you send is safe upon arrival) it does make me much happier to see a CA taking proper verification measures instead of just handing out certificates at random – and it’s much cheaper for me too, being verified once and then issuing as many certificates as I need. Highly recommended.

*that is, more trustworthy

(for international readers: it’s the night of the U.K. census, which with a little imagination has the potential for all sorts of fun.)

“Thank you for your feedback regarding the security of our platform. We are constantly reviewing these processes and regard our members security as paramount, whilst ensuring our processes are navigable to the majority of the UK.   We have had the platform professionally penetration tested but your email demonstrates an excellent understanding of the challenges and we would welcome your suggestions on our options of improving the password reset process.

“We will be extending our SSL certificate to the publicly accessible  website and please be assured that this is held on a different architecture to that of the Member application.”

This is very promising!

“Our technology is built using some of the best and most secure tools in the industry. We have partnered with infrastructure providers who handle some of the most sensitive data in the UK (such as medical and financial records). Both the digital and physical security measures we have implemented are amongst the strongest available anywhere. This includes full encryption of all data at all times, full implementation of secure socket layers, security certificates and physical restriction of access to the data, our servers and our offices. Our systems have been fully penetration tested (that means we’ve asked people to try and break in).”

(There are other suitable assertions in various places – they even have a set of ‘principles’ about safeguarding data.)

Unfortunately, this promise is rather undermined in several ways – after noticing the first couple, I did a little digging to see what else was exploitable.

Here’s the final part of the joining process, where you choose a username and password combination:

The text I’ve cropped too eagerly says “Choosing a secure password is an essential part of protecting your personal information”, or thereabouts. I duly chose a complex password that fitted the requirements, and to my surprise it was rejected. I tried another, and it was rejected; then a third and a fourth. By trial and error I worked out what was going on:

1. The password must contain only the listed special characters, not just include one of them.

That’s a bit of a problem, because even assuming a basic ASCII set, 15 characters are unavailable to users; 80 are left, so that’s about a 15% fall in the available combinations*. Not a good start.

More concerning is the presence of a “security question” field. It’s used for resetting the password in the event losing it, but this technique for recovery has long been ridiculed – the shared secret is often common knowledge amongst friends, and sometimes (as in this case) the available questions are fairly easy for an attacker to find the answer to in public records or solicit from the victim without arousing much suspicion:

2. The security questions available include “First pet’s name” and worse, “First school name”.

It’s pretty pointless enforcing stringent password requirements, and then bypassing them with something so susceptible to a dictionary attack.

I was pleased to find that failing to log in to an account more than a few times results in a temporary lockout, which should deter casual brute force attacks. But I wanted to know how that security question would be used, so I ‘forgot’ my password and followed the links to reset it. Here’s the form:

Actually the first form, not shown here, initially just asks for a username, giving an error message if it isn’t registered, and here’s another problem:

3. The password reset process confirms the existence, or non-existence, of a given username – half the credentials required to log in – to any visitor.

I’d be prepared to take a bet that most users will choose “What was the name of your first school?” as a security question. The first pet you have is often at such a young age you can’t remember it clearly; the name of the street you grew up on might change a couple of times if you moved house. But first school I attended? I’ll never forget that, so it makes most sense to use as a ‘backup password’. It’s also the best one for an attacker to try and find out from public sources.

But that aside, as you can see the password is not generated at random and communicated to the real account holder out-of-band, in the manner of many other sites. Instead:

4. A new password is immediately set to a value already known by the attacker.

Once inside, an attacker can also change the security question or answer, or both, so you can’t even regain your account by telephoning the company – unless you can convince them you’re genuine, in which case the “security question” was a total waste of time anyway. I awarded some marks for notifying the user by email that the password has been changed, but immediately docked them again because – bingo! You’re now a victim of identity theft!

Let’s assume you’ve been locked out, the security question has been changed and you want your account back. ALLOW don’t let you telephone them; you either have to dig around and find an address to send an email, which we all know can be intercepted, or (and you’re encouraged to) contact them through a form on the site. You’ll probably include some personal details, because you want to convince them of your real identity; indeed, two of the options on the form are “I’ve got a question” and “Something doesn’t work”. I sent my findings through this very form, under the latter heading, and to my surprise:

5. Despite promises of “full encryption of all data at all times, full implementation of secure socket layers”, the contact form is transmitted to ALLOW in the clear, with no protection whatsoever.

So now anyone listening in your connection knows all about you too: your ISP, any of the peers along the route, the deep packet inspection advertisers if your ISP is less than reputable, and the neighbour who connects to your wireless and slips you a fiver every month for the privilege. Nice work, privacy specialists.

(For the record:

• I have sent these findings to ALLOW and await a response have a response.
• Yes, this scenario is unlikely, but it doesn’t fill me with confidence about the rest of their business activities.)

* please feel free to correct my maths. It was never my strongest subject.

Since a couple of years we’ve been handing off security issues of minor or
theoretical impact but for which a fix would be desirable at some point, like
certain classes of denial-of-service attacks, off to stable point updates.
We’re looking for a person that wants to coordinate this: monitor the Security
Tracker for issues classified as such by the Security Team, converse with
maintainers to get such updates done and coordinate with the stable release
managers on this.

I’m happy to confirm, now that it’s been announced, that I am that person: point release security co-ordinator.

Affected packages

If your package fulfils these criteria:

• it had a security problem reported in the past (that is, it was allocated a CVE number);
• it didn’t get a Debian Security Advisory, but it wasn’t marked “unimportant” in our tracker;
• it has been fixed in unstable, but the version in stable or oldstable is still vulnerable

it is a candidate for updating in stable or oldstable, and you’ll probably receive a mail from me at some point asking you to do so.

You can pre-empt this mail of course, by backporting your fix to the affected versions and contacting the release team to get your fix into stable, without waiting for me. In such a case, please drop me a note with the details so I can tick your off on my hit^W candidate list.

Making a stable/oldstable upload

This is documented in the Developer’s Reference, but to summarise:

1. Prepare your fix, targetting stable or oldstable, and build it in an up-to-date chroot for that release
2. Send a diff of the new package to the release team, asking for permission to upload
3. Upload as normal, and wait for it to be included in the next point release. Meanwhile, notify the security team of your upload, if it fixes a CVE.

Tracking candidate packages

I’m going to start off tracking filed bugs for SPU candidates and OSPU candidates with usertags in the BTS, under my own address. In time that might be merged into an address used by the security team, but for now I’m still finding a good workflow so it’s much easier this way.

• #607958 (apt): replied and tagged ‘moreinfo’; jmm later downgraded it to normal
• #606951 (nsca): agreed with the submitter and reverted the change, uploaded straight to unstable
• #605784 (nagios-statd): thanks to the great debugging work of the submitter, uploaded a fix to DELAYED/2 (giving the maintainer time to make his own planned upload)
• #598588 (json-glib): cannot reproduce; reduced severity and emailed the submitter

The remaining bugs are either removal candidates or no longer low-hanging-fruit, so I don’t expect to keep squashing very many more before Squeeze is released.