Backing up with rdiff-backup

I recently tried out rdiff-backup for my backups and really like it a lot. It is a command-line utility written in Python that can operate locally or remotely via SSH. When it is first run, it copies over all of your files to the backup directory. On subsequent backups, it only copies whatever has changed since the most recent backup, updates the mirror, and stores the changes it made to the mirror. The end result is that you always have a fully up-to-date mirror of your files, but at any point you can restore from previous backups. The backup directory consumes minimal disk space, and the backup process is very fast since it is only copying the changes to your computer.

 The syntax is similar to the "cp" command: the command itself, followed by the source directory, then the destination (backup) directory, like so:

     rdiff-backup /home/jizldrangs /usr/backups

When using SSH, use the server name, followed by double colons and the absolute path, like this:

     rdiff-backup /home/jizldrangs fileServerName::/home/jizldrangs/backups

As with many command-line tools, there are a lot of options, most importantly the option to include or exclude certain paths or files. See the examples and man page for details on how to fine-tune your backup.

I put this on my netbook and desktop, which are both running Ubuntu Maverick, but my wife's machine had only the sporadic backups I had made to our USB hard drive, and I wanted a more consistent plan. Fortunately, rdiff-backup, being a Python application, also has a Windows version. If you backing up to a local directory or mounted network drive, you are good to go.

If you want to back up over SSH, it gets a little sticky but it can be done. Using the instructions on this post I downloaded plink.exe (the command-line version of Putty), and created a batch file with the following:

"C:\Program Files\rdiff-backup.exe" -v5 --no-hard-links --exclude-symbolic-links --remote-schema "plink.exe -i rsa.ppk %%s rdiff-backup --server" "C:\\Documents and Settings\Mrs. Jizldrangs\My Documents" mrsjizldrangs@myfileserver::/home/mrsjizldrangs/backups-my-docs

This batch resides in the same directory as plink.exe, which is why the full path isn't specified. Here is a breakdown of the arguments:

• no-hard-links and exclude-symbolic-links: these are necessary for windows machines per the blog post above
• remote-schema: The method of contacting a remote server (in our case, an SSH server over plink.exe)
• The last two arguments are the source directory and destination (i.e. backup) directory

Plink has some arguments as well, here is the breakdown:

• i: the name of the ssh key to use for authentication. I created an SSH keypair using PuttyGen, which generates 2 files, a public key and a private key. I added the contents of the public key to the authorized_keys file on the server, and the argument specified above is the private key, which is also located in the same directory as plink.exe and the batch file
• The %%s tells rdiff-backup to run what follows on the remote server
• rdiff-backup --server: This is run on the remote machine and all it does is start rdiff-backup in server mode

All right! All of the machines are backed up and everything is peaches and cream. In fact, my newfound confidence gives me a case of Linux Distro Wanderlust, and I got it in my head that I wanted to switch my desktop from Ubuntu Maverick to Linux Mint 10. There's no reason not to, especially since I can just do an automated restore and all my files will come rushing back. So I wiped the disk, installed Mint 10, and began the restore. After a long time I received an error that permission was denied on ~/.gvfs. Fortunately, rdiff-backup allows you to include or exclude as many folders as you want with the --exclude argument. I excluded it, and tried again. I got that same error on ~/.local and ~/.subversion, so I ended up excluding those directories as well, with the final command looking like this:

     rdiff-backup -v5 --force -r now --exclude **/.subversion/** --exclude **/.gvfs/** --exclude **/.local/** myFileServer::/home/jizldrangs/vengeance-backup ~

Here's a breakdown of the arguments:

• v5: Verbosity level 5. The available levels are 1 being the lowest through 9 which outputs so much info that it is impossible to read. 5 is a nice happy medium as it lists the files it is working on.
• force: this is necessary to add when doing a restore to a directory that already has some version of the files you are trying to restore. In my case, the default Home directory created for me by Linux Mint already had some default folders, so I had to force rdiff-backup to overwrite them with the version from my backup.
• r: specifies a restore
• now: tells rdiff-backup when to restore as of (see the man page for alternative options if you want to go to a past backup)
• exclude: tells rdiff-backup that these folders exist in the backup but not to restore them
• the last two arguments specify where to restore from (i.e. the backup directory) and where to restore to (in my case the Home directory, you can change this to restore somewhere else and have access to multiple versions of your files)

After several days of trial and error in figuring out which directories are going to cause problems when copying over, I was able to do a restore of my files, which is a relief because you never know whether your backups are any good until you've done a restore. Happy backing up!

Whole-disk encryption on Ubuntu

I've had my wife's laptop running whole-disk encryption with TrueCrypt for a couple of years now, and I always wanted to get that level of security on my Ubuntu machine.  It really makes a lot of sense for the laptops to have as much privacy protection as possible, since we travel with them and therefore they are at higher risk of being stolen.  This week I finally got the opportunity to try it out.

The Plan

I knew I was going to take a performance hit, so I got myself an 8 gig SD card.  The root of my Ubuntu installation would go on it, with an unencrypted /boot partition on the hard drive as well as a 60 gig encrypted /home partition to store all of my files.  The home partition needs its own encryption key, so I decided on a key file that would be stored on the already encrypted root parition, allowing the OS to automatically unlock the home parition and mount it at boot time.

Installation

I downloaded the Ubuntu Alternate Install disk, since the standard install does not include the option to install logical volumes, which are necessary for thinks like encryption or software RAID.  The installation process was somewhat involved as you need to manually configure your partitions.  There's simply no way around it as your /boot directory needs to be unencrypted (this is because the software that performs on-the-fly encryption and decryption is a kernel module, and it can't be started if it is encrypted).  I tried to configure my /home partition on the 60 gig partition on the hard drive like I wanted, but I kept running into weird problems with the installer so I decided to take my chances installing /home on the SD card with everything else and seeing if I could set up /home on the 60 gig encrypted partition later.

So I ended up with /boot on an unencrypted partition on the hard drive, my swap space on an encrypted partition on the hard drive, and the root directory with everything else (/etc, /usr, /bin, /home, etc) on the encrypted partition on the SD card.

Configuration of the /home partition on the hard drive

Less than 8 gigs of space was just not going to cut it for my /home partition, so I was anxious to get the 60 gig encrypted parition configured.

After much Googling, I learned that encrypted volumes in Linux are configured on logical volumes.  This means creating a physical parition, configuring it as a physical volume, adding it to a volumne group, creating a logical volume inside the group, and finally installing a file system inside the logical volume, but fortunately a handy program called cryptsetup takes care of most of that for you.  Encrypted volumes use LUKS, the Linux Unified Key Setup along with dm-crypt.  One of the nice things about LUKS is that it contains 8 key "slots", meaning that you can have up to 8 passphrases or key files, any of which will unlock the volume.  This allows me to have a backup passphrase in case I need to reinstall the operating system or the key file goes corrupt.

So I formatted the 60 gig parition with ext4, and then I ran the following to initialize it as a physical volume for use as a logical volume, giving it the name pvHomeDir:

    $ sudo pvcreate /dev/sda7 pvHomeDir

Then I create a volume group called vgHomeDir and added that volume:

    $ sudo vgcreate vgHomeDir /dev/sda7

I then ran the following line to create an encrypted volume in that virtual group:

    $ sudo cryptsetup -y --cipher aes-cbc-essiv:sha256 --key-size 256 luksFormat /dev/sda7

It asked me for a passphrase, which I supplied.  When it was finished, I unlocked the new encrypted partition with:

    $ sudo cryptsetup luksOpen /dev/sda7 pvHomeDir

I then created a 60 gig logical volume called lvHomeDir in the encrypted parition with this:

    $ sudo lvcreate -n lvHomeDir -L 60G vgHomeDir

Then I installed an ext4 file system in the new logical volume (note the name and location of the volume; it is located in /dev/mapper and has the name of the volume group prepended with a dash to the name I gave it):

    $ sudo mkfs.ext4 /dev/mapper/vgHomeDir-lvHomeDir

OK, at this point the drive is all set up but it needs that key file so that I don't have to enter in two passphrases every time my computer boots.  I create a folder under /usr called keyfile, and copy a picture of myself and my daughter into it, and rename it "file".  To add the file as a key to the new partition I ran the following (I'm pretty sure this is the command but not 100%; I'm sorry if it is wrong and please post a comment if this line needs correcting):

    $ sudo cryptsetup luksAddKey --key-file=/usr/keyfile/file /dev/sda7

Almost done!  The partition just needs to be configured so that it will be used for my /home directory.  I edited 2 files:
    1. In /etc/crypttab I entered:
       
        pvHomeDir /dev/sda7 /usr/keyfile/file luks,retry=1,lvm=vgHomeDir

    2. In /etc/fstab I entered:
       
        /dev/mapper/vgHomeDir-lvHomeDir /home ext4 defaults 0 2

I created a directory called crypt in my home directory on the SD card and mounted the volume with the following command:

    $ sudo mount /dev/mapper/vgHomeDir-lvHomeDir /home/jizldrangs/crypt

I then restored my home directory to that folder, and when I rebooted, I had my old desktop background and all of my files! 

Drawbacks

    1.  I have noticed a performance hit when booting and every so often when browsing the web in Firefox
    2.  During the installation process, I chose the "random key" option for my swap space, so there is no way to do true hibernation, where the state of the machine in memory is saved to disk and restored later.  Suspend, which is where the computer turns off most components and uses minimal power, still works just fine.

Great Linux Games: Extreme Tux Racer

Since upgrading to the beta version of Ubuntu Lucid I've rediscovered a classic Linux game: Extreme Tux Racer.  This game features the Linux mascot, Tux, sliding down the side of a mountain, collecting herring, avoiding trees, and going off jumps on his way to the finish line at the bottom.  It is a timed race (no other contestants involved), and you can start a campaign and work your way through a series of races or simply "practice" on any track you want.

I absolutely love insanely fast, out-of-control speed racing games such as this (I was a big fan of Star Wars Racer for the Nintendo 64 back in the day), and this is the perfect outlet for me.  Trying to keep Tux under control as he is screaming along a half-pipe of solid ice at breakneck speed while collecting herring and racing against the clock is such a blast.  The versions that have shipped with the last few releases of Ubuntu have added some great tracks to choose from (I recommend "In Search of Vodka" and "Candy Lane").

There are several other games for Linux that I want to write about at some point, but this is the one I caught myself playing this evening.  If you haven't tried it I highly recommend it (it is in the Ubuntu repos and the website is here).  Have fun!

From the website:

Ubuntu Netbook Remix

I received an Acer Aspire One netbook for Christmas, and after installing Ubuntu Netbook Remix it has replaced the Dell Inspiron 5100 laptop as my main day-to-day machine.   UNR is a very slick modification of Ubuntu that is well-suited to this type of computer.  Some initial thoughts:

1. Hulu works!  Yay!
2. The smaller screen has been harder to get used to than I thought it would be.  I always thought of the larger screen size of my Dell Inspiron laptop as an asset, I didn't appreciate what an asset it was until I switched to this 10.1" screen.  Fortunately UNR has a neat little feature that helps you get the most of your pixels: when a window is maximized, it will merge the title bar with the menu at the top of the screen.
3. The battery life on this thing is fantastic.  Thanks to the power-sipping Atom processor, the battery will last about 5 1/2 hours.  The Dell laptop's Pentium 4 would guzzle down the battery's juice in about 45 minutes (if I was lucky).
4. Unfortunately the touchpad driver does not support multitouch, so 2-finger scrolling does not work.  UNR provides an "edge scrolling" option, which is what I'm using now, but it would be great to get 2-finger scrolling back.
   
5. Like other netbook OSs that I've seen, the menu is integrated with the desktop.  It looks really slick, is easy to use, and does a good job of utilizing the netbook's limited screen real estate.

I love this little unit, but I'm not sure that I agree with the initial netbook vision.  Netbooks were supposed to be little more than a dedicated web-browser, with most of the applications, and therefore computing, done in the cloud.   Although most of what I do on this machine involves the web browser, there are plenty of client applications I use.  Sure, I am not going to be running VirtualBox on this machine any time soon, but this machine wouldn't have one-tenth the value to me if it couldn't run applications like Liferea, Zim, Unison, Empathy, and Rhythmbox in addition to Firefox. 

Furthermore, I'm not sure that the manufacturer of this device really believes in the original vision for the netbook either.  The original netbooks (and here I'm speaking of the Asus Eee) had a single-core Celeron processor, with a trimmed-down customized version of Xandros Linux, and 4 gigs of internal flash storage.  That was much more consistent with the web-browser-only ideal than today's netbooks.  This Acer Aspire One has a dual-core Intel Atom 1.6 Ghz processor, 1 gig of RAM, and 160 gig hard drive, and it shipped with Windows XP Home Edition.  It seems clear to me that the public liked netbooks but wanted a higher level of functionality than was available in the first generation of netbooks, and Acer along with the manufacturer of every other netbook I am aware of, has delivered.  I'm absolutely thrilled with the results.

It appears that some hardware manufacturers have picked up on the recent netbook trend and have decided to declare war on it.  Litl, LLC along with a few other companies are attempting to return to the original netbook vision with the "webbook", which is truly a web browser with a keyboard attached.  I wish litl the best of luck in their endevours, but I fear for their sake that the days of terminal-mainframe topologies are behind us, and that people will always want their machines to have some modicum of capability.

Hulu quest, Part 1: Ubuntu gets the axe

I have an old Dell Inspiron 5100 laptop that has a Pentium 4 processor, and came with 256 megs of RAM, which I upgraded to 1 gig, the max for this unit. I'm not trying to game on it or anything (that's what my desktop PC is there for); I generally just use it for the less resource intensive computing tasks.

I was running the "flashplugin-nonfree" plugin, which is the official Flash plugin from Adobe, and to its credit it generally worked pretty well. Grooveshark and Homestar worked just fine, and YouTube worked OK as long as the video wasn't too long, so up until this point I didn't have any complaints. But suddenly my wife and I started watching a few shows a week on Hulu and my Flash plugin fell flat on its face.

It fell hard, too. The show would play for about 1 minute, then the picture would freeze for a few seconds, then pick up for a few seconds and freeze again, with the audio cutting in and out accordingly. Needless to say this does not make for a relaxing evening, so I started a quest to get Hulu working on my laptop.

I Googled all over the place and tried a bunch of stuff to get it working. I created the config file /etc/adobe/mms.cfg and added "OverrideGPUValidation=true" to it and that didn't work. I tried Gnash and Swfdec and both were worse than the Adobe plugin. I tried installing libflashsupport but that didn't help (I found out later that this library was created for earlier versions of Flash and Flash 10 no longer needs it). I even tried using Hulu in Seamonkey to see if the problem was in Firefox, but no dice. I read in several locations that recent versions of Ubuntu had messed with some of the PulseAudio settings and this was the cause of the problem, so I decided to do a little distro hopping to see if the grass really was greener on the other side.

More on that later...