PSA: Typos in mkfs commands are painful

david@systemoverlord.com (David Tomaschik) — Mon, 20 Oct 2014 14:19:40 +0000

TL;DR: I apparently typed mkfs.vfat /dev/sda1 at some point. Oops.

So I rarely reboot my machines, and last night, when I rebooted my laptop (for graphics card weirdness) Grub just came up with:

Error: unknown filesystem.
grub rescue>

WTF, I wonder how I borked my grub config? Let’s see what happens when we ls my /boot partition.

grub rescue>ls (hd0,msdos1)
unknown filesystem

Hrrm, that’s no good. An ls on my other partition isn’t going to be very useful, it’s a LUKS-encrypted LVM PV. Alright, time for a live system. I grab a Kali live USB (not because Kali is necessarily the best option here, it’s just what I happen to have handy) and put it in the system and boot from that. file tells me its an x86 boot sector, which is not at all what I’m expecting from an ext4 boot partition. It slowly dawns on me that at some point, intending to format a flash drive or SD card, I must’ve run mkfs.vfat /dev/sda1 instead of mkfs.vfat /dev/sdb1. That one letter makes all the difference. Of course, it turns out it’s not even a valid FAT filesystem… since the device was mounted, the OS had kept writing to it like an ext4 filesystem, so it was basically a mangled mess. fsck wasn’t able to restore it, even pointing to backup superblocks: it seems as though, among other things, the root inode was destroyed.

My Time at KSU

david@systemoverlord.com (David Tomaschik) — Fri, 03 Feb 2012 04:24:58 +0000

As you might have seen, I'm leaving my position at Kennesaw State University to take a position as a Site Reliability Engineer at Google. This is something I'm very excited about, but I thought I'd take a look back at my time at KSU as I approach the end. It's worth mentioning that I'm not leaving KSU because of KSU, but because this is an opportunity I just could not turn down. For the most part, I like my position at KSU, and I really like most of the people that I work with. There's a particular group that's become three of my closest friends and one treasured acquaintance.

Lessons Learned on System Overlord

PSA: Typos in mkfs commands are painful

My Time at KSU