I’m trying out Beat Saber mapping! It’s a mix of fun and tedious so far. In that way it reminds me of animation.
Adventures in VeraCrypt Bruteforcing
I wrote a script – one that facilitates bruteforcing VeraCrypt passphrases – with the intent that the user probably remembers most of the passphrase. It’s available here.
It didn’t end up working for me, outside of test cases to make sure the script itself worked, but hopefully it’ll prove useful for others. One day I realized I was in the process of forgetting my VeraCrypt FDE passphrase, even though I’d been using it for months. It was odd – as though the harder I tried to hold onto it the more damaged my recollection of it became. Even though I still think I remember it, the passphrase I remember is wrong. I used the script to check similar passphrases, but they too were wrong. Going forward, it seems wise to not rely on muscle memory, and instead repeat to oneself the actual content of one’s passphrases from time to time. Go over mnemonic devices. Fight against it slipping away.
C.H.I.P SSH Troubles
I was working with a C.H.I.P and the SSH host keys were regenerating each boot after upgrading to Stretch. This caused a host key mismatch every time. It turns out /etc/rc.local was a script which checked for the presence of SSH host keys, including DSA, and if found all of them it replaced itself with /etc/rc.local.orig, which is the stock does-nothing script. If it didn’t find all of them, it would delete any existing keys and regenerate all of them.
I still don’t understand why this produced the behavior it did, because the script did succeed in producing DSA keys, but replacing the weird /etc/rc.local with the /etc/rc.local.orig that just exit 0s seems to have solved the problem.
St. Louis
I was recently in St. Louis. It’s a big city! 2.8 million in the metro area compared with 340 thousand here in Ann Arbor. The buildings bear that out: the tallest one in Ann Arbor is Tower Plaza at 267 feet (81 m) tall, whereas in third place in St. Louis is the Thomas F. Eagleton United States Courthouse at 557 feet (170 m). I guess the courthouse puts a lot more into each floor, because for all its height advantage it only has two more than the 26-floor Tower Plaza. I don’t mention the tallest because the courthouse was the reason for my visit – I’m an expert witness in a court case involving Freenet.
I learned many things on this trip. Among them were that I should not have wandered around until I found a place for lunch because the city has violence-prone areas. Another was how much one’s life can be destroyed by the legal system before even going to trial: apparently it is common for defendants to be prohibited from leaving a county, using alcohol, or using the Internet without being convicted. Yikes.
Cat Time Lapse
My cat is usually waiting at the door when I come home. I hoped he wasn’t waiting there long, so I set up a time lapse to find out. Initially I tried an old webcam connected to my Linux server with a script to periodically capture an image, but that failed after 4 hours. The kernel logged something about video URB and UVC probe control. (Not USB. Dunno what it was. Didn’t care to find out.) Continually connecting and disconnecting from the camera probably isn’t the friendliest of use cases for a video subsystem.
What I ended up doing was using the built-in Windows 10 camera application, which I was happy to discover supports time lapse. I used it to save images at 5 second intervals. In contrast with my hack using the other webcam, this application continually captures images, meaning the camera stays active, and it only saves them at the interval. The first time I tried it the camera captured fine until I got home, but I got home late enough that it was too dark to see the cat anymore. Today I got home early enough to find this sequence right before I got home:
He enters from the left of frame. Either behind the camera or below its field of view, perhaps eating food from his bowl there.
He looks out the window for close to a minute. Judging by the timing this was just before and as I was parking (visible from his position) and walking up to the building.
He then waits at the door for about 20 seconds:
Then I greet him and turn off the capture:
A fun experiment! It resulted in 1.3 GiB of JPEGs. I wonder what cues he has learned for my arrival. The stairs up to the second floor squeak so I assume he notices that at least. Earlier on he was on the couch looking around, and at the window looking around. This sequence here was just the last few minutes. He certainly didn’t seem to be actively waiting for long, which I’m glad to see.
Filesystem Transplant
Not having snapshots on ext4 finally got too annoying. I was able to copy the root filesystem off, then format with btrfs and copy it back on. I used System Rescue CD to do this, turn off Copy on Write on database files, and edit /etc/fstab with the new filesystem and UUIDs. It worked! Eventually. There were a few snags:
- Grub gets angry when you wipe all your partitions. I’m still not clear on what the UUID it was looking for was, because it didn’t look like the old root filesystem. Using System Rescue CD’s Super Grub Disc image I was able to boot into the system and run update-grub and install-grub, which fixed it.
- tar with bzip2 is slow. Using tar without compression ended up being much, much faster.
- Taking out the drive with the swap partition caused the boot to hang until timeout. There isn’t an mkfs.swap, but there is a mkswap.
- The script I used to disable CoW didn’t preserve ownership information, so I had to re-chown things appropriately. Oddly PostgreSQL still started, but MySQL did not. That was nice because it alerted me to the problem.
Hooray for snapshots! I’m hoping to set up snapshot backups Soon. (TM)
Purpose
I recently resigned as the Freenet project release manager. The reduction in the amount of obligations I have is refreshing, but I do feel I’m lacking a project to focus on. I’m curious to see how this turns out.
Treacherous Variable Names
Now for the chronicle of a bug clearly caused by poor variable names. (And arguably also the lack of semantic meaning given to string types.) This buggy function searches for a device with a given uuid:
/*
* Search the specified glob for devices; return error code. On
* success dev->fd is set to a valid file descriptor and the file
* is locked.
*/
static int __find_dev(struct vdev *dev, struct uuid *uuid,
const char *const path)
{
glob_t paths;
int ret;
size_t i;
ret = glob(path, 0, NULL, &paths);
/* Error handling omitted... */
for (i = 0; i < paths.gl_pathc; i++) {
const char *const fname = paths.gl_pathv[i];
dev->fd = mopen(fname, O_RDWR);
if (dev->fd < 0)
continue;
if (is_requested_device(dev->fd)) {
ret = __try_dev(dev, uuid, path);
if (!ret)
goto found;
}
close(dev->fd);
}
ret = -ENOENT;
found:
globfree(&paths);
return ret;
}
mopen() is given fname, the path produced by glob(), but __try_dev() is given path, which is actually a glob. This caused the device to import correctly but report as its path the glob used to find it instead of its actual path. path is now named search_glob, and fname is now named path. Our existing tests did check that the device path was reported correctly, but they didn’t catch this because they all explicitly specified the exact path to the device to use, so the search glob was the correct path. If the search glob had been some kind of glob type instead of a string then the patch that caused this bug wouldn’t have compiled.
Fun With Concurrency
First a bug, then an exercise in assuming the common case.
We were parallelizing a single-threaded operation to increase throughput and came upon a race condition that at first manifested as crashing only on Ubuntu. One thread read jobs from the disk and passed them to an existing worker pool for processing. Once all processing was complete, the reading thread exited. When a worker finished processing it marked its job as complete, then released its locks. This could result in releasing a freed lock – because the thread pool was also needed for other purposes it wasn’t joined first. My best guess as to why it only showed up on Ubuntu (and Debian, it turned out) is that it has different defaults for what we assume to be stack protection, though that wasn’t clear from listing them with gcc -Q --help=target. Though changing the order of lock releases solved the immediate problem there were also issues with deadlock due to dependencies involving the thread pool. We ended up having an additional processing thread that was joined before exiting, which avoided the problem.
While this parallelization does get its speed from doing previously single-threaded work in a thread pool, it has a twist. A final processing step must be done in disk order, but waiting for the processing slowed down reading. By reading speculatively as though processing had succeeded, we achieved a ~2x performance increase. Yay!
Network Interfaces
If I set the scene with remotely configuring network interfaces at 3 AM you’ll probably already guess it was a bad idea. In retrospect I would have been better served by my reservations that I should do it when I was better rested and had given it some thought. As it was, I was configuring a bridge for a KVM according to the excellent Debian Handbook.
/etc/init.d/networking reloading the network interfaces didn’t bring the new ones up, so I tried restarting them. One of the last messages I saw was "Running /etc/init.d/networking restart is deprecated because it may not enable again some interfaces". The last message I saw was a DHCP release. Whoops. What I probably should have done is run ifup on the new interfaces directly instead of risking bringing down the interface I was connected over.