Minimal Stack

Ha. If at first you don't succeed, cheat. For the last week or so, I've been looking into how one could build a minimal OS that would have just enough stuff in it to support some full-featured Java apps (ie, a Hadoop/Accumulo stack). Someone got me thinking about starting with nothing and working my way up, as opposed to what most people do, which is start with a bloated distro and try to strip it down.

The first part was easy: I pulled the latest Linux Kernel and built a stripped down version with basic drivers to get a VM running. I then installed BusyBox to a blank initramfs to give the node something to do after booting the kernel. Static programs worked great, but when I tried plugging in any executables with dynamic libraries, they of course didn't work because there was no libc. Given that Java needs libc and can't be recompiled (thus no uClibc), I had to figure out how to get glibc in the initramfs.

Every time I've tried to build glibc I've failed- sometimes spectacularly, destroying the machine (funny how much breaks when you overwrite a working glibc with a broken one). Over the last few days, I've tried a number of different things, but none of the installs ever worked. Things landed in the wrong places. Glibc took issue with my kernel. Older versions didn't stick, either.

I finally gave up and cheated. I took a stock fedora glibc rpm, manually unpacked it into my initramfs, and then everything magically worked. I plugged the JRE in as well, and voila I got the below message in the VM. It's annoying to not get it to work from scratch, but whatever, it works. And since KVM boots directly from the kernel/initramfs files, the VM is up and giving a prompt after about one second. The initramfs is 55MB compressed. Not exactly floppy drive material, but also, much better than any other distro installation that includes Java. Guess I've got to figure out how to get Hadoop going on it now.

Pelvis

After a few days of fiddling with VTK's DICOM reader, I decided it'd be easier to just ask David Thompson (now at Kitware) what he would do, since he's the smartest viz guy I know. Since it's CT data, he pointed me at Slicer, a great viz tool that's specifically designed to look at medical datasets. It was pretty straightforward to get up and running and parsed my data no problem. After a few tutorials, I found the ray casting options I was needed to turn slice data into volumes. There you have it- my pelvis. I wonder if that hole on my right side is from that time Benjamin and I flew off that slide in Berkeley.

Fun. And kind of nasty at times (the misses got the giggles every time I set the threshold low and all my "bidness" came into view). I still haven't found the abscess the doctors noticed so easily, but I think I can now point out where that John's Charburger from the previous night was hanging out. I think they should install scanners like this in all airports, and then give people their data. Everybody wins.

Sudoku

"Hey Amy, everyone else is playing games on their ipods, kindles, or laptops, is it ok if I do this Sudoku puzzle in the paper?" Ten minutes later I find myself thinking the same thing I always do when I try the Sudoku puzzle: "This is boring, I should just write a program to do this for me." Well, I'm on vacation, so last night I just went ahead and finally wrote that program. It took me a lot longer to write it than it took for Amy's (puzzle-fiend) father to solve the puzzle that was in the paper, but it was a lot more interesting to translate my fuzzy strategies into functional algorithms.

I started by writing a function that would generate all the vals that a missing entry could be. Since that only shakes out obvious missing entries, I wrote another routine to do eliminations (ie, this is the only cell that a number could be in for each of the three series). A little debugging and the C program spit out the solution as far as I had been able to do with paper (not guessing values). Last, I added a brute force guessing routine to iterate when no path forward was available. Following some tuning to eliminate dumb guesses, the program spit out the right solution in about a second.

Fun. Yeah, I know, they ask CS students to code these things up on AP exams and hot-startup job interviews. It was still fun to me, largely because none of the real world CS/CompE problems I've worked on have ever had such well-defined rules, easily-quantized inputs, or easily-validated solutions. Some day I'll work a CS job where all we do is implement quick sort instead of resolving configure dependencies between different aging source trees.

Camera Phone Blog

Back in 2003 I noticed that cell phones were beginning to include digital cameras that allowed you to take pictures and email them to others. The cameras and software were awful, but it occurred to me that posting cell phone pictures of what I was doing would be a lot easier than all the writing I was doing for my homemade blog. I bought a phone and service, and then put together some scripts to make my own camera phone blog. A decade later this would all just be "what phones do", but at the time it felt wildly inventive and a major step forward in connecting with my friends and family.

Old School Blogging

I've been interested in using the web as a way to share info with my friends for a while. In 1998 when I spent a Summer in France, I decided to post our misadventures on a web page instead of just emailing me because I didn't want to have to go back and retell the same stories to people that I forgot to cc. After enjoying my friend Kelly's Kellegous blog, I decided it'd be fun to start my own blog which I called AngryNoodle.com. Rather than use one of the pre-made blog engines, I cobbled together my own set of Perl scripts to convert my text entries into static web pages. I learned a lot along the way, from CGI handling to html quirks, as well as the wild west of dealing with hosting companies (the first one I used went out of business and left me hanging).

The problem with the blogging was that even simple posts took a long time for me to do. In addition to having to manually resize pictures and upload the static pages whenever content changed, I am a very slow writer. I'd start out with a little idea that I thought I could get done in just a few minutes and then it'd snowball into a major chunk of time. I liked sharing info and love having a history of what I've been up to, but blogging as I was doing it is just too time consuming.

Cell Phones and a Cross Country Trip

I took some time off after graduating in 2002 to decompress and do a thorough job search. In the end I decided to take a job in California, which meant we'd have to relocate from Atlanta to Livermore, CA. A moving company took care of shipping all of our things, but we decided to take a week and make the cross-country drive ourselves. It seemed like the kind of trip I should be documenting in full on AngryNoodle, but I knew I wouldn't be able to get them written and posted while the driving was still happening (hotel internet was still pretty haphazard back then).

Since it was going to be a long drive and I was worried about the car having problems, I decided it was time to upgrade my cell phone and splurge on a better plan. Looking around at all the options I noticed a couple of the new cell phones had a "camera option" that they said would enable you to take pictures with the phone and send them to others. It seemed like a great opportunity to modernize my blog and do status updates from the road. I signed up for service from TMobile and bought a brand new Sony Ericcson T300 with a camera attachment.

The T300 Hardware

Just one look at the T300 and you can really tell how primitive cell phone cameras were at the time. The phone didn't have a built-in camera- it had an external camera module that plugged into the charger port of the phone. The module was an extra $50 on its own (though free with a plan). Its resolution was a whopping 640x480, and there was only enough flash on the module to store about five pictures at that resolution. It had an "optical viewfinder", but all this was was a glass window through the device. The phone was able to display a lowres pictures while you were aiming, but only at a few frames per second.

Using the T300 as a camera phone was an adventure. Every time you wanted to take some pictures you had to plug the module in and wait for the hardware to figure itself out. Sometimes it didn't and you had to unplug the module or power cycle your phone. After a few months of use the connector loosened up, and if you didn't hold it right while taking pictures, the phone would lose the connection to the camera and you'd have to reinitialize the unit. Another problem was that picture uploads were sent using GPRS, which was very slow (definitely less than 32Kbps) and error prone. Quire often you'd send a message, watch a slow progress bar for 30 seconds, and then get an error message about a communication failure. Even with all these problems, it was still very exciting to be able to send pictures while out in the field.

Data Transfer Software

The T300 provided minimal apps for doing things with the phone. They wanted you to use MMS to send pictures to other phones, but they also had an option to email pictures. The pictures were simply mime encoded and tacked on to the end of an email. I realized that if I wanted to make a camera phone blog, I needed to create some kind of handler on my server that would look for incoming emails with mime attachments and then extract the pictures to a webserver directory. I hadn't scripted much with emails so I enlisted the help of John Lockhart and Kelly Norton. John and Kelly gave me tips on how to parse email folders on a server, and provided me with an example of how to extract mime data in Perl. Initially it seemed like procmail would be a nice way to automatically parse emails when they arrived. However, there were some issues with using procmail on my ISP, so I needed to find an alternate method. I decided to just go with a cron script that checked the email file every 10 minutes. My Perl script parsed the email file manually, validated the sender address was my phone, extracted pictures to a web server directory, and then made backups of the data in case something went wrong. Using cron meant the photos weren't instantaneous. However, it was good enough that my friends could check in during the trip and see what was doing on as the stories unfolded.

Web Page

The last thing to do was build a simple web page that let you scroll through the list of pictures. I wrote a plain old Perl CGI script to dynamically create the pages as people asked for them. Initially I just displayed the pictures with their dates, but later I added a comment system where anyone could write some thoughts. Unfortunately, some SEO spammer found my pages and filled the comment section up with spam. I manually wiped the comments and added a simple posting password. The password worked pretty well- people who knew me could easily fill in the blank of a specific phrase. The comments also let me explain what was going on when I got back to a place where I could type.

Camera Phone Blog Use

I was able to get the whole system working before we started our long drive to California. It was great to be able to take pictures and share them as they happened instead of waiting until the next hotel. It was cool to get emails from friends saying they had checked on us and were wondering where we'd taken that day's picture. The camera phone also helped keep our sanity on the long drive. I think we were more willing to stop and look at things because it was fun to post goofy pictures.

Pictures from the drive cross country:

Pictures from our initial adjustment period to California:

Pictures from our "people in California go to Hawaii for Christmas" realization.

I continued to use the camera phone blog long after the drive. I periodically had to make changes to the scripts when either TMobile or the ISP messed things up. My ISP eventually switched to a more elaborate mail caching system that made the email parser impractical. At the same time, phone cameras and social media rose up and provided a more convenient outlet for sharing pictures. In retrospect, I probably should have been more enterprising and found a company that also saw the power of putting cameras in everyone's cell phones. I think the bigger point here though is, don't listen to anyone that says they "invented" some new idea. When new technologies become available, many clever people will think of the same ways to use the technology.

Code

I don't expect anyone to use it, but I've checked the code into github.

gitub:camphone-blog

Disclaimer

The personal hobbies portion of this website is a collection of stories about projects that I've worked on in my free time outside of work. Opinions expressed in this portion of the website are solely my own and do not express the views or opinions of my employers. No employer-owned resources were used in these projects.