I'm giving up on the fancy viz, from here on out I make everything look like Pac Man. So, a few weeks ago I got interested in how public sites track ships and planes. I wound up doing some simple (infrequent) scrapes to see what the data looks like over time. Tonight I finally reassembled the data and plugged it into Pylab. Now all I need to do is figure out how to plot Blinky and I'll be set.
10GigE SFP+ is bogus. Or more to the point, some high-end network companies are doing some bogus things with SFP+. SFP+ (Small Form Pluggable) is a standard for packaging electrical/optical transceivers. Basically, when you buy a switch or line card these days, each network port is an empty socket with electrical connections. If you're going short distances (9m), just plug in a direct-attach cable ($40-100) and you're done. If you need to go farther or don't like heavy cables, buy fiber and a pair of SFP+ modules. The SFP+ pinouts are pretty straightforward: the transceiver has differential Tx/Rx lines, power, ground, clocks, status lights, etc. Oh, and there's also a 2-Wire serial line.
That serial line is the problem, because it connects to a small eeprom that stores info about the SFP+ module. Some of it is good stuff: optical freqs, fiber type, nominal encoding mechanism, etc. However, they also provide fields for a vendor IEEE company ID, as well as vendor-specific data. Sadly, several switch companies are taking advantage of this to make sure you buy both the switch and all SFP+ modules from them. I believe the switches read the ID tag out of the SFP+ and complain if they don't get the magic values. If you look on Amazon, you'll see three pricing tiers for SFP+: $700 "vendor X", $200 "vendor Y", and $90 "no-guarantees". I recently tried the $90 version I was relieved to see they worked just fine with my cheap, no-frills switches.
Interestingly, some other companies are doing a good business selling compatible SFP+s. There's a company called SFPEX that looks interesting. If you pay them an extra $2, they'll even slap a custom label of your own design on the module.
Anyways, I think our future is going to all be about Monster Cable. More standards (eg thunderbolt) will plug eeproms into the cable heads and more vendors will abuse these ids to make sure we all have to buy their double platinum certified cables, just to hook up our printers.
Ha. If at first you don't succeed, cheat. For the last week or so, I've been looking into how one could build a minimal OS that would have just enough stuff in it to support some full-featured Java apps (ie, a Hadoop/Accumulo stack). Someone got me thinking about starting with nothing and working my way up, as opposed to what most people do, which is start with a bloated distro and try to strip it down.
The first part was easy: I pulled the latest Linux Kernel and built a stripped down version with basic drivers to get a VM running. I then installed BusyBox to a blank initramfs to give the node something to do after booting the kernel. Static programs worked great, but when I tried plugging in any executables with dynamic libraries, they of course didn't work because there was no libc. Given that Java needs libc and can't be recompiled (thus no uClibc), I had to figure out how to get glibc in the initramfs.
Every time I've tried to build glibc I've failed- sometimes spectacularly, destroying the machine (funny how much breaks when you overwrite a working glibc with a broken one). Over the last few days, I've tried a number of different things, but none of the installs ever worked. Things landed in the wrong places. Glibc took issue with my kernel. Older versions didn't stick, either.
I finally gave up and cheated. I took a stock fedora glibc rpm, manually unpacked it into my initramfs, and then everything magically worked. I plugged the JRE in as well, and voila I got the below message in the VM. It's annoying to not get it to work from scratch, but whatever, it works. And since KVM boots directly from the kernel/initramfs files, the VM is up and giving a prompt after about one second. The initramfs is 55MB compressed. Not exactly floppy drive material, but also, much better than any other distro installation that includes Java. Guess I've got to figure out how to get Hadoop going on it now.
After a few days of fiddling with VTK's DICOM reader, I decided it'd be easier to just ask David Thompson (now at Kitware) what he would do, since he's the smartest viz guy I know. Since it's CT data, he pointed me at Slicer, a great viz tool that's specifically designed to look at medical datasets. It was pretty straightforward to get up and running and parsed my data no problem. After a few tutorials, I found the ray casting options I was needed to turn slice data into volumes. There you have it- my pelvis. I wonder if that hole on my right side is from that time Benjamin and I flew off that slide in Berkeley.
Fun. And kind of nasty at times (the misses got the giggles every time I set the threshold low and all my "bidness" came into view). I still haven't found the abscess the doctors noticed so easily, but I think I can now point out where that John's Charburger from the previous night was hanging out. I think they should install scanners like this in all airports, and then give people their data. Everybody wins.
"Hey Amy, everyone else is playing games on their ipods, kindles, or laptops, is it ok if I do this Sudoku puzzle in the paper?" Ten minutes later I find myself thinking the same thing I always do when I try the Sudoku puzzle: "This is boring, I should just write a program to do this for me." Well, I'm on vacation, so last night I just went ahead and finally wrote that program. It took me a lot longer to write it than it took for Amy's (puzzle-fiend) father to solve the puzzle that was in the paper, but it was a lot more interesting to translate my fuzzy strategies into functional algorithms.
I started by writing a function that would generate all the vals that a missing entry could be. Since that only shakes out obvious missing entries, I wrote another routine to do eliminations (ie, this is the only cell that a number could be in for each of the three series). A little debugging and the C program spit out the solution as far as I had been able to do with paper (not guessing values). Last, I added a brute force guessing routine to iterate when no path forward was available. Following some tuning to eliminate dumb guesses, the program spit out the right solution in about a second.
Fun. Yeah, I know, they ask CS students to code these things up on AP exams and hot-startup job interviews. It was still fun to me, largely because none of the real world CS/CompE problems I've worked on have ever had such well-defined rules, easily-quantized inputs, or easily-validated solutions. Some day I'll work a CS job where all we do is implement quick sort instead of resolving configure dependencies between different aging source trees.
