Only care about what I built? You should skip to Numbers. Otherwise… let us begin.
One Late Night on Hacker News
A Baller came to my Aid
Summoning Spell: SMTP
and a potent arrangement of bits
In the beginning, Nathan did not have a job for the summer. After his finals, the “potential-job-list” was formless and empty, darkness filled his job prospects, and his browser was hovering over the job boards.
And Nathan said, “Let there be cold-emails”, and there were not cold emails, because he actually had to go and write the emails and hit send after agonizing over whether it was good enough (until he had sent so many emails that he just did not care anymore). So he went and did that, and it was good.
And Nathan said “Let there be replies to my cold-emails”, except cold-emails don’t answer themselves, and the humans behind email@example.com were sometimes fickle, as humans are expected to be. But lo! there were replies to the emails, and some of them were timely, and it was good.
And Nathan said “Let there be a job offer”, but he wasn’t the Flying Spaghetti Monster who could touch the humans behind firstname.lastname@example.org with his noodly appendage and create job offers out of thin air. However, it became so, and in one case in a timely manner: and this is the story of what came of that job offer. <SPOILER ALERT> I took the job offer </SPOILER ALERT>
And Nathan said “Let me accept the job offer”, and he did so. That brings us to…
And the wanderer came out of Seattle, screaming:
IA! IA! NAYR NOSLRAC FHTAGN!
So now that I had a job waiting for me down in the Bay Area, I had to actually get down there and not die in the process. This meant frantically craigslist-ing for about a week before my flight down, canvasing the wider bay area for places to stay.
There was a decision to be made: find a room in Fremont, or in Milpitas, or in San Jose, or in Mountain View, or in Palo Alto. The first 3 places were even under consideration because Redwood Systems (who I was to be working for) was based out of south Fremont, which is less a traditional startup area and more a cleantech center. However, proximity, smelliness, and being-far-away-from-all-the-awesome-tech-stuff-in-the-valley were beat out by carpooling, nice neighborhoods, and being-close-to-all-the-tech-stuff, so I opted to find housing in the traditional Silicon Valley area, which I considered as Palo Alto and Mountain View.
And in retrospect, that was a very good decision, for all the reasons stated. Although, I will log my complaint with Caltrain’s lack of dense scheduling (which didn’t have anything to do with location in the valley): the guys running the Caltrain need to learn to party like the NYC subways, which run all night and run so frequently New Yorkers get annoyed when the trains don’t come within 5 minutes. On the other hand, the NYC subways need to learn to use RFID (like the Clipper) instead of stone age card swipes, so I guess it’s even.
Back to housing, a note: the short-term housing market in the valley is very competitive, so I eventually adopted the strategy that if there was a decent room, then I would take it. Yes, on the spot. Perhaps one should see a few rooms first to get a sense of the spread of quality of rooms out there, but I lost a number of rooms because I put off making a decision to “sleep on it” (which is generally good advice, but also prey to “hesitation is always easy, rarely useful”). Especially if you’re only around for a couple months, then it won’t matter quite so much where you live, and really, having a place to call your own is nice.
Packing, I decided to pack pretty minimally, and now that it’s over it feels like I have too much stuff. Yup, too much stuff in a fairly bare college dorm room. Well, let me clarify: too many clothes (NEVER TOO MANY MULTIMETERS OR BOOKS). Why in the world do I have so many t-shirts? But I digress into the present.
Oh, and getting to the bay area? Easy. Leaving without having housing tied up? Even easier, because having friends is awesome, especially friends you can bum off of for a week or two while you look for housing (friends that were more proactive about housing *cough*). Even if I had no friends, then I could have just hit Airbnb until I found a place.
d = math.sqrt(randx()**2 + randy()**2)
Yes! Yes! Everyone was laying down the law on me!
Yeah, this doesn’t fit with the Pentateuch theme. So I guess I’ll just talk a bit about the place I worked, Redwood Systems.
Time for a tldr on Redwood Systems if you haven’t checked them out yet: they make LED light drivers for commercial installations, adding bundles of sensors and networking everything together along the way. Try looking up at a ceiling in some commercial space (data center, office, retail) and imagine each light being “smart”. Lots of interesting information to be had there. Oh, and they’re a startup.
However, they’re a strange startup, by which I mean they’re more of a traditional business. As I mentioned previously, the company was based out of the cleantech sector of the valley, working out of a bland office building (contrast glitzy startups in New York). The office was surprisingly old: I estimate the office would have had an average age of around 30, versus a bunch of kids piling out of the classroom and into an office space.
This showed in the culture: instead of having regularly scheduled drinkups or an annual trip to foreign lands, we had a fortnightly barbecue and everyone had kids. Nothing wrong with that: I’m a low key person, there was plenty of other things happening in the bay area, and there were young people around to associate with (I will note I was the youngest person in the company, at 22, even among the interns). Besides, I have nothing against children.
However, it might be hard to get younger people excited about the company when the median age is pretty high, and it doesn’t have the perks all the cool kids have. It does have a different field, which is interesting and also offputting, since stereotypical startup person is a web engineer, or maybe a mobile app developer. Well, as long as the lights stay flashy (har har) they should get some interest: plus, there’s tons of data for the taking.
Size-wise, the company was about 50 strong when I left. According to whatever definition you use, this may or may not be a startup: it’s not 2 guys in a garage, but it’s also not a multinational. With growth and more investment, it’s growing, and looks like it’s set to continue growing.
But time to answer the real reason I’m here: so what did I really end up doing at Redwood? (other than drinking all the snapple) Well, doing stuff with numbers (badum-tsh), and, uh, other stuff. Officially, I was an Application Engineer, essentially someone that would build demos for the sales team. So, why don’t I just enumerate all my children? And Nathan begat…
When I got to Redwood, the first thing that struck me was the fact there was no benchmarking going on. No one was doing manual data gathering to corroborate and test the data gathered by all the sensors, although I was probably just being anal about the whole thing, since what I wanted was much more high level than what the rest of the company wanted, but nevermind… So I set out to rectify this, counting people in conference rooms that had Redwood engines installed. I built a small tool that used Python, pygtk, and pyOpenCV to help speed up the process. Alas, this effort came to naught (I don’t believe anyone has used the data since I left, either), as I got swept up into other projects such as…
I didn’t understand it then, and only barely understand it now: why anyone would want their alerts to go to their Facebook stream? I suppose others like mixing their work and personal lives more than me, or people check their Facebook more often than their mail, but… okay, I admit: I am not a Facebook person. However, if I remember correctly I got to work with some Python Library (probably CherryPy) and play with Facebook’s Graph API, so it was okay. However, this also came to naught, for one day my boss came into the office and said DROP EVERYTHING AND WORK ON…
Historical Data viewer
It turns out that for all the data being flung around by the Redwood engines, there weren’t any really good long-term visualization tools yet. Plenty of aggregate data viewers built into the web interface, but nothing meant for years of introspection. Also, it turns out flash doesn’t work on iPads, and guess what we wanted to do?
And here is where the cool projects start: you can tell, because we actually started giving names to them.
This particular project was suggested by my boss, inspired by a sub-par offering from an office furniture company, the product bringing a physical interface for scheduling systems to conference rooms. However, the only information it would convey was the reservation status of the schedule, functioning as a very expensive notification and CRUD app mounted outside of conference rooms. On the basis of price alone one could do an order of magnitude better with mere iPads (if you’re using base 2).
Hence, Stonecold was an attempt to use iPads as a replacement for such systems, and doing them one better: instead of only pulling information from some calendaring system, it also polled the occupancy of the room from the associated Redwood engine, displaying a status based on a combination of whether people were in the room and whether anyone was scheduled to be in the room at that time.
This project was done solo, although there wasn’t much of a backend component: I used Cherrypy to poll the Redwood engines and serve up the pages/AJAX requests. On the frontend, it was a dollop of jQuery and an interesting application of iframes (now that I think about it, the other content should have been in the iframe. Oh well). Cute little project, if I dare say so myself. The demo iPad might still be hanging outside of a conference room somewhere, but as far as I know it’s hasn’t been picked up by anyone else.
And yes, you can haz screenshots.
And the name? An inside joke.
Since I’m writing this some 4 months after leaving this project in the dust, tears in my eyes as I drove away, the project drawing into the distance… and the past. Wait, what was I talking about?
Right, so I don’t know why I named this project Icicle. You’re going to have to live with the uncertainty, methinks.
The big deal with latency (the reason for using websockets) was because I wasn’t very interested in historical data: instead, I was all about real time data, partially because I wanted instant gratification and partially because I wanted it to give me insight into how to extract person positions from raw motion data, and testing would be faster if the data was available in real time. However, I soon dropped the position extraction as infeasible in the given time span, and just went with displaying motion “hits” on a floorplan.
And SURPRISE THERE IS A YOUTUBE VIDEO.
Also, the cool as a cucumber YosemiteBandit (have we met before?) took this idea and ran with it. Awesome, makes me feel all warm and fuzzy inside.
There’s a tool internal to Redwood Systems called Pynecode: it’s primarily a tool to look at data dumps from Redwood engines with graphs. However, it’s a command line tool, with all that entails. I myself am a proponent of CLIs, but I thought that having non-technical people learn CLIs just to run data analysis just was not a winning proposition. Also, there were some problems with installing Pynecone dependencies, especially with OSX.
So I set out one fine morning, questing to put an end to this nonsense with one webapp to rule them all. The end goal was an application at which you could throw all your h5 files (data format of choice), which would be turned into graphs with a bunch of flexibility over the data processing and display. Ambitious? Yes, especially since it was nearing the end of my time at Redwood.
However, I gave it my best shot, with my trusty, straight edged Cherrypy at my side, and the durable (if boring) pickaxe h5py in my hand, and the humongous mongodb on my back. Trying to cover the distance proved too much for me, though, and I succumbed in midstroke, working until the last minute, when I left for Oakland International Airport.
I don’t know if anyone else picked it up: Matt did something similar while at Redwood, but it’s probably based off the Historical Data Viewer code.
And so it goes,
Forever and ever,
Yeah, this heading doesn’t quite fit either. Maybe we could do that new dance instead? Do the rotate feet?
Instead of subjecting you to my horrific (or even worse, nonexistent) dance moves, we’ll wrap up with lessons learned.
Speed is god. The faster you move, the more iterations you can do, and the more projects you can do! Spydercone might’ve had a chance of living, the poor thing. Of course, there’s a terminal velocity, but I don’t think I’ve reached it yet.
I should not have used Cherrypy for everything: when you’re getting paid to make things, and learn whatever you need to make those things, you might as well learn everything. I have too big a button in my brain labeled FINISH THINGS to just sit around and take undue advantage of such learning, but it would’ve been nice to investigate, say, Tornado or flask.
And in closing, I enjoyed working there: the people I worked with were pretty cool (why do you think I have so many memes linked?), and I learned something about working outside of the academic world. A+, would do again if I was in the same situation. And, I’m looking forward to looking up and seeing them installed in a building near me.