Your prototype is your final product (part 1)

About 5 years ago, I worked at a division of Brooks Automation which built semiconductor wafer-sorting robots. My most interesting project was creating an experimental image-processing algorithm to detect a reference notch on a silicon wafer (which is used as a reference position for etching — we used it to locate ID numbers and barcodes). It was a challenging project — the reference notch is a rather small feature that only occupies a few pixels in a wafer image.

I was excited to demonstrate that the project should work in principle, so I threw together a prototype as quickly as I could. I made it very clear to everyone (including management) that this would be a prototype only, that it was unsuitable for mass-distribution, and that I would need some time after the demonstration to rewrite the code.

So, after about 3 weeks and with the help of some hardware tweaks (to provide a high-contrast image) I got it to detect the notch location about 90% of the time. Not bad for a first pass, but clearly it would need a lot of work to get that up to production quality (somewhere above 99%).

But, there was a problem…I had used a lot of “rapid prototyping” techniques (and hacking) to get things done as quickly as possible. I had used smart pointers instead of proper memory management, and as a consequence, there was a lot of memory waste (holding on to unnecessary images in the image processing chain) and thrashing/fragmentation (caused by allocation/deallocation of image buffers which could have been reused). The algorithm worked, but it was slow. I don’t recall exactly, but I think it took about 3 seconds to process an image.

But I thought “Not to worry, with some additional time to rewrite proper memory management routines, plus optimizations, that will easily reduce to half a second or less.”

The next day, my boss told me that we would use the code as-is.

I explained that the code was not ready for production, that I needed time to rewrite. I explained about the memory management problems and how optimization would require a different code structure. I also reminded him again that I had been upfront about the need to spend extra time for a rewrite (I had reminded him almost daily because I anticipated such a problem). He still wouldn’t budge and insisted that I continue with the current code base.

That’s when I learned the lesson which became the title of this post: Your prototype is your final product.

It’s a good thing to remember. Whenever you work on code (or engineer any product), what you think of as a prototype is likely to be seen as a final product by management. Once management sees visible results, they want it right away. And, in many cases, they just don’t understand the value of improving things that are “under the hood”. After all, what they see seems just fine to them, so what value could there possibly be in improvements?

[to be continued...]

Tryin’ ta git back ta duh bloggin…

As you can tell, there was a really long gap in my posting. During that interim, I had worked on several posts, but none of them were completed. There are a few reasons for that:

1. I like to make my posts as clean and clear as possible, so I tend to spend a lot of time writing them.
2. The posts I had been working on are quite long, so there is a lot that needs to be edited.
3. I’ve been trying to delve into topics that are not yet “solved” (e.g. parallel programming). I’m still trying to formulate my own thoughts on those topics, so I’m never sure how to present them.
4. I also write on topics that go against the grain of “accepted practice”. That requires compelling arguments to keep people from thinking I’ve gone bananas, e.g. a post about why innovation isn’t always a good thing… See, bet you’re thinking “bananas” already [you'll understand when I get to it].

Anyway, the end result is that I haven’t posted anything.

So, I’m going to get back to it by breaking up long articles into series of posts. That will probably be better, anyway. Who really wants to read a gigantic article all at once?

Now I just have to figure out a good way to link series together so they can be found as a unit (on the off chance you do like reading gigantic articles after all).