Have you ever come to the realization that, looking back, you were in the right place and time to have gotten on board with the next big thing but instead badly missed the boat? It’s a familiar feeling to many software developers, maybe especially those of us who started in the nineties or earlier. Lots of successful products were developed and sold by very small teams or a single person, so the belief that “I could have done that” isn’t too delusional, given you had decent skills and energy.
And I’m sure aspiring entrepreneurs and inventors have seen infomercial’s or learn of a product at a trade show and not only say “I could have done that” but “I actually thought of that, mine would have been better, though.”
Thre varieties of missing out:
- Didn’t recognize a great idea even when it was right in front of you. Ten years later it’s as big as the Internet.
- Thought of an idea or was given one you liked but after initial enthusiasm, you set it aside or procrastinate on follow-up. Two years later you discover something just like your idea out there, doing well.
- You jump in, try, and fail. While you didn’t succeed, other similar products do, proving you were on the right track. If only you had known for sure you had a winner, you think you would have tried even harder.
Closer examination of the circumstances of a “missed” opportunity, though it may feel bad at first can help you put regret to rest.
Here are five stories from my own life as a developer.
I Missed the Web
One day in 1992 while visiting one of the more reputable FTP sites I liked I found something called “www-client-xxx” something something. I really don’t recall exactly, but I believe it was this, coupled with a file of public web addresses. The README on the site described it as some program for accessing and searching the “World-Wide Web,” whatever that was. The interface was bad on this client and the whole system felt really awkward.
I didn’t see the point.
For those of you who weren’t there or weren’t paying attention, the web really took off in 1994 (or as early as mid ‘93 or as late as later ‘95 depending on how plugged in you were.)
If only I had understood what I had on my hands. Unlike most people I was both a heavy user of the Internet (pre-web) and possessed the skills to make a really nice looking PC web browser with graphics that didn’t suck – I’d just been working on some high performance PC graphics libraries. The quality of the source code I would have written, on the other hand… Still, I could have done it if sufficiently motivated. Whether the browser I would have made could have gotten widespread use is doubtful but it could have led to better things for me.
After a day and a half I deleted all the files for this “WWW crap” off my hard drive and removed the “.arj” archive (remember those?) and moved on. Oh well.
TLDR: Perfect timing, didn’t appreciate the opportunity
I Thought of a Multi-Million Dollar Idea
…and built it too, but other parts of the plan were lacking.
You know those recovery CDs that used to come with laptops? The discs that will recover the initial state of your computer in a matter of minutes? I thought of that (I was far from the only one however.) It wasn’t exactly a flash of genius: combine computers from 1995 with Windows 95 and you’ve got a problem – installs took forever.
If you were running a technical support center you had to help customers install Windows 95. A lot. And it took forever. Want to save money? Speed up the install process and make it fool-proof while you’re at it.
Manufacturers had just started including compact disc drives in every PC and nearly every laptop. CDs had the capacity to hold the operating system install software, so I reasoned why not store a completed installation? It would just require imaging a clean hard drive on the model of laptop or PC in question, saving only the used part of the file system, and compressing while you’re at it. Then you would print CDs with this image and include them in the box with the driver disks and other materials.
Not only do you save time on the setup, but since you’re imaging a specific configuration, you can iron out all the various driver and other quirks ahead of time (there were a lot of these back then.)
You could license the technology to any PC builder / OEM, or even independent computer shops. I worked out the potential for saving on support on the part of the licensees, and what (in my mind) a reasonable license fee would be; it was a lot of money. The value proposition was clear and convincing.
So I went and built it. I read up on FAT-16 and FAT-32 Microsoft file systems and the ATA hard disk interface (the product required filesystem drivers as well as partial low-level hard disk interface drivers.) I came up with something really fast and reliable, very user friendly, though not totally polished.
These days, rather than recovery discs, you will often get a “recovery partition” on your hard drive. My software (called eSM: Easy Setup Media) would actually create these as well, and I made one on my own PC.
What was the problem, then? By the time I’d started on a prototype I learned about GHOST (General Operating System Transfer.) It made images of hard disks, usually placed onto a shared network drive for an IT department to use when setting up employee machines. Obviously this did essentially the same thing as my software, minus the idea of putting the image on a bootable CD. (But those were soon to appear.)
Soon GHOST and its maker Binary Research got sold to Symantec. They kept adding features, more than I could keep up with. NTFS was – I partly incorrectly assumed –going to rapidly replace FAT and documentation was almost non-existent, plus NTFS was much more complex. I’d have to support that and so much else. No way to keep up.
In the end though, technical challenges comprised only a small part of my problems: I simply didn’t know how to sell it. I knew something about the market but certainly had almost no industry connections. Though the project needed more developer time than I had, if we could have pulled in some money on initial sales or raised investment (even a little) that would have been solved. What I lacked was a complete business plan and non-technical skills. Sure, I knew how to write a business plan, but not how to execute it.
Then the 2000 bubble burst (March 2000 to be precise) and everything slowed way down.
TLDR: Perfect timing at first, good concept, good software. Under-capitalized, missing the right team, sudden recession
Developed a File Format for “Big Data”
It was ten years too soon and nobody ended up using it including myself. This was a case where I saw a very specific need and met it; but to make the format (columnar tabular data) really useful for other users hardware needed to improve a bit, and more than that, an ecosystem of data tools needed to grow up around the format.
The columnar format I developed in 2003 certainly saved space and time: About 60% smaller than CSV or fixed width data files and orders of magnitude faster to access a few columns at a time.
Compared to similar modern formats, my format was inflexible with respect to nesting data in columns and was not as robust. It only supported fixed byte size data types. Due to slower processor speed at the time compared to today, compressing individual columns was rather expensive, so my version only used a sort of bit-packing approach.
This is a case where I wish I’d have really believed in my own idea. I could have built a query engine of some sort to access it and some tools to convert to and from CSV (I had something but it was only a prototype.) However, because there wasn’t a critical mass of tooling and users around the format I lost interest even though it was my own idea.
One of the big problems a columnar format like Parquet solves is the issue of querying “big data” and distributing that data to nodes in a big data cluster; those concerns didn’t exist in 2003. (Well they did but most of us hadn’t ever encountered the concept as a best practice / well understood principle.) I was motivated mostly by the goal of making a faster data extraction system for IPUMS micro-data. It did succeed in meeting that goal in a limited fashion, but making it part of the main data pipeline required too much work so it got sidelined.
There was never a thought of making money with this idea. I did contemplate releasing open source code but never got around to it. For me, success would have been using the columnar format more widely at my employer and perhaps getting the code out to the world as a library.
It wasn’t all a waste of time: Having designed such a format and supporting libraries I easily grasped the reasons behind design decisions in Parquet and other column-oriented storage approaches.
TLDR: Wrong timing (too soon,) good concept, decent software. Future compelling use cases unknown, so it was a hard sell even to myself.
Rejected the Idea to Build an Online Dating Site
In the year 2001 my friend Andy and I were at a bar throwing around software business ideas. He pointed out how a person could set up a site using PHP-BB or similar systems. It wouldn’t be too difficult to throw together an online dating platform, for example.
I rejected the idea hard. How could we compete with Match and Craig’s List? There were a number of other systems out there even in 2001; it felt like the market was already flooded with dating sites.
Now, to the average person that may come across as a strange objection, as the number of people using online dating systems was really quite small compared to today. It was a case of being too familiar with the internet and not seeing the larger trends at work. Also, the 2000 tech bubble had just burst and perhaps I was feeling pesimistic.
Apparently I believed Internet use would not grow at all from that time forward, and social life would not change at all after 2001. Of course these were unconscious unexamined assumptions.
Instead I should have thought about the question like this: If more people used the Internet in future we could get customers not by stealing them but by new people seeking out dating sites. And if more people already on the Internet found online dating acceptable we could get users that way as well. As it turned out both trends happened in a big way.
Even in 2001 it was clear that there were many online dating niches; no one company was totally dominant. If we could provide a better or unique service in some way we could have had a chance.
As we know now, many more dating services popped up over the years and many were successful: OK Cupid, Plenty of Fish, Tinder to name just a few. Plenty of Fish, a basic PHP application written by a novice programmer, became really successful in 2009 or so and is still out there.
I’m not too regretful. It’s just amusing to look back and realize how wrong I was.
TLDR: Believing incorrectly all the exciting growth was behind us.
Part of a Social Media Start-Up
It launched in 2008, needed more capital, couldn’t get it because 2008.
Unlike the other stories here this wasn’t my idea in any way. It’s interesting though because it’s such a great example of bad timing. The business concept seemed excellent. Though the team was young, they had enough of the right people and some business skills and connections.
A cut-rate offshore firm produced the initial mostly unusable version of the web site in 2007 using some pretty exotic PHP with no discernable framework. I came in around the new year along with a couple of other developers to get it working. Quickly we determined we’d have to over-haul most of the code and build a database from scratch.
This we did, and had a mostly functional site in a few months. It served as an MVP but we knew it wouldn’t scale. Scaling would involve not just handling lots of traffic but internationalizing the system and effectively hosting ad campaigns on the site. In our case both of those were important enough to the core business that we could not stick with the MVP for long.
We began a total re-write at the same time the founders pursued more funding. Unfortunately this was the fall of 2008: Total financial meltdown, no credit available. The word from Silicon Valley was not to even bother making the trip out there. Of course that soon would change but not fast enough for the business.
The re-write was shaping up well but in the face of diminishing capital we pivoted to build a Face Book application instead; FB would provide a login system and lots of data on our users essentially for free. Plus being on Face Book gave a way for our app to spread to millions of users in theory if they shared it with friends.
This was just when Face Book was getting really big. When the project started the notion of creating a social network from scratch didn’t seem crazy; by early 2009 it looked increasingly unrealistic so the move to Face Book was probably inevitable.
Unfortunately they had no more capital available and converting the stand-alone app to FB would be a lot of work; the alternative, keeping the stand-alone web site up to support advanced features and use the FB app for collecting users wasn’t viable either.
TLDR: Great idea, terrible and unlucky timing, needed more capital earlier and better technical guidance early on.
Confronting Failure
It’s natural to latch onto feelings of regret when these sorts of missed opportunities come to mind. However, if we allow ourselves to examine the situations honestly and really think them through we’ll probably conclude that they aren’t so black and white. We may just be fooling ourselves that we could have pulled something off; and on the other hand even a “failure” can teach us something, even if that’s only a technical skill and not a general life lesson.
Rather than dwelling on regrets, can we learn from these experiences? Ask yourself these questions next time you get what feels like a “big idea.”
Can you Pullit Off?
Sure, you might understand the technology, you could build a product, but can you build one that’s good enough? Can you build it fast enough? Can you turn your product into money?
What do you know about marketing and sales? How about raising capital? If financial success is part of the deal, make sure you’re realistic about your ability to execute every part of what’s needed for success or enlist others to help. Know when to look for a co-founder if you’re starting a business.
Understand that even having done all this nothing will go to plan.
All that said, you can’t have all the answers immediately. Sometimes diving in is the only way to figure out what you don’t know.
What are you Waiting For?
If you believe in your idea, and you can do it then what are you waiting for? Either you’re lazy / afraid of failure, or perhaps you unconsciously know there are steps you’re not equipped to take, but you like remaining in the state of mind that says “I have this great idea and I’ll make it big any day now.” There comes a point when more planning and dreaming add nothing to your odds of success.
Obviously, outside constraints (demands on your time, no money) will prevent you from embarking on a solo project; but perhaps you could hand off the idea to someone else and contribute as much as you can. Keeping it to yourself won’t do anyone any good if you’re not doing anything.
So Why Start Anything:
When thinking about the big thing I “missed out on,” I realized there were a couple of things I did in fact catch onto early and even did my best to follow through on; though they didn’t make me money or fame at the time pursuing the projects, I’m glad I tried.
- Skills I used developing the disk imaging software helped sell my resume to my current employer. Nobody else had anything like it, I’m fairly certain
- I published my libraries behind the disk imaging software and quite a few programmers used it in their own software in the late nineties.
- The graphics libraries I wrote that I could have used to build a slick PC web browser helped sell my resume to several employers
- Having an intimate understanding of column oriented formats and how to use them has served me well for years as a back-end developer, most recently in the development of an online census data tabulator.
- Just knowing a thing is possible before most people do can be a big advantage (columnar data; copying and manipulating disks at a low level.)
- Starting a business once lowers the psychological barrier to doing so again; it’s just a series of practical steps you take. My previous experience helped a lot when going into real estate even though it’s a totally different business. Some people have a mental block surrounding the business start up process. Once upon a time I was one of them.
- Good stories