Kevin’s Career Upgrade v6.0 – Amazon AWS

I started my career in software development 34 years ago. I was 16 years old and a relative needed a program that helped her with a file classification project at a local hospital. I wrote a really bad piece of software in BASIC that took hours to solve the problem each time it was needed. I refactored the program over and over again until I got it to run in a few seconds instead. The process of refactoring the software to improve it was exciting to me. It wasn’t a battle with the machine. It was a battle in my mind and I believed, step by step, that I could win through intelligence and sheer will. My relative showed the program to the hospital staff and they bought it from me for $50. I was ecstatic. Moreover, I was hooked on software development for life.

Over the past 3.5 decades, I’ve made four major shifts in my career. When I started out, I wrote code in assembly language and C language. If you don’t know what those are, no worries. Just suffice it to say that they are super low-level abstractions and close to the actual hardware. I was really good at understanding the machine architecture and using it to my advantage. I even claimed that for a few minutes in 1984, I knew everything there was to know about the PC. Since then, I’ve been slipping. That’s true of everyone in this field though. There is no person who can know everything about all the complex systems that make up the modern PC and the Internet. I know a lot more than the average developer given my background that reaches all the way back to those low-level language days. But truth be known, I rarely use that old knowledge about electrical engineering and processor architecture to get work done today.

The first major shift in my career, which I’ll call v2.0, happened in the mid-1980s. The out-of-date mainframe pre-compiler I used during the metamorphosis was called C with Classes but the language had already been renamed C++. The idea of object-orientation was dazzlingly cool to me. Being able to hide data inside of objects that expose safe access methods was liberating and empowering. I became a real expert in object-oriented design and in the use of the C++ Standard Template Library, riding that wave for more than a decade to create some very cool software.

Career v3.0 for me came in the late 1990s when Java appeared on the scene. This highly expressive but simpler derivative of C++ promised to make our code safer and portable from one processor architecture to another. I was working with a group within the Intel Architecture Lab (IAL) that was implementing a high-performance Java Virtual Machine for the Intel processors. My team was creating all sorts of system-level software in Java that was going to change everything about the PC ecosystem, we believed. Then, one day, high-level folks from Microsoft visited our campus in Hillsboro, Oregon and spent all day in conferences with our IAL managers. Within days, all of the Java projects in IAL, including the screamingly fast new Java compiler and virtual machine were shut down. It was a real tragedy that changed the history of the PC forever. I left Intel and spent the next few years doing all sorts of interesting Java work that I enjoyed.

In 2001 while teaching C++ at a local college, a student asked what I thought of C# (pronounced C Sharp for those who don’t know). I had no idea what it was so I called a friend in IAL and he explained that the reason Microsoft had abandoned Java (and ostensibly got its partner Intel to do the same) was to make way for a new language that would compete with Java. I was intrigued so he put me in touch with someone at Microsoft who sent me a gold CD-R with “Cool” scrawled on it in red Sharpie ink. (Cool was the project code name for C#.) I popped the CD in a drive, ran the installer and started playing. In the matter of minutes, I could tell that C# was different from Java in some interesting ways. Over the next hour, I fell in love with it. Thus began v4.0 of my career. I dove head first into C# and into Microsoft’s .NET ecosystem. I was probably one of the first college professors to teach C# in the world, filling my first classroom with .NET fledglings in 2002. By 2008, I had become a Microsoft C# Most Valuable Professional (MVP) and stayed in the award program for the next seven years.

At the height of my experience as an MVP, I started work on a very large scale database project. Every software developer works with data but this project was really huge, at least one hundred times the size of any data project I’d worked on to date. I discovered that my skills as a programmer were massively inadequate to operate in that environment. The big problem was that my programming brain was wired to be iterative and imperative. I had spent 20 years telling computers what to do. The problem with really big data is that it’s often so big and so unwieldy that you just can’t tell it what to do. Instead, you must work with languages that allow you to describe what you want done instead of how it must be done. This is called the declarative programming model which is essentially the opposite of the imperative (command-based) programming I’d been doing for a couple of decades.

For the first time, I really needed to use declarative, set-based languages like Transact Structured Query Language (T-SQL) to get my work done. There was really no other way to pull it off in C# (or any other language I knew at the time). My mind was transformed through the process. I could never see the world in the same ways again. My job, which had always been to virtualize the world through silicon, became a search for patterns instead. There were scores of data patterns to be discovered. Code patterns emerged at every turn when I started looking for them. I became a pattern junkie in v5.0 of my career. I even wrote a book with my friend Jason Bock called Metaprogramming .NET which focuses on techniques for generating code based on patterns to make software fault tolerant and adaptive to change.

Version 6.0 of my career starts tomorrow. As I write this, I am sitting in a hotel room in Las Vegas, Nevada waiting for the start of’s 2015 AWS re:Invent conference. I am convinced that most companies will shift their infrastructures to the cloud in the next few years. And I’m sure that the way Amazon AWS products are structured, they will capture and keep the majority of cloud market for the next decade.

The reasons for my beliefs about the success of AWS are complex but it boils down to this. AWS is all about micro-architecture at any scale. That may sound like gibberish to those who haven’t built an enterprise system but it’s demonstrably true that all successful, complex systems are built from small parts working in concert to create value. AWS has generally designed their products in this way, deliberately or otherwise. Each product is a kind of gear that can connected to the others to create really interesting new things. There are serious integration challenges remaining but the AWS products fit together quite well, in general. Moreover, they scale dynamically so there’s appeal to small companies and large ones, alike.

We’re already seeing the leading edge of the conversion as small- and medium-sized companies are testing the waters of cloud computing in large numbers to gain some operational advantages over their larger competitors. In the next five years, the motivation for moving to the cloud will shift from operational, financial and tactical to strategic. Services will become available in the cloud that we could never even imagine in our racks of single-purpose servers sitting in private data centers. Machine learning will give way to what’s next: predictive analytics that permeates all of our data, all of our code naturally and automatically whether it’s shaped and purposed for that or not. Those companies trying to compete by running single-purpose, monolithic software will simply fall behind.

In reality, I started the shift to v6.0 of my career a while back. I’ve been using Microsoft Azure and Amazon AWS for a couple of years for my clients. But I’ve been using these services in the old-school way. When AWS Lambda was released a few months ago, that’s when the light bulb in my mind really lit up brightly. AWS Lambda is the simplest idea. What if you could write functions that could be instantiated in microseconds to get any sort of generic work done? And what if the system could scale out the number of available machines for running those functions heuristically and automatically?

Lastly, what if that system of scalable functions could be triggered from anywhere? Saving a file might run a function. That code might write to a database which invokes other functions. Et cetera. Suddenly, I saw a way to compose complex, scalable systems with simple, interconnected building blocks. Moreover, this is all done without owning a single server in the classical sense. In 34 years, I’ve come full circle from wanting to master everything about the machines I use to not wanting any machines at all. Version 6.0 of my career will be just as exciting as v1.0 was to me. For that, I’m truly grateful to be in this profession.

Let me finish by saying that although I think Amazon AWS will dominate in this space, Microsoft and Google will also do quite well. I’m not ignoring them. But the elegance, simplicity and highly composable nature of Amazon’s AWS products will make them a great choice for my clients. Viva Las Vegas!

1 comment

Leave a comment

Your email address will not be published.