Saturday, August 12, 2006

The Construction Worker and The Architect

I just began reading Programming Pearls, a book I bought almost 2 years ago after seeing it on a recommended reading list Microsoft sent me after I'd accepted their job offer.

I've been reading and working through the problems in one chapter every day. It's incredibly engaging and fun - a cross between my old computer science exams in college and a more challenging version of those puzzle books I used to pick up at airport news stands as a kid (ok, I admit.. I still pick them up sometimes).

I feel that I'm at the tail end of a generation of computer programmers who were able to witness a paradigm shift in software programming. The first programming languages I learned were gwbasic/qbasic/Batch programming (cool! I can make a computer do stuff!), followed by Pascal and C (needed to mod my bbs!), C++ (high school AP and college classes), Perl/PHP/bash/tcsh (cool! I can make computers do even more stuff super-quickly and build web-apps!) and finally C# (project work during senior year of college, and Microsoft-land).

The paradigm shift I'm referring to though is mainly tied to the decrease in the amount of knowledge required to make a computer "do stuff". With the introduction of simpler languages like Visual Basic, and frameworks like .NET, 80% of the programming ability that was once required to build and end-to-end solution is no longer necessary (I've used Microsoft-centric examples here, but this is undoubtedly a platform-independent phenomenon that pre-dates both VB and .NET).

Even the most complex of operations continue to become simpler. Let's look at 2 examples I stumbled upon recently...

Example 1: Sorting an Array of User Defined Objects

Let's say you have an array containing username strings. Once upon a time, you, the programmer, would write your own sorting function if you wanted to order these strings. Modern programming frameworks have built-in functions to sort arrays of strings, and you don't need to know a single thing about sorting algorithms.

Okay, so now let's say you've created an object (i.e. a struct, class) to represent a "user". It contains a username string, and a whole bunch of other information like a full name and email address. In the not so distant past, you, the programmer, would write your own sorting function if you wanted to order these user-defined user objects (because the programming framework did not know how to sort your object (should they be sorted by username, by full name, by email address?). Modern programming frameworks allow you to implement a comparison function (i.e. IComparable.CompareTo(), which will be used by the framework's built-in sorting functions to sort your user-defined object.

Example 2: Multithreaded Windows Forms Programming

It used to be the case that if you wanted to build even the simplest Windows Forms application that did something time-consuming in the background without locking up your UI, you needed to master the art syntax of making asynchronous calls with delegates and callbacks. With the introduction of the new BackgroundWorker component in .NET 2.0, it's pretty easy to build simple multi-threaded WinForms applications.

So, getting back to my point here, 80% of what once required significant programming ability (i.e. ability to devise a sorting algorithm), now requires significant programming framework domain knowledge instead - something which, in my opinion, is much easier to teach and learn (and it is now probably most of what is taught and learned at trade schools and training classes).

So what about the other 20%?

I believe that this is where the true value of a software engineer still lies - along the lines of the Pareto principle - where 80% of value is brought by 20% of resources. Lots of people out there have the knowledge and tools to be a construction worker, but how many can architect a skyscraper?

At The University of Michigan, I spent 4 years studying Computer Science, and learned nothing about Visual Studio, C#, .NET, or many other skills required to function as a Windows software developer today. In their place, I learned about propositional and predicate logic, set theory, function and relations, growth of functions and asymptotic notation, combinatorics and graph theory, discrete probability theory, techniques and algorithm development and effective programming, top-down analysis, structured programming, testing, program correctness, program language syntax and static and runtime semantics, scope, procedure instantiation, recursion, abstract data types, parameter passing methods, structured data types, pointers, linked data structures, stacks, queues, arrays, records, trees, algorithm analysis and O-notation, priority queues, hash tables, binary trees, search trees, balanced trees and graphs, searching and sorting algorithms, recursive algorithms, basic graph algorithms, greedy algorithms, divide and conquer strategy, instructions executed by a processor and how to use these instructions in simple assembly-language programs, stored-program concept, datapath and control for multiple implementations of a processor, performance evaluation, pipelining, caches, virtual memory, input/output, finite automata, regular languages, pushdown automata, context-free languages, Turing machines, recursive languages and functions, computational complexity, database design, integrity, normalization, access methods, query optimization, transaction management and concurrency control and recovery, and much much more.

Getting back to the paradigm shift... I think the short of it is that abstracting out most of the underlying knowledge and ability to engineer great software is a great thing - especially for professionals who view "software engineering" as a means to an end, rather than the end itself.

But for those out there who are looking to engineer truly original, scalable, and otherwise high quality software, Programming Pearls is reminding me so far that a great set of tools, and knowledge about those tools, can't replace an underlying depth and breadth of knowledge and ability of a true computer scientist.

I'd highly recommend every software developer reading this blog check out the book...

No comments: