Coding Without a Net - The Week of No IDE

I can't deny that most of my programming career has been supported by one or more Integrated Development Environments (IDEs) helping me along.  In fact, it was only my early coding days back in the '90s when I was using Perl in a Linux environment that I worked without one.  It's always been Visual Basic, Eclipse, InterDev (don't ask), or Visual Studio guiding my hands along the surface of the code.  But what if there wasn't an IDE to help me?  Could I go back to the basics and still be productive?

It's uncommon these days for a developer to work completely without an IDE.  While we may cut over to a plain text editor to get some low level work done, we are typically happily pecking away within the square confines of a syntax-colorized, project-analyzing, "intellisensing" application that automatically compiles and often deploys our applications.  They are generally a great boon to productivity.  And then, sometimes, they're not.

My current IDE is Visual Studio. (By the way, did you know Visual Studio has a free version?)  We have just upgraded to VS2015 in my shop and it's looking promising, but VS2012 had a tendency to simply go out to lunch for a while, usually when I was right about to complete a coding task and see it work.  Sometimes, its syntax checking or intellisense would just hang up and refuse to work until I restarted the whole IDE.  And then there are the file management operations that would sometimes take 30 seconds or more, even if it was just a file rename.  Sure, it was probably some plug-in or a specific sub-feature that was causing these issues, but the sheer complexity of the system makes it nearly impossible to track these down.

Most of us, once we get past the basic learning phase of coding -- beyond "Hello World" in C, Java, Python, or whatever -- will start using an IDE and never look back.  And who could blame us?  It helps us create the function signatures, warns us when we use bad syntax or sometimes even incorrectly refer to a class or variable.  Heck, the best ones give us drop downs in context of what we're doing and show us what methods we might want to call on an object, or what public classes are available which start with the letters "Person."  These hundreds of tips from moment to moment save us incalculable time.

But one day a few months back, during a particularly bad morning with Visual Studio, after many freezes, two crashes, and a co-worker suffering the same agony, I began to wonder:  What if I kicked Visual Studio to the curb for a week and made a go of it alone?  Just me and a basic text editor.  Could I actually get work done?  How long would I last?  I began to secretly formulate a plan for a "Week of No IDE"

First, I'd want a text editor beyond just notepad; there's no reason to hamstring myself further.  Thinking back to my Linux days, I decided it might be fun to give my old VI skills a reprise and see if I could still hack with it.  I found a Windows version of gVim that looked promising and grabbed a few VI keystroke cheat sheets to use until I get my chops back.

Second, I'd need to figure out how to do all the other coding tasks (the ones that don't involve actually writing the code) without an IDE.  My two main operations that are not coding are building the project and using source control.  Fortunately, Visual Studio and Team Foundation Services (our source control system) both have decent command line tools for those tasks.
Third, I'd need some human help, because, hey, it's been a long time since I worked without an IDE looking over my shoulder and correcting my syntax.  Having a coding partner would help catch the stupidest of errors and would also subject someone else to this pain... uh... experience.  Misery loves company.  So I asked my co-worker Todd if he'd give this a try with me in a multi-day pair programming session that would last as long as we could both stand it (and each other).  He agreed in spirit, although I think he's likely to bail if things get too hairy.  I won't blame him if he does.  This is above the call of duty.

Fourth, I needed a week where not a lot was going on.  It's not that I don't plan to get some work done, but I don't want deadlines breathing down my neck because I'd be immediately tempted to go back to IDE-land.  As it turns out, we are in the Christmas season, and this coming week is likely to be a bit low-key.  We've just deployed a new update to our flagship product and work on the next phase has just begun.  It's a perfect time to try this temporary experiment.

Check, check, check, and check.  All this leaves is for me and my co-conspirator to give it a go, so starting next Monday morning, Todd and I are going to be pair programming in gVIM for Windows, using the command line Team Foundation Services client, and using the command line Visual Studio compiler to build.

I am also going to be live-tweeting our experiences, trials, and tribulations, at @jptillman (my personal twitter handle) and will roll those tweets up into a post on the Above Average Programmer blog soon after we are done.

Additionally, I want to invite anyone else interested in participating in the "Week of No IDE" with us, even if it's not the same week, to try it and post your experiences here in the comments or in the comments of the tweet-blog I post later.

Even if you don't decide to give the "Week of No IDE" a try, even if you do try and it ends up more like the "20 Minutes of No IDE," I encourage you to raise your awareness of your IDE and how it supports and guides your work.  Is that support causing you to lose certain coding skills?  Is your thinking perhaps too IDE driven?  Do you rely on debug-mode to solve every coding issue?

After some careful thought and experimentation, your answers to these questions may surprise you.  Investigating these questions and answers more closely just might help you on your path to becoming an Above Average Programmer.

Words matter

Maybe it's the English major in me -- yes, I came into software development through the "back door" of web publishing back in the 90's as a writer, not an ordained computer science or IT major -- but I am often struck by how much words matter when you're developing software.  I'm also often surprised how often we coders can remain oblivious to it.

It isn't enough to just skim over the processes of our user community, the parts of their daily lives we are attempting to absorb into the ever-hungry maw of software automation.  Although we may fool ourselves as we walk away from the project launch meeting with visions of classes, interfaces, and object graphs dancing in our heads, we simply can't get away with having only a superficial understanding of what our users need to do.

The required understanding can only be achieved if we internalize and regularly use the vocabulary of our users.  We have to stop saying "insert the Person record" and start saying "receive the initial incident report."  This is because Cheryl in records is right this very minute having an incident report placed in her hands and that is not likely to change between now and when we complete our project.  And the further away from that reality we allow our vision to drift, the further away from success we will end up.

Robert Heinlen, in his wonderful sci-fi novel, Stranger in a Strange Land, coined a wonderful word that perfectly captures what we must do:  We must grok our users' process and their language.  To grok is to delve so deeply into a particular knowledge that we come to intuitively understand it.  It becomes part of who we are and we are able to actually predict behaviors and outcomes based on that intuition.

Some developers and project managers work under the impression that it is possible to "parachute" into an organization, document their processes, and code a working system from specifications written by a third party.  While this may often be required by a misguided client, I have yet to see it end well.  I've seen millions of dollars and multiple years spent on systems that became only marginally appropriate within a few months of completion.  And they never became what the users deserved.

Deserved?  Yes, deserved.  Your users are working hard doing their jobs and you have waltzed into their organization with promises of efficiency and easier access to information.  Every day, they are telling you what they need, what they deserve, but not in the way you would expect.  They aren't speaking directly to you.  They are speaking through the vocabulary they use in their daily activities, among themselves and the people with whom they interact.  They speak through the things they do to get the job done.  Only if you listen to this unspoken dialogue can you truly grok your user community's needs.  And the only way to do that is to walk among them as much as you can.

Domain Driven Design has an explicit acceptance of this kind of understanding.  Used as intended, DDD actually makes your software project into a repository of knowledge about your users' activities.  The "Ubiquitous Language" referred to in DDD, in which you document the words used around your project, is the perfect example.  The "domain" in Domain Driven Design is not an Internet domain, and certainly is not your domain.  It's the domain of your users.  Their world, their rules, their words.

Are you listening?  Can you hear your users and the words that get them through their day?  If not, get out there among them and listen hard.  You'll be amazed at what you discover and how the course of your development efforts can be guided to success.  There's no better way to be an Above Average Programmer than by becoming a better listener, because Words Matter.

The Above Average Programmer (A Presentation)

I gave a talk today at TalTech Alliance's monthly session, sharing my ideas about Above Average Programmers and why someone might want to be one.  The presentation is the polar opposite of what I had originally envisioned I would give, focusing on the high-level aspects of upping your game instead of specific technical techniques.

Here's a link to the Google Slides presentation.

goo.gl/ieNLkJ

I may draw a few topics out of it over time and post expansions here mixed in with the technically targeted articles.

Stand on the Shoulders of Giants

In the quest to become Above Average, it’s easy to give in to the temptation to do everything yourself.  What better way to prove you’re better than average by doing everything yourself, right?  Just one problem:  It’s a trap!  And it’s one some coders never escape.  The plain truth is, you can’t do it all, and you wouldn’t want to.  To raise your programming game, learn to let go of the “Do it All” mentality and instead learn to Stand on the Shoulders of Giants.

So, let’s say you’re building a web application, and somewhere you discover that you need to parse some HTML.  What do you do?  Well, you’ve got 3 choices:

1. Quit your job
2. Write some code that parses HTML
3. Use someone else’s code that parses HTML

Now, unless the programing language you’re coding in has no business being used for web development or you were already on the verge of walking off the job, you’d probably find that choice #3 is the quickest way out.  You wouldn’t write your own HTML parser would you?  Parsing HTML correctly looks deceptively easy, but it’s tricky at best.  It would take some serious guts to try to write your own HTML parser and still get your app coded in this decade.

And yet, this is exactly what I witnessed several years ago, when a Java developer I met decided that the HTML parsing routines provided by the Java language were somehow not good enough for his application.  Today, HTML has progressed well-beyond what the built-in Java HTML parsers can handle, but at the time, what was available would have been more than adequate.  I suppose maybe there was something in particular he needed that none of the available libraries could provide, but it’s very hard to believe that this required a completely new parser implementation, rather than extending an existing one.  Besides, several open-source parsers were available at the time, just waiting for contributors.  

I don’t have to tell you, it didn’t end well for our hero.  Several weeks in, he abandoned the overall effort and just started looking for angle brackets and quotation marks.  So much for home-brew.

Hang around professional programmers long enough and you’re likely to hear someone toss out the acronym NIH, which stands for “Not Invented Here.”  Although it can apply to just about any professional discipline, in the context of software development it’s a distrust  towards code or concepts that they did not create by themselves.  You’ll often find sufferers of NIH hiding behind concerns about security, maintenance, or compatibility.  More often than not, these concerns are really just symptoms of deeper problems, such as mistrust, fear, or an inflated ego, which are just going to hold you back, anyway.  It’s time to let that stuff go.

So how can we effectively use this practice of “Standing on the Shoulders of Giants” (henceforth “SOTSOG,” to save me some typing)?

First of all, it helps to make sure we're asking the right question.  For example, maybe we should first ask:  Do we really want to parse HTML at all?  We answer that by surveying the landscape of our problem.  HTML is tortuously complex with its rules, but there may be an alternative path to getting at the data we really want in that HTML document.  Doing some research on our problem turns up the fact that HTML is a subset of SGML, which is also a predecessor to XML.  Nearly every language worth its salt has an XML parser built-in.  

A little more research turns up a project call HtmlTidy, which while intended to help make HTML more standards compliant, also promises to be able to whip your HTML into XML-compliant shape.  So perhaps, if we are lucky and have some say in the matter, we could run our HTML through the HtmlTidy processor and get XML out the other side.

 

Let’s pretend for a moment, however, that we are the original HTML-parsing Java hero I just brought up and we are stuck with free range HTML, not "tidied-up" XML.  Where are the giants in this particular arena?  Well, a simple Google search for “java html parser” turns up a whole lot of choices, some of which are:

• jsoup Java HTML Parser, with best of DOM, CSS, and jquery

• Jericho Java HTML Parser
• TagSoup
• VietSpider
• And the mother-lode on Wikipedia: http://en.wikipedia.org/wiki/Comparison_of_HTML_parsers

Note that this is just a smattering of what comes up in such a search as of the time of mywriting.  If I was doing a real “market scan,” I’d have a closer look at everything.  Chances are, your favorite isn't here.

Now, just to keep things simple, let's set aside any licensing concerns, and focus just on what these HTML parsing giants have built for us:

• jsoup is a GitHub-hosted project with 21 contributors, and has been around since 2010, with its latest release in late 2013.  It has an active bug tracker and uses StackOverflow as its discussion forum.  They provide a full API documentation set, a Cookbook with examples, and a live web-based demo of the parser’s capabilities.
• Jericho looks like a lone-wolf project (single developer “on a mission”) hosted on Sourceforge, started in 2004, with its last release in late 2013.  Seems to rely on DOS batch files in its release, which if I were a non-Windows developer might be a turn-off.  Sourceforge stats show decent utilization and the developer responds to tickets and posts on the forums.  Example code provided.
• TagSoup looks to be self-hosted, with a Google Groups discussion forum that shows little activity.  Can’t tell its age, although it claims to have had 20+ releases.  Site disclaimer states it won’t build on the newer versions of Java without adjustments.  Focuses on forgiving parsing techniques and uses an event-oriented approach called SAX.
• VietSpider appears to have been a loose match on my search, and seems to be a full-fledged web spider, rather than a low-level HTML parser.  Looks interesting, though.

Based on the (admittedly cursory) analysis above, I’d probably go with jsoup or maybe Jericho for my first round of evaluation, unless I needed something more happy-go-lucky like TagSoup.

So, what did I gain from this little exercise?  Well, I know there are a heck of a lot of HTML parsers out there for Java, several of which have a huge amount of coding time invested.  Their community sizes are nothing to sneeze at, either.  What’s more, I learned that there are different approaches to parsing HTML (I saw mentions of “nodes” and SAX) and I also found that people are writing their own web crawlers, which may come in handy later if I need one.

Just these few minutes of time have given me a much broader understanding of what’s out there and I also have some great candidates for my project needs.

And, finally, perhaps the biggest win of all when practicing SOTSOG:  I don’t own the code.

Why’s that a good thing, you may ask?  Simple.  The more code I “own,” the more code I have to worry about.  Sure, all code can have bugs, even code written by geniuses and reviewed by thousands, but it’s a darn sight more likely to work than code I wrote yesterday, and if I choose wisely the giant’s code I find is pretty much guaranteed to get me to the finish line more quickly and with fewer bugs.  And that means fewer support tickets or phone calls.

If the practice of SOTSOG was good enough for Isaac Newton, then it deserves serious consideration.  Give it a try, and you’ll find that it does more than help you deliver on time and keep your phone from ringing.  It will broaden your professional horizons and help make you an Above Average Programmer.

Use a Time Machine

Above Average Programmers can travel back in time.  At least, that is, they can travel back through their project’s history.  This is because all Above Average Programmers use “Version Control” when they work.  Version Control Systems (or VCS’s) give developers the ability to pull up any previously recorded version of their source code, right back to the beginning of the project.  What’s more, it lets them keep track of their current work more accurately, and makes it easier to work with others.  It’s even useful for stuff other than code.

VCS comes under a variety of names:  Source Control, Revision Control, Software Configuration Management (a larger topic).  Sometimes, teams just refer to it by the specific implementation they use:  CVS, Subversion, Git, Github, SourceSafe, TFS, etc.  No matter what it’s called or what the implementation is, you’ll find that you’re almost always better off with it than without it -- yes even the crappy ones.

Consider the scenario in which you are working on the next release of your code and suddenly a bug report comes in about a problem with the last release.  While you might fire up your IDE and start bug-squashing on the current, unreleased build, how can you be sure you haven’t already fixed the bug in question?  If you’ve touched the suspect section of the code since the last release, how can you even be sure that you are inspecting the right lines of code?

If you’re not using VCS, perhaps you were smart enough to create a working copy of the release version of the code for just this occasion, which is great.  Now you can confidently view the source that you know is running in release and fix the bug.  Good.  Now what?  How do you carry that fix forward to the new work you’re doing.  If the code has been modified heavily since the last release, that’s going to be one heck of a job to merge the bugfix in (although there are merge tools out there which could help).

Having a VCS makes maintaining multiple working versions of your code much easier than managing it yourself with copies.  For example, you can have a “release branch” of your code for bugfixes and a “main branch” for the upcoming release, both with their own tracked, inspectable set of changes.  It will also facilitate the merging of changes, not only from “main” to “release,” but also in the other direction for those times when you make a quick bug fix on the release branch and want to backfit it into the “main” branch where you are doing your major development.

Here’s another situation to think about:  What if your co-worker wants to help out with the code?  Sure, you could throw the code up on a fileshare and both hack on it simultaneously, or you could send zip files to your colleague every day which contain the latest version of the code, and then manually merge his or her changes back into your “master” version every day.  Both of these are fraught with trouble such as accidental overwrites, locked files, and time-consuming drudgery.

With VCS, you can both simply hook into the “repository” of source code, get your own copy of the code, and begin “checking in” your changes.  The VCS will automatically merge what it can figure out on its own and alert you to any “conflicts” resulting from complex changes you both make to the same code.

Any good introduction to VCS will also show you how to branch off from your main version and work on specific things in your code with the intention of merging those changes back in later.  This allows you to work on complex features without disrupting the main codebase for other team members.

VCS is so powerful for supporting collaboration that I believe it deserves a lot of the credit for the open source revolution in software.  If it weren’t for the easy source code management that VCS provides to distributed teams of programmers, open source would probably never have gotten to where it is now.

Here’s an extra thing to consider:  VCS can also version control things besides source code.  In fact, it works just fine with binary files like images and office application documents.  If you find yourself wishing you had previous version of files in your home directory, it is perfectly feasible to set up a VCS repository for it.  I’m not the first person to think of this:

• http://www.storelocatorplus.com/git-version-control-your-home-directory/
• http://unix.stackexchange.com/questions/1875/tips-for-putting-under-source-control

Convinced?  There are many introductions to Source Control on the web and they will get you there faster and with more flair than I ever could.  Some of my favorites are:

• Joel on Software, What is Source Control?
  http://discuss.joelonsoftware.com/default.asp?W190
• A Visual Guide to Source Control
  http://betterexplained.com/articles/a-visual-guide-to-version-control/
• An Introduction to Version Control
  http://guides.beanstalkapp.com/version-control/intro-to-version-control.html

Check these articles out and see what you’ve been missing.  Maybe it’s time to tap into the power of VCS as part of your journey toward becoming an Above Average Programmer.

What is an “Above Average Programmer?”

So what exactly is an “Above Average Programmer” and why on Earth would anyone want to be one?  Why shoot for simply “above average” instead of “rock star” or “genius” or some other superlative?

The answer is really quite simple:  An Above Average Programmer is someone who builds software and who has applied some simple, no-nonsense techniques and principles which raise their mastery and the quality of their work to an “above average” level.

All right, you’re thinking, That answers the “what.”  How about the “why?”

Most professionals who have worked in their field for a while come to realize, assuming they didn’t know going in, that they are not going to be the “rock star” of the profession.  We use “rock star” in its widest meaning here, including both notoriety and mastery of the craft.  The vast majority of musicians do not play Carnegie Hall.  There are always more NFL players who have not played the Super Bowl than those who have.  For every Mark Zuckerberg, there are thousands of unknown programmers working every day on software applications which will never be seen outside the four walls of the organization that issues their paycheck.

But I’m not being a kill-joy here.  Reach for the brass ring if that’s what you really want.  Strive for super-stardom.  But in the meantime, there’s a living to be made.  Besides, on the way to god-like powers you’ll have to cross the mean at some point and thus, by definition, become Above Average.  So, anything that gets you there sooner can’t be bad, right?

All dreams of glory aside, the real reasons most of us would want to be Above Average are a bit more pragmatic:

Looking for a raise?  There’s no better way to get it than to be better at your job than 50% of your peers?  

Want to survive that next round of company layoffs?  Well, then it’s best to not be in the lower half.

And finally let’s not forget the one thing that nearly all of us, greenhorns and oldtimers alike, crave:  The feeling of a job well-done.

That’s the stuff:  To have the confidence that we’re not merely doing the job, we’re doing it well and getting better.  Even if the honeymoon was over a long time ago -- even if you’ve got more in common with Dilbert these days than you’d like to admit -- there’s still a fire inside waiting to be rekindled.  That’s what being “Above Average” is all about.

It’s not about being the “best” (whatever that means).  It’s about being your best and finding that, lo and behold, your best is actually pretty darn good.  In fact, it’s Above Average.

O.k., you say, we’ve covered the “what” and the “why.”  Can we get to the “how”  already?

Sure, that’s why I’ve created this blog.  I’m fully aware I’m not the best in the world at what I do.  Heck, I might be only barely above average.  But over the years, I’ve seen the good, the bad, and the ugly.  I’ve seen things that work, and things that definitely do not work.  Software development is a confusing, mixed up mess sometimes.  But when software works, it is a thing to behold.  At this point in my career, I think I can identify Above Average when I see it.

I’ve also seen so many of my fellow software professionals struggling with their craft, laboring in mediocrity.   They’re held back, not just by the things outside their control, like insane management, skimpy budgets, and sub-par tools, but also by things they can control.  Simple things that have the power to lift them into the realm of the not-so-average-after-all.  And that realm, my brothers and sisters, is a very fulfilling place to be.

This blog, then, is a way for me to share what I’ve learned over the years and (I hope) to give a hand up to a few of my colleagues.  For the most part, I’ll just be taking undeserved credit for showing you what you already have within you, with some concrete recommendations thrown in for good measure.  A few of my friends will also probably stop by from time to time as guests to share some “Above Average” things they’ve learned, too.  

So, if you’re currently feeling just about average (or below) and are looking to raise your game, have a look around my blog.  I think you just might find something that will help you.  Perhaps you will find some surprisingly simple things you can do right away.

And remember, it doesn’t matter how old or young you are, or how long you’ve been in the biz, it’s never too late or too early to become an “Above Average Programmer!”