Category: Professionalism

Software Craftsmanship and Heroism

A certain restaurant in the Silicon Valley has a sign which says “This is a bad place for a diet. This is a good place for a diet.” There is no contradiction – just 2 different contexts.

So it is with Software Craftsmanship and heroism. The opposite of heroism in software development is a smooth operation high up the CMMI ladder, where everyone knows what to do and no one wastes time or stresses out because the process is moving forward. On the bottom of the CMMI ladder, on the other hand, it’s sometimes chaos, and a developer can be put under pressure to deliver in a hurry because of bad communication or poorly designed and/or poorly tested code encountering demanding clients.

Software craftsmanship is bad for heroism because it reduces the chaos that requires a heroic developer to step in and fix things in a hurry. There’s a software factory pipeline that doesn’t let logic failures reach the client.

Software craftsmanship is good for heroism because sometimes even with a solid agile process and a finely-tuned software factory pipeline and clean, unit-tested code, misconceptions can slip in and a client can do something that the unit tests did not foresee and (rightly) complain about it. And why is craftsmanship good for heroism in this case? Because the code with the error is clean, extensible and unit-testable. If you have an idea for a fix, you can try it out in minutes in a unit test instead of having to code blindly, build, run and test manually and then repeat the process until it’s debugged or you decide to take a different approach. Ultimately, the software craftsperson has to build and run and test manually, but only AFTER the key idea for a solution has been tested and debugged. So with craftsmanship, you may have some shining moments of glory, but the heroic struggle happens gradually, every day, maintaining the quality of the code and the software factory.

They Might be Giants

Don’t let’s start. This is not an article about music from back in the day, but about the elusive “20X” developer who is 20 times as productive as “normal” engineers.

I recently read the recently blogged guide Recruiting Your Next Anakin by “The Sith” Recruiting Training and Retaining Giants (pdf file) whose author (who calls himself “The Saint”) has earned a fair amount of bad buzz on the web. The TLDR is that you should look to hire people with “a calling” who don’t want “balance” in their lives and then work them 80 hours a week to “season” them (i.e. accustom them to working 80-hours a week for less salary than they’re worth). The guide comes out of the video game sector, where there seem to be a number of horror stories of exploitation. It’s a sector which attracts some programmers who have more talent and passion than maturity and who are thus easy prey for exploitation. I’ll leave it to others to criticize the guide in detail – for a sane perspective on recruiting talented engineers, I recommend Joel Spolsky’s approach.

Here I would rather discuss one of the ideas behind the article in question. The article states: “GIANTS can be worth 10X-20X ordinary engineers”. This is not a new idea, and it sounds very “truthy”, but has it really been scrutinized?

When I think of a GIANT who might be worth 20-times the ordinary people in his profession, rather than any software developer, I tend to think of this tall gentleman:

Madison Bumgarner
In software development, while I’ve seen some great engineers, the paradigm does not seem to apply well.

For the sake of argument, though, let’s assume that there’s some objective measure of individual developer productivity and worth in the context of software teams (I doubt it), that it can’t be “gamed”, and let’s assume there are some 20X giants and that there’s way to hire them, and that (as the afirementioned pdf guide seems to assume) 20X giant-ness is an inherent quality of a young developer which can’t be taught to “normals”.

Imagine that you in a team of “normals” and you are then joined by one 20X giant.

Imagine the sprint planning. The first 200 sprint points go to the giant and then everyone else gets 10.

It’s as if your cellar-dwelling adult-league basketball team just got Stephen Curry. You’re no longer playing basketball – on offense you just give the ball to Steph and maybe (if you get past half-court before he’s drained a three) set a pick or box out in the low post. If you play any defense at all, it’s a box-and-Steph.

Imagine the daily stand-up meeting. The part which gets everyone’s attention is when the giant has an obstacle (probably caused by one of the “normals”).

Kind of demoralizing for everyone, but the assumption inherent in the thinking around 20X giants is that developers, once hired, can’t be improved (except by making them work longer hours if they’re willing and able to).

Fortunately, this is a ridiculous fantasy. In reality, with some effort and intelligence, teams and individual developers can improve. And in reality, if someone in your team is accomplishing 20 times as much as anyone else in the team, something is horribly wrong. Maybe the others are busy cleaning up messes left by the “giant” or decrypting unreadable code written by the “giant” at 3 a.m. when the effects of the energy drinks have worn off.

The software craftsmanship movement embraces the idea that not all developers are at the same level, but part of a software craftsperson’s job is to help the rest of their team to improve. We value the quality of our work as much as the quantity, because software is never really done, and going too fast now will force us to go much slower later. We strive not to be giants among Liliputians but highly competent professionals on teams of profesionals and in a larger community of professionals.

Quick thoughts about Seth Godin’s article “Our Software must get better”

I’m a long-time Seth Godin fan, so I read with interest his recent article about software quality.

With the examples he gives of inadequacy in software – iTunes user experience, the Macintosh’s built-in address book’s slowness and difficulty with importing and exporting data, and general unreliability and lack of Macintosh-compatibility of stamps.com – he’s focused mostly on the user experience aspects of software, which I think misses a key point. I think the biggest problems for software users these days are not slowness, confusion, frustration and extra clicks – though these can be real problems. The big issue for software users today, IMHO, is in the realm of data security, transparency and privacy.

Software in general is getting really good at collecting data profiles of the people who use it. I think this can be done in an ethical, transparent and generally secure way which generates value for the software’s creator in a fair and honest exchange for the user experience that consumers want. Is that what’s happening now? Not enough to satisfy European governments, who are taking action on this issue.

Seth Godin is smart enough and experienced enough to set up a kickstarter for a better Mac address book that is fast and can handle imports. But he would like software users to take ownership of the quality of the software that they use, by explicitly paying for what they use and by insisting on a user experience that meets their needs. On a large scale, that would be a pretty radical change, which would surely have a positive impact.

For the moment, most users either don’t know what “better” is or don’t care enough to pay for it. The trend among the giants of the software industry landscape seems to be that better means “smarter”, which means “knows what you want”, which is a problem in the aforementioned realm of data security, transparency and privacy. If you collect personal information, you must protect it forever and use it in a way which is ethical and transparent. If your smart assistant in the cloud knows that you have an appointment next Tuesday with Doctor So-and-so and the Such-and-such Clinic, who else will know that? The only data which is 100% guaranteed to be private is data which is never collected.

Ironically, the respect for privacy seems to be better in free software (Mozilla, Linux, etc.) than in software which people pay for. Godin alludes to this in his point “C” as to why software is mediocre.

Seth Godin would obviously like us, the software craftspeople of the world, to insist on better user experience, and, ideally, he’s right. It should be part of the software craftsperson’s ethics to improve the user experience. It’s not always clear, though, who should be responsible for defining what constitutes improvement or deciding what time and resources should be allocated for it.

At a minimum, we as developers can strive to create software which is easy enough to change that the user experience can be improved easily once we understand what improvements need to be made. We can also strive to treat the users’ personal data as a precious commodity to be protected, we can sanitize our inputs, and we can use the best encryption techniques available for passwords and other sensitive data.

The Boy Scout Rule vs. the Pottery Barn Rule

Scenario 1 (The Blind Side)

You’re fairly new here, but it you thought it would be a good idea to rename the method “doProcessing” to “updateSalesStatistics”. It’s true, the code is much more readable now. Did you know that our largest paying customer loads our jar file and calls “doProcessing” via reflection? They’re not happy.

Scenario 2 (Danger UXB):

We had to revert your last commit. We’re 2 weeks from code freeze, and you refactored the doProcessing method. That method is 400 lines long and contains little mountains of indentations, and there’s no way to unit test it. It’s too important to risk breaking it now.

With his Boy Scout Rule, Robert C. Martin encourages us as software craftsmen and craftswomen to be brave and clean up the messes we find, at least a little bit at a time. Unless you have a team experienced in and fanatically devoted to clean code, design, and TDD plus a clear shared vision of the functional specifications, technical debt is added to your project on a daily basis. So the Boy Scout Rule proposes that each of us reduce technical debt on a daily basis, hopefully at least as fast as it accumulates.

This excellent approach sometimes runs afoul of its corporate nemesis, which is best summed up as the Pottery Barn Rule: You break it, you own it.

It can create a real dilemma for a software craftsman or craftswoman who wants to do the right thing but can not promise that every improvement poses no risk. We see in the first scenario above that we can be blindsided by undocumented usages of the code which are not enforced by the compiler. This kind of problem is technical debt that keeps on giving. In the second scenario above we see technical debt which protects itself using fear. It’s like an unexploded bomb, but there’s no bomb squad, just you and the people who will blame you if it explodes after you’ve touched it.  If it explodes when left alone, responsibility for the explosion is diffused.

Overcoming these sorts of blockages requires courage, an effective code review process, and what I might call “good management”. According to an extensive internal study done by Google, the number one key to having an effective team is “psychological safety: Can we take risks on this team without feeling insecure or embarrassed?”. That’s partially down to management, but I think it’s also linked to the number 2 key: “Dependability: Can we count on each other to do high quality work on time?” That’s where the developer can make an impact. If most of the time you get it right when you take a risk, the team will likely cut you some slack on the rare occasions when you don’t.

 

Care

With software craftsmanship there’s a lot to say about tools and methodologies, but if I had to sum up in only 1 word what you need to do to be a software craftsman or craftswoman it would be “care”.

As in:

  • care about the people (including your memory-challenged future self) who will have to work with the code you’re writing today – 6 months, 2 years or maybe even 10 years from now
  • care about the people who will be using the software you’re developing today
  • care about the people who have staked their reputation on the product you’re developing today (and about whatever promises you make to them)
  • care about learning to be better at what you do
  • care about what your fellow team members are working on and about being able to give them a hand if necessary
  • care about mentoring your fellow team members and others

Some cynical people think that Software Craftsmanship is a fad to sell training and consulting. There may or may not be a grain of truth to that, but there’s no certificate for caring – either you care, and the results will gradually become evident, or you don’t care, and the results of that will also be evident at some point to somebody. If you do care, there are many free resources, including this blog, which can help you learn about the methodologies and tools you need to become a software craftsman or craftswoman.

 

 

Uncle Bob’s Oath Proposal

Robert C. Martin recently proposed an oath for the software professional. I applaud Uncle Bob’s efforts to encourage higher standards of professionalism, which have inspired me personally, but his oath is based on premises which seem unrealistic to me. The oath has already received much intelligent criticism, from Ron Jeffries, for example. Uncle Bob seems to want us to have a “priesthood” aspect to our profession, but it’s based on questionable analogies.

We are not like doctors and lawyers, who receive quasi-governmental powers in exchange for a difficult certification process, a code of conduct and possibly governmental or quasi-governmental oversight (which, by the way, does not always eliminate abuses of the aforementioned powers and sometimes serves to cover them up). We are not like academics, either, where a “priesthood” is necessary to protect people who invest time and resources into a career path which in many cases has limited commercial value (For example, how many of us would pay big bucks to hire someone to read Sanskrit or analyze Tolstoy or survey the spectral emissions of a distant galaxy?).

Clearly, we have some important issues to address in software development, including:

  • technical debt
  • ethical implications of the products we develop (the potential for fraud or invasion of privacy, for example)
  • risk of project failure
  • handling and security of sensitive data
  • the rapid and continuous proliferation of new tools and approaches (and the need for training/education that this creates)
  • diversity issues (notably gender, race, and age)
  • repetitive strain injuries

Uncle Bob’s oath attempts to address a few of these issues, putting much of the responsibility on the individual developer, who is assumed to receive in exchange for this responsibility the additional autonomy and the time to do his or her job as he or she sees fit. That sounds nice, but software development is often a collaborative team effort, often serving a goal which is not chosen by the developers themselves, so responsibility should be shared.

I think a lot of companies could gain a competitive advantage by treating programmers as respected, autonomous professionals and to provide them with training or time to self-train (and some of them actually do it). However, I can’t support an oath which assumes that all employers do that to the extent assumed by Uncle Bob.

Here’s my humble (rough draft) revision (my additions in italic) of Uncle Bob’s oath, with commentary in blue:

I Promise that, to the best of my ability and judgement:

  1. I will not produce harmful code which is intended to defraud or to harm innocent victims.(Uncle Bob’s version would eliminate from the profession anyone programming weapons systems, for example. My version is more permissive but still covers cases like Volkswagen and Bernie Madoff )
  2. The code that I produce will always be my best work. I will not knowingly release code that is defective either in behavior or structure. conceal defects in the code’s behavior or structure from my employer or customer.(Living with defective software is what made certain software companies what they are today. Far be it from me to say they made the wrong choice, but an ethical developer will be transparent about this choice. Many developers have loyalty clauses in their contracts which would prevent them from announcing a defect publicly, but it’s always possible to inform the product owner.)
  3. I will produce, with each release, a quick, sure, and repeatable proof use the most appropriate techniques available to me, taking into account my employer’s or customer’s wishes, to ensure in repeatable fashion that every element of the code I produce or modify works as it should.(I think the employer/customer wishes and business needs can not be ignored just because there’s a “right way” to engineer, and a developer can’t take responsibility for the entire base of legacy code for a short-term add-on project.)
  4. I will make frequent, small, releases so that I do not impede the progress of others. There is no number 4. (This is micro-management by oath. If a company has a 6-month release cycle, I can encourage them to do 2-week sprints, but it’s not always my call, and they might not be ready to handle it.)
  5. I will fearlessly and relentlessly improve the code I work on at every opportunity. I will never make the code worse without a commitment to correct the worsening by an agreed-upon date.(This one is not bad, if you interpret it such that “every opportunity” gives some leeway to respect the wishes of an an employer who discourages certain code improvements. I added a fail-safe for cases where a short-term need requires a quick hack to be repaired later. I also added a specifier so that I’m not responsible for all the code in the world but only the code I touch. I removed the notion of fearlessness because I know of code that only a fool would change without fear.)
  6. I will do all that I can to keep the productivity of myself, and others, as high as possible. I will do nothing that decreases that productivity.(The first sentence is enough. The second is too inflexible.)
  7. I will continuously ensure share information to facilitate that others can cover for me, and be open to receiving information to facilitate that I can cover for them.(Staffing and how to cover for an employee’s absence are management’s call – a professional who is not a fully independent contractor should facilitate management in this effort but should not have responsibility for it. )
  8. I will produce estimates that are as honest as possible both in magnitude and precision. I will not make promises without certainty transparency about uncertainty.(Honesty and transparency are important for estimates, but no one can eliminate all uncertainty. Some slack should also be built into estimates to account for uncertainty and interruptions and tasks not directly related to the development itself.)
  9. I will never stop continue to learning and improveing my craft.(Let’s not judge the developer who chooses to attend their child’s piano recital instead of going to a coding dojo or a user’s group meeting. We have a long-term commitment to improve, subject to short-term interruptions.)
  10. I will keep informed about security issues and follow important security guidelines as appropriate in order to ensure the security of sensitive data.(This is my own addition. It could be covered under #2 and #9)
  11. I will not knowingly post a complete solution to what is obviously a computer science homework problem on StackOverflow or other public forums.(A recent pet peeve of mine. It’s good to help people learn, but helping someone cheat while spoiling a good homework question for posterity seems like a breach of ethics.)