Sunday, December 30, 2007
Code Artisan's New Year's Resolutions

Well, it's a new year, so time to sit down and draw up a list of development resolutions. Without further ado:

1. I will create unit tests for all my classes. There's no excuse not to have at least some kind of unit test for each class that gets created; this at the very least provides a stratum on which other people can flesh out the tests to cover things that come up. This may not be full-blown TDD, but we could at least enforce this with some kind of unit test coverage tool like Cobertura.

2. When debugging a defect, the first step will be to write an automated test case that captures it. Nothing is more annoying than a bug that can't be reproduced. If you can actually get to the point where you can reliably trigger the bug, you're probably most of the way towards finding the problem. Furthermore, in the process, you'll have created an easy-to-run way to know when the bug is fixed. Since the test is automated, there's a good chance that test will get re-run a whole bunch more times in the future of the codebase, making sure that once a bug gets squashed, it stays squashed.

3. I will write Javadocs for all my public methods, except where they can be inherited from the Javadocs in an interface. This is just common courtesy to myself and the other developers. I'm cutting myself a break here, since one could argue that you really ought to document the protected ones too, but we'll start small. This will also encourage definining interfaces and coding to them, as it saves a lot more effort if you can just Javadoc up the interface itself! For bonus points this year, write or find a tool to automatically check this.

4. I will have explicit up-front design sessions before writing any code. I really don't know why we don't do this more than we do, other than we forget to do it. There are all kinds of good reasons to do this, like the fact that we always come up with better designs in a small group than we do by ourselves, or that this will help all of us become better designers by being present while good designs are being born. Or that a good design is usually easier to implement, and with all the hard work I've already set out for myself, I don't mind finding resolutions that save me some effort.

5. I will relentlessly learn to estimate better by considering my past estimating history when providing a new estimate. First step here is to actually look at my past history, so I know what it is! (Fortunately, we are already collecting the data for this...) Then possibly use techniques like those I laid out in a previous post about improving estimates.

Well, that's it. Those are some pretty specific resolutions, which, I'm given to understand, makes it more likely that they'll be adhered to. What are your coding resolutions for the new year?

Thursday, December 27, 2007
Who designed this stuff, anyway?

Many development organizations have an "architect" role, meant to provide technical direction over an entire engineering process [As far as I can tell, "architect" really seems to be shorthand for "someone with deep and broad technical understanding and experience"]. How does this mesh with scrum-based, grass-roots development by empowered teams? Or perhaps, the real question is: if I have folks that would traditionally fill an architect role, what's the best way to utilize them in a scrum setting?

Three areas in which these folks tend to contribute are (although this list is certainly not exhaustive):

  • R&D: advanced prototyping/proof-of-concept work
  • Due Diligence: evaluation of emerging technology
  • Design: participating in system and/or software architecture and design

Let's take each of these areas in turn.

R&D.

Here the notion is to invest some amount of exploratory effort to remove technical risk from a product or feature--to the point where the feasibility of an approach can be established and an associated rough LOE (level of effort) can be given for a real implementation using this approach. We typically hand this type of task off to an architect, hoping his/her broad technical background can help provide hints/leads to navigating the solution space, and thus reduce maximum uncertainty with a minimum of effort (bang for the buck!).

Depending upon the nature of the problem at hand, and/or the structure of the available scrum teams, there are two readily apparent approaches:

  1. run an R&D scrum team; this makes sense when there are sufficient architects around to form a team of them, and when the explorations are not directly tied to an existing product (e.g. technical feasibility for a new product)
  2. embed your architects on the scrum teams themselves: this makes sense when your architects are more scarce (because it will be beneficial to have them on the teams for the other reasons we'll mention below), but also when the explorations are directly related to an existing product. In this case, the rest of the scrum team they are on will benefit directly from (and help participate in) the exploration--this learning process will make a later production implementation easier, and will furthermore help teach the other team members how to do this type of work. This is vitally important if we are short on architects and good ones are hard to hire--then we need to view these other folks not as "non-architects" but rather as "architects-to-be." This model is also easy to follow if you are already using the idea of prototyping sprints.

Technical Due Diligence.

Here, again, we are counting on our architect's experience to enable a thorough and accurate assessment of some technology. If there is already an "architecture scrum" running as described above, this might be a natural place to fit these efforts. However, in the absence of a dedicated group, it may fall on an architect that is embedded on a scrum team (ideally one working on a product that might be impacted by the technology in question), which would also be perfectly fine.

Design.

Architects' experience helps most during design exercises--knowing what kind of pitfalls to look for, what kind of generality to build in, and what kind of approaches/patterns are available and relevant. Design is also an area where collaboration is both possible and usually helpful (at least up to the amount of people that can fit comfortably around a whiteboard!). So, no doubt--we want our architects participating here in order to get good designs up front and hopefully minimize refactoring later.

Again, I am going to argue for the embedded approach here, as opposed to running an architecture scrum that spits out designs. I think the benefits here are overwhelming:

  • by generating designs as they are needed for feature development, effort will not be spent building in generality that may never be exercised
  • by having the scrum team participate in the design, those familiar with the details of the existing system will have input, ensuring the design is complete
  • by having the team participate, they will learn to become better designers themselves (again, a key benefit if there are not enough architects to go around--and there probably won't ever be, by the way--as the architecture will have to live and grow regardless of whether an architect is available to design it all).

Archi-who?

Well, I seem to have convinced myself, anyway, that the right thing to do with your architects is to just put them on your scrum teams and let them have at it (Ken Schwaber would be so proud!). In other words, don't treat them any differently than you would any other engineer in your development organization (indeed, in our current organization, we have "architects" that do production development, and "engineers" that do software architecture, so we're probably not far from this anyway). The main benefits of this approach are:

  • relevance - by staying grounded in the day-to-day realities of creating products, our architects will be focused on real needs
  • integration - by having architecture activities happen in the context of normal scrums, those activities will have a high likelihood of impacting production code
  • collaboration - as a team member, there are no artificial barriers to communication, no "us" and "them" mentality between architecture and engineering
  • resourcing - architects can leverage their teammates to provide extra horsepower towards architecture tasks
  • learning - by having scrum teams responsible for architecture tasks, experienced team members will teach less experienced teammates how to accomplish architecture-related tasks

Tuesday, December 18, 2007
What's the APR on your technical debt?

Today we're going to discuss technical debt--what it is, what its impact is, and what to do about it.

One of the key tenets of scrum and other agile development methodologies is the idea of producing production-ready increments of work at the end of each iteration. Ken Schwaber calls this "sashimi" in the scrum framework--a slice of product. You may have also heard scrum practioners talk about the definition of "done", i.e. what steps have to be completed before we can consider a user story satisfactorily completed, which may include steps like:

  • code written!
  • unit tests written and passing, with a certain degree of code coverage
  • automated acceptance tests written and passing
  • code refactored
  • code documented/commented (e.g. javadoc)
  • product owner signoff
  • documentation updated (e.g. UML diagrams)
  • etc.

In general, the notion is that the definition of "done" should capture the degree of code quality the team and product owner mutually agree to.

Now, anyone who has tried to do this has probably run into this (as we have): a sprint gets started, and at some point through the sprint, you realize that you're not going to be able to finish all the user stories you signed up for by the end of the sprint. Or at least not get them all the way "done". So, you yield to the temptation and cut corners, just doing the implementation without all the rest of the supporting infrastructure, so that at the sprint review you can call this done. Oooh, it's so tempting! C'mon, the product owner isn't going to look at your CruiseControl outputs....

But now, what you've done is that you've incurred technical debt. All that code doesn't have automated test coverage or adequate documentation, and probably has more complexity than it needs because you didn't get a chance to clean it up and refactor it. Now it's going to be harder for someone else to work on it (including you, two months down the line!), because it's inadequately documented and complex, and it's even hard to refactor it without introducing bugs because you don't have the automated tests to help you know if you covered everything! Thus, future work on this codebase gets bogged down a little bit by all this cruft.

Now, technical debt works just like credit card debt: if you keep racking it up, it really starts to kill you. Because you are now working slower, you again find yourself behind the 8-ball of not being able to finish everything by the end of the sprint, and you rack up even more technical debt. Our product owner recently commented to me that he felt like we were getting less done than we used to at the beginning of the project with fewer people, and I'm starting to think it's all this accrued technical debt.

Unfortunately, getting out of technical debt is pretty similar to credit card debt, and is also probably psychologically just as hard. The first step is cut up the credit cards -- refuse to accrue any more technical debt from this time forward, and instead only demo things at the review that are really fully "done", even if this means you don't get to everything you thought you would. This takes a certain degree of courage, and is probably something to do right after a major release, rather than just in front of one....

Then, start paying down the debt. There are a number of ways of measuring this debt, many of them easily automated with Maven, such as:

  • Cobertura - unit test code coverage measurement
  • PMD - static analysis to identify unclean code patterns, including code complexity and design issues
  • Javadoc output - how many of your packages are missing documentation? how many classes don't have class documentation, and how many of the public/protected methods are missing javadocs? Do you have any javadoc warnings?

So the point is, you can measure all of these things, and come up with some numerical measure (pick some formula you like) of your technical debt. Furthermore, if you are tracking the user story velocities of your teams, you can actually start correlating your technical debt measurements at the beginning of a sprint with the story point velocity output by that sprint. In fact, you may even be able to figure out exactly how much drag your technical debt is putting on development, which means, based on your average cost for developer time, you can actually put a dollar amount on it. You can actually tell how much interest you're paying on your technical debt in real dollars.

Then see how the conversation with your product owner goes when you ask for some technical debt paydown to get prioritized on the product backlog....

Wednesday, December 5, 2007
Scrum thoughts: Improving individual estimation

In my last post, I described that our team routinely overestimates the amount of work it can finish in a sprint. The amount of this overestimation is quite high at times - sometimes we only finish two-thirds of what we initially plan.

So how can we get our estimation better? I think there are two sources of estimation error: one is how many hours per day you actually have to apply to sprint tasks; the other is how many hours it will take you to actually finish something.

In his article on Evidence-Based Scheduling, Joel Spolsky describes how we can compare the history of a developer's estimates vs. the actual time required to get an error factor in their estimates. I think this does a good job of addressing the second type of error, but you need to account for both if you are going to get an accurate estimate of how much you can do in a sprint.

For example, as a team lead, I often get interrupted while coding when someone asks me a technical question, or to participate in a discussion on the dev mailing list, etc. I usually "leave the clock running" for these things, so I think evidence-based scheduling would cover this kind of inaccuracy in my estimation. However, I also get scheduled to attend extra meetings which don't go towards finishing sprint tasks, and I don't leave the clock running for that. I also sometimes forget to log all my time spent against tickets, even though I update the remaining time on the tickets to make the burndown accurate. So that's another source of inaccuracy.

Fortunately, I think there's a pretty simple solution that covers all these things. Let's assume we recorded in the past sprint (or past few sprints) how many working days someone was going to be available. Then we simply take the time logged against tickets that were originally planned for in the sprint and divide that by the number of days worked to get a per-day burndown figure. If someone works nights and weekends, they'll just have a higher per-day burndown rate. If someone routinely has other duties that keep them off sprint tasks (e.g. doubling as a scrum master), their per-day burndown will be lower. If someone forgets to book time, their per-day burndown will be lower. If they routinely forget to plan for required elements of a task or if the product owner injected some new requirements mid-sprint, the per-day burndown goes down.

Now, once you know that, you can use the evidence-based scheduling trick from Joel's article to use the actual task estimation history (here's where we compare original estimates vs. actual estimates for finished tasks) to put an actual confidence level on how likely that developer will have a set of estimated tasks done by a certain time.

So, during planning, you let someone estimate hours as if they would get to spend uninterrupted time on it, and then you can use the EBS plus per-day burndown to monitor whether something will likely get done by the end of the sprint.

To make it concrete, let's say we want to put a 95% confidence level on being able to finish all sprint tasks. So, for each new ticket I write, we go through the following exercise:

100 times, do:

  • for each ticket so far, select a random estimate ratio from my estimation history, multiply by the original estimate to get a likely actual time
  • add up total likely time, divide by per-day burndown figure to get a likely number of days
  • if likely days <= days available this sprint, trial is a success

If you have at least 95 successes, you may take on that task for that sprint. Otherwise, you're not allowed to commit to it.

Scrum thoughts: Fixed feature set vs. fixed time

I think we're going to enter into an interesting discussion with our product owners where they would like to propose moving away from a fixed-length (in time) iteration model to a fixed feature set model. The claim is that it is too hard to maintain a release roadmap when the set of features that comes out of a sprint is fluid.

[Sidebar: our sprint teams routinely overestimate how much we can accomplish, so items are always falling off the end of our sprints. We've mitigated this somewhat by at least marking the bottom part of our sprint backlog as "low confidence items", but the truth is we rarely get to those items, i.e., we never totally finish our sprint backlog.]

So I think there's a valid complaint here, however, I think it is misplaced. We also never actually get far enough ahead on story point estimation with the product team to have more than a sprint's worth of rough LOEs on things. So I'm not sure that any roadmap that would get produced would have any grounding in actual engineering estimates.

So now that I think about it, this is probably more of a version numbering problem more than anything else. We increment our version numbers with every sprint, and furthermore release the output of every sprint to production. So I think the issue is that, for internal consumption's sake, our product owner can't say "the following features will be in 1.7, these other ones in 1.8, etc."

It sounds like what we want to do is to choose a set of features for a release, and then simply run iterations until either that feature set is complete, or the product owner decides that the output of some iteration is good enough to release. Naturally, of course, now that I've written this out, this is the by-the-book definition of how to do release management with scrum. Not quite sure how we got away from doing things that way.

I'm pretty convinced we should stick with the fixed-length iteration method, mainly for the reprioritization effect it has at planning. For every sprint where we had carryover, big chunks of that carryover were routinely deferred down the priority list from sprint to sprint, suggesting that we actually produced more product value than we would have if we had just finished each sprint straight out.