Saturday, August 9, 2014

Agile Software Development

Agile software development
Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development and delivery, a time-boxed iterative approach, and encourages rapid and flexible response to change. It is a conceptual framework that promotes foreseen interactions throughout the development cycle.

Agile Manifesto
They published the Manifesto for Agile Software Development to define the approach now known as agile software development. Some of the manifesto's authors formed the Agile Alliance, a nonprofit organization that promotes software development according to the manifesto's principles.
The Agile Manifesto reads, in its entirety, as follows:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

Twelve principles underlie the Agile Manifesto, including:
1. Our highest priority is to satisfy the customer through early and frequent delivery of valuable software.

We are interested in delivering software that is fit for purpose. Oddly, some of the companies I visit don't seem to value actually delivering software. Agile development is focused on delivering.
Delivering early allows for quick wins and early feedback about the requirements, the team, the process, as we have seen throughout this book.
Delivering frequently allows for continued wins for the team, rapid feedback, and mid-project changes in project direction and priorities.
The duration used for deliveries needs to be negotiated on a project-by-project basis, because delivering updates on a daily or weekly basis can cause more disturbance to the users than it is worth. When users can't absorb changes to the system as often as every three months, then the project team needs to arrange some other way to get that feedback and to make sure their process works all the way through test and integration.

Statement emphasizes delivering those items that have greatest value to the customers. With
consumer mood changes, intensive competition, and stock market swings, it is nearly impossible to guarantee a revenue stream for a project that takes a year or longer to deliver.

This statement indicates that value will be delivered early, so that in case the sponsors lose funding, they will not be left with a pile of promissory notes, but with working software that delivers something of value to the buyers.

2. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

This half to the "early and frequent" delivery specifies the lengths of the work cycles. I have encountered the occasional project that can run incremental development with four-month cycles, but most use one- to three-month cycles. Using shorter cycles is rare, because the users usually can't take in more frequent changes than that.
On project Winifred (Cockburn 1998), a fixed price contract involving 50 people over 18 months, we fixed our cycles for deliveries to users at three months. Knowing that this was really too long to wait for feedback, we made sure that some expert users came and had two chances to review running code inside each cycle. These two user viewings were scheduled flexibly, usually around the 6-week and 8-week marks.
If the users can accept changes every month, and the development team can match the ongoing requests for changes, then the shorter feedback cycle is better.

3. Working software is the primary measure of progress.

This is the third reference to working software. This principle puts it firmly: rely on the honesty that comes with running code rather than on promissory notes in the form of plans and documents. You are welcome to use other measures of progress as well, but working code is the one to bank on.
Agile methodologies place a premium on getting something up and running early, and evolving it over time. Not all projects are equally amenable to tiny evolutionary steps. Deciding how to break up the giant architecture on a large project into smaller pieces that can be built and tested incrementally, does take some work. It can be done, however, and is worth the effort.
Stephen Mellor is careful to point out that in model-driven development, two pieces of working
code must be demonstrated. One is the executable model, which is evaluated for fitness to the user needs. The other piece of working code to be demonstrated is the mapping algorithm that generates the final code. This one is more easily overlooked. A number of projects created a gorgeous executable model, and then couldn't get the code-generation algorithm to work properly in time.

4. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Agile processes can take on late-changing requirements exactly because of early and frequent delivery of running software, use of iterative and timeboxing techniques, continual attention to architecture, and willingness to update the design.
If your company can deliver quickly and respond to late-breaking information, and your competitor's company can't, then your company can out-maneuver your competitors on the software front. This often translates to a major difference in the marketplace.
All of the agile methodologies have some mechanism to incorporate late-breaking changes in requirements, as already discussed. The details differ by methodology.

5. Business people and developers work together daily throughout the project.

The industry is littered with projects whose sponsors did not take the time to make sure they got what they needed. Frakes and Fox reported a study showing a strong correlation between links to users and project success or failure (Frakes 1995).
The best links are through on-site business expertise and daily discussions, which is what the statement calls for. The word "daily" refers to the sweet spot, where discussions are ongoing and on demand. Daily discussions are not practical on most projects, which mean that the project is not sitting at the sweet spot. The statement indicates that the longer the time to get information to and from the developers, the more damage to the project.

6. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

We would rather see motivated, skilled people communicating well, and no process at all, than a well-defined process and unmotivated individuals. Dee Hock's story about the early VISA system gives an extreme example of this.
Individuals make projects work. Their motivation relates to the pride-in-work, amicability and community on the project.
I first encountered the above statement in a project interview with Dave A. Thomas, then Presdent of the very successful company, Object Technology International. He said, "We hire good people, give them the tools and training to get their work done, and get out of their way." I keep finding evidence supporting his recommendation,.

7. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

This falls directly out of Chapters 3 and 4 in this book. I won't repeat the discussion and caveats here. Review those chapters if you are just dipping into the book here.

8. The best architectures, requirements, and designs emerge from self-organizing teams.

We had some discussion around the choice of words in this principle. How self-organizing do we intend: completely self-organizing, or merely allowing good ideas to come from anyone on the project? Do we mean emerge mysteriously, emerge in small steps over time, or emerge as a logical consequence of the human-centric rules the team uses?
I prefer the middle of the three choices. Jim prefers the latter of the three. None of us intend the first of the three, which comes from a misunderstanding of the word emergent as "lucky." Our common point is recognizing that the details of system design surprise even the most experienced designers.
We insist that the architecture be allowed to adjust over time, just as the requirements and process are. An architecture locked down too hard, too early, will not be able to adjust to the inevitable
surprises that surface during implementation and with changing requirements. An architecture that grows in steps can follow the changing knowledge of the team and the changing wishes of the user community.

9. Continuous attention to technical excellence and good design enhances agility.

A tidy, well-encapsulated design is easier to change, and that means greater agility for the project. Therefore, to remain agile, the designers have to produce good designs to begin with - and -also have to review and improve their design regularly, to deal with the better understanding of their design that comes with time and clean up from when they cut corners to meet a short-term goal.

Ward Cunningham sometimes compares cleaning up the design with paying off debts. Going further, he discusses managing the technical debt on the project.
Making hasty additions to the system corresponds to borrowing against the future, taking on debt. Cleaning up the design corresponds to paying off the debt.
Sometimes, he points out, it is appropriate to take on debt and make hasty changes, in order to take advantage of an opportunity. Just as debt accumulates interest and grows over time, though, so does the cost to the project of not cleaning up those hasty design changes.
Cut corners in the design, he suggests, when you are willing to take on the debt, and clean up the design to pay off the debt before the interest grows too high.”

Given the deep experience present in the room, I found it interesting to see this attention to design quality at the same time as the attention to short time scales, light documentation, and people.
The conflicting forces are resolved by designing as well as the knowledge at hand permits, but designing incrementally.

10. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

There are two sides to this statement. One relates to social responsibility side, the other to project effectiveness. Not everyone at the meeting was interested in signing onto the social responsibility platform, but we all agreed on the effectiveness issue.
People tire as they put in long hours. Their rate of progress slows, not just during their overtime hours, but also during their regular hours. They introduce more errors into their work. Diminishing returns set in with extra hours. This is part of the non-linearity of the human component.
An alert and engaged staff is more agile than a tired, slogging staff, even leaving aside all of the social responsibility issues. Long hours are a sympton that something has gone wrong with the project layout.

11. Simplicity--the art of maximizing the amount of work not done--is essential.

Simplicity is essential. That much is easy to agree on. The notion of simplicity is so subjective, though, that it is difficult to say anything useful about it. We were therefore pleased to find we could all sign up for this statement.
In the design of development processes, simplicity has to do with accomplishing while not doing, maximizing the work not done while producing good software. Jon Kern reminds us of Pascal's remark: “This letter is longer than I wish, for I had not the time to make it shorter.” That comment reveals the difficulty of making things simple. A cumbersome model is easy to produce. Producing a simple design that can handle change effectively is harder.
In terms of methodology and people, Jim Highsmith likes to cite Dee Hock:
“Simple, clear purpose and principles give rise to complex, intelligent behavior. Complex rules and regulations give rise to simple, stupid behavior.”

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

It is fitting to end where we began. How light is right for any one project? Barely sufficient, and probably lighter than you expect.
How do we do this on our project? Bother to reflect on what you are doing. If your team will spend one hour together every other week reflecting on their working habits, you can evolve your methodology to be agile, effective and fitting. If you can't do that, well ... you will stay where you are.

Agile methods
Well-known agile software development methods include:
  • Agile Modeling
  • Agile Unified Process (AUP)
  • Dynamic Systems Development Method (DSDM)
  • Essential Unified Process (EssUP)
  • Exia Process (ExP)
  • Extreme Programming (XP)
  • Feature Driven Development (FDD)
  • Open Unified Process (OpenUP)
  • Scrum
  • Crystal Clear
  • Velocity tracking
  • Kanban (development)
  • GSD