Project management is a complicated field; there are a plethora of methodologies, software options, team member roles, and expectations that vary, not just within organizations or industries, but across the entire field of project management itself.
There’s rarely a process improvement claim that the project management community doesn’t scrutinize. From emphasizing extroversion versus introversion for project manager personalities to which project management software is best for the business, there’s rarely agreement on anything.
One trend, however, has been growing since 2015 and earlier: Agile project management use is on the rise (and it’s great for software development!).
With all the emphasis on agile customization, or Scrum vs. Kanban, or any other project management methodology quibbles, it’s easy to lose sight of what agile software development actually is--and how it works.
Let’s do a deep dive of what agile is—and how you can use it to kick your software development into high gear.
This is a complete guide to agile management—it covers the process’ history, development cycle, significance, and a discussion of its benefits and drawbacks. Use the table of contents below to jump through the history if you want to only learn about the agile process and not where it came from.
The Complete Agile Software Development Guide
The History of Agile
The Institute of Electrical and Electronics Engineers can trace rudimentary agile project management methods all the way back to 1957 in Los Angeles, where Bernie Dimsdale, John von Neumann, Herb Jacobs, and Gerald Weinberg worked closely together on developing software for IBM and Motorola. Craig Larman, while quoting Weinberg in his article “Iterative and Incremental Developments. A Brief History,” said,
"All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities. I think what the waterfall description did for us was make us realize that we were doing something else, something unnamed except for 'software development.’"
“Waterfalling,” or the waterfall model, was the prevailing form of project management at that time. It’s closely affiliated with Gantt charts and lays out what tasks need to be done at what points in the project in order for the project to move forward.
Using this method, software development became incredibly difficult. While the waterfall method relies on predictability and sequence, software developers needed a more flexible project management method that made room for error, bugs, setbacks, and the independence affiliated with the software industry.
It wasn’t until the 1970s that a more formal approach to agile took hold, led by Dr. Ernest Edmonds, Tom Gilb, and the New York Telephone Company's Systems Development Center. The ideas were not widely appreciated. In fact, when Dr. Edmonds presented his idea to the Journal of Computer Aided Design, it was immediately rejected. The sole comment the journal sent back to Edmonds was, "If you don't know what you are going to do before you start you shouldn't start!” He then took his work to General Systems, which eventually accepted his work.
In the 1990s, software developers were at their wits’ end. This cohort of programmers, largely of the Generation X demographic (born between 1961 and 1983), were “latchkey kids” and carried over their anti-establishment teenaged years to the office. As Craig Larman discovered, developers found waterfall “heavily regulated, regimented, and micro-managed.” It wasn’t working for this cohort.
The 90s brought forth many lightweight and versatile project management approaches to software development, including dynamic systems development method (DSDM), XP, Scrum, and feature-driven development (FDD). While all of these systems existed before the Agile Manifesto, they are now considered to be part of the agile methodology.
The Agile Manifesto
Between February 11-13, 2001, seventeen individuals met at The Lodge at Snowbird ski resort with three things in common: a love of food, a love of skiing, and a “need for an alternative to documentation driven, heavyweight software development processes.” They, to this day, unironically call themselves “organizational anarchists.” From this gathering, all participants ceremoniously signed the Manifesto for Agile Software Development [PDF], a seven-page document centered around four concepts:
“Individuals and interactions over processes and tools.
Working software over comprehensive documentation.
Customer collaboration over contract negotiation.
Responding to change over following a plan.”
In his “History: The Agile Manifesto,” co-founder Jim Highsmith writes,
In order to succeed in the new economy, to move aggressively into the era of e-business, e-commerce, and the web, companies have to rid themselves of their Dilbert manifestations of make-work and arcane policies. This freedom from the inanities of corporate life attracts proponents of Agile Methodologies, and scares the begeebers [sic] (you can’t use the word “sh-t” in a professional paper) out of traditionalists. Quite frankly, the Agile approaches scare corporate bureaucrats—at least those that are happy pushing process for process sake—versus trying to do the best for the "customer" and deliver something timely and tangible and “as promised” because they run out of places to hide. Editor’s note: we edited the original language to be blog appropriate.
The Agile movement is not anti-methodology, in fact, many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment. Those who would brand proponents of XP or SCRUM or any of the other Agile Methodologies as "hackers" are ignorant of both the methodologies and the original definition of the term hacker.
Editor’s note: we edited the original language to be blog appropriate.
The Agile Manifesto emphasizes that none of the authors believe in a “silver-bullet theory,” hence why nailing “What Is Agile” gets so complicated; the founders did not think that they had all the answers, and they didn’t think there was a perfect one-size-fits-all methodology for every development team. Their approach was necessarily libertarian. They stressed in the document at almost every juncture: “this group is about helping, not telling. The Alliance members want to help others with Agile methods, and to further our own knowledge by learning from those we try to help.”
Perhaps it was the “we don’t know everything, but we do know something” approach to software development that helped the system catch hold.
Agile grows in popularity
From 2001 on, project management processes deriving from agile exploded. The Agile Alliance and the Scrum Alliance quickly emerged, along with the 2001 classic, Agile Software Development with Scrum. Certified Scrum courses became commonplace, and terms like “Scrum Master” and “self-organization” entered the project manager vernacular. By 2007, IBM declared,
Agility is everywhere. Ignore it and you will be lost in the current technical dialogue. Learn about it and you will be able to make intelligent decisions about whether it is right for you. You will also be able to understand many of the other practices and methodologies that are emerging. If you are a manager of technology at any level, this is your responsibility as well as a survival necessity.
Now, almost ten years later, agile development, in one of its many forms, is used in most software development.
Agile Software Development
“Agile” project management is an umbrella term that covers many different kinds of software development. It includes:
While each of these approaches to software development is distinct, this piece is only going to focus on how they tie up into agile. This section is divided up in the Agile Manifesto’s four core beliefs.
1. Individuals and interactions over processes and tools
Any way you cut it, it’s people who are developing software, not computer programs or cogs in a corporate wheel.
Don’t get me wrong; agile project management software is a great resource for project managers. This kind of project management software offers tools to improve communication, track bugs, and map out software releases. But the software itself isn’t what makes the team. VersionOne explains:
We should be interacting with one another, collaborating on stuff, ironing out details, confirming understandings, asking questions, going to lunch, getting to know each other better, developing trust and bonds. Face to face interaction. In this new Agile world we live in, producing software requires us to be more social than we may have been in the past. And that’s not something that’s going away. The folks that got into IT specifically because they didn’t want to deal with people may be in trouble. Yes, we all still need our quiet/focus time. That’s what focus spaces are for. Most companies recognize this and in addition to providing small focus spaces outside the team room, are issuing laptops or tablets now, and not desktops. Mobility provides the opportunity to get away and focus when necessary. Ironically, it also provides greater opportunity for collaboration.
In other words, while the folks at Snowbird were focused on creating a method for software development, they wanted the first focus to be on the team. It is their first value for good reason. The philosophy is that teams that don’t cooperate are necessarily going to be slower to adapt to adversity, and thus drag their feet on creating high-quality, shippable software.
Should an Agile team adhere to this value, the following are the ideal outcomes:
Communication is quick, clear, and efficient between team members.
Team members form a strong bond that enables effective teamwork.
All processes are fluid enough to customize to events.
Team members feel ownership over their parts of the project.
Team members are encouraged to innovate and take risks.
2. Working software over comprehensive documentation
In large corporations, governmental entities, or even teams with overzealous bosses, it’s easy to dedicate entire portions of one’s day to documentation.
Don’t believe me?
How many times have you filed a lengthy expense report, waited for a manager (or senior manager, or VP)’s approval to move forward with a task, or even plug hours in for a client?
Doesn’t that get… tiring, at some point? Doesn’t that stifle innovation?
In fact, it does. And agile methodologists want to do away with this obvious roadblock.
Agile teams aim to have the smallest amount of documentation possible. Each document should support product development—and not much else.
Instead of spending mindless hours recording what has been and will be done, agile teams focus on the actual product. Instead of approaching the product as a whole, the agile method emphasizes functionality.
For example, take a house.
In a traditional method, construction managers would use construction management software to evaluate how far along the project is. If, say, the foundation was set, the framing was done, the siding and wiring were set in place, and insulation was put in, the new house would be 65% done. However, the house isn’t functional.
You are 65% done working on a house, but currently have zero completed houses.
In agile software development, the “65% done” would mean 65% through a completed project. So for software, this would mean that the most important parts of the software are completed. Done. Finished. It’s operable. If it’s not finished, the team should go back over it in iterations until it’s completed.
Should an Agile team adhere to this value, the following are the ideal outcomes:
Teams spend far more time developing than documenting.
Requirements are clearly documented and feed into software development.
Team members are participate in regular code review.
Software designs go through an iterative revision process.
Project features are not “done” until created, tested, optimized for quality, and approved.
3. Customer collaboration over contract negotiation
This third point is often a misunderstood concept in agile.
Not all customer collaboration is informal, and contracts certainly exist between stakeholders and teams. No one who adheres to agile’s philosophies would argue that point. However, in order to avoid miscommunication, agile enthusiasts emphasize collaboration over conflict. Using software like Meisterplan, Sciforma, and Eylean, project managers can quickly communicate where their team is on a project in real time, so no customer is left ignorant to the state of their project.
Agile users understand that an initial contract does not always reflect what a stakeholder really wants—part of the reason why an iterative, agile approach is so popular is because stakeholders can tweak their requirements as features and functionality become clearer.
Agile thus supports the flexible sale; having a fixed budget will not necessarily determine the best software outcome because it may deny the necessary number of iterations it takes to get the software right. The end result is a happier customer and a better product.
Should an agile team adhere to this value, the following are the ideal outcomes:
Contracts define when a project starts and its scope.
Contracts are flexible to change for the sake of a higher-quality product.
Changes to a contract are only made for the purpose of a higher-quality product and/or to increase customer satisfaction.
The responsibility for a successful project is mutual between the client and team.
Communication between the client and team is regular, and regular conversations about iterations and change are present.
4. Responding to change over following a plan
The rigidity of traditional project management methods is what drove the founders of agile to create a more flexible project management system. There is no surprise that their final principle addresses the importance of adaptability and quick response to change.
There are many forces that can easily change the direction of a product. These can include, but certainly aren’t limited to:
A change in budget or contract.
A change in public opinion.
A change in knowledge about what the final product should look like.
A change in stakeholders.
Naturally, lots of change is bad news for a business trying to create a finished product. Thus, agile replaced traditional due dates with burndown charts, retrospectives, risk assessments, and even story card lifecycles. Agile aims to give team members the tools to prioritize, plan, and work through any unforeseen changes.
The final idea is for any new challenge to become a net benefit to the project as opposed to a setback.
Should an Agile team adhere to this value, the following are the ideal outcomes:
Team members can easily respond to project changes.
Change is predictable.
Teams are able to track progress over what has actually been completed.
Planning is an ongoing activity from the beginning to the end of the project.
The overall project status and next steps are transparent between team members.
The Agile Software Development Cycle
Agile Software Development
As I mentioned earlier, there are lots of different kinds of agile software development, but they all have the above four cornerstone characteristics. Regardless of if you’re using Scrum, XP, Crystal, Feature-Driven Development, or another Agile method, the basic agile approach remains the same.
Start the project.
Define what the project is and what its goals are.
Create guidelines for the project’s requirements.
Develop a software function.
Integrate the function.
Test the function.
If the test succeeds, move on to the next function and repeat steps 4-6.
If the test does not succeed, record what went wrong and implement changes until the function works.
Reflect and reprioritize based on client feedback and new challenges.
Once tested and integrated, release the function to the market.
Move on to the next function development and repeat steps 4-10 until the project is done.
A particularly good example comes from the ISTQB Exam Certification site. They show the development of a word processor through iterations. While each iteration improves on the last and adds new features, they also all build on each other. Each is like a new product, built using the the previous iteration as a foundation.
ISTQB Exam Certification
Commentary on Agile Software Development
While agile software development is currently in vogue, there are naturally benefits and problems with the project management technique.
Advantages of Agile Software Development
The agile methodology can be broken down into seven advantages:
1. A focus on quality
When software development is broken down into small chunks, it’s a lot easier for project teams to focus on testing and reviewing each iteration. Agile allows teams to quickly find and fix software bugs.
2. A focus on the customer
Agile emphasizes continuous delivery of high-quality software. Customer feedback and satisfaction directly informs product development.
3. A focus on transparency and communication
Agile software development focuses on user involvement and collaboration. Stakeholders at all ends of the project actively provide software feedback.
4. A focus on flexibility
Project managers can easily reprioritize and clarify what is important to the product—even if that changes week to week. Backlogged goals can easily filter into subsequent iterations.
5. A focus on speed
Regular releases allows software developers to always have a part of a completed product on hand. Speed is likely where agile wins out the most. In the 9th State of Agile survey [PDF], the highest-reported benefit of adopting agile practices was “accelerated product delivery.”
6. A focus on mitigating risk
Project managers can use iterations to adapt to a changing software environment. Incremental software releases makes it easy to identify bugs and problems early, before the product is finalized and more difficult to fix.
7. A focus on managing a lean budget
Project costs become more predictable when broken down into small pieces (often called “sprints”). Teams also have the opportunity to be paid as each release is delivered.
Disadvantages of Agile Software Development
While agile is great for many software development teams, it’s not right for everyone. Before adopting the project management methodology, consider these five drawbacks.
1. Agile isn’t great for government and large organizations
Because of agile’s lack of emphasis on documentation and its need for transparency, organizations that tend to be inflexible would feel hamstrung by the project management method.
2. Delivery can trump quality
Quality assurance can easily fall into the depths of a backlog when constant software iterations are expected.
3. The cost of change is ignored
As Lajos Moczar points out in CIO, “People often change really big things late in the game using the rationale that since it's an agile project, it can handle it. The only way the project can handle this is by adding iterations. As that happens, defects that might have been easy to fix at one point get harder and harder to fix, since the code base keeps changing.”
4. Agile doesn’t work well with teams needing close supervision
Agile is a great product development process for small, smart teams. If the IT department has lots of team members who need to consistently be managed, agile will quickly break at the weakest link.
5. Agile requires training
In the State of Agile report, the number one reason of why agile projects fail is because of a “lack of experience with Agile methods.” While waterfall project management is more intuitive, agile takes time to learn.
Depending on your needs, you can hire a company to help implement your agile software. Check out our hiring guides for web design companies and custom software development companies to determine which one would be a better fit for you.
Conclusion
Agile software development is more than just a project management fad; it’s a project management discipline with clear benefits, especially to IT teams. The project management methodology emphasizes transparency, iterative process, communication, and final product quality.
For teams looking to adopt agile project management software, check out Capterra’s free software advising service. We can help you find the right software for your business—at no cost to you.
I’d love to hear your thoughts on agile software development. Is there something I missed? Leave your thoughts in the comments below!