Critical Thinking about Agile Methods

1. Introduction to Agile Methods

Agile software methods are a new type of software methodology that is human-centric, iterative, and incremental, gradually gaining attention since the 1990s.

The term "Agile" originated from the Snowbird Conference in 2001, where attendees collaboratively drafted the famous Agile Manifesto, which includes four "valued over" statements:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

While the items on the right have their value, we place greater emphasis on the value of those on the left. This is the core value of Agile methods.

Agile methods encompass and describe several development methodologies, including:

• Extreme Programming (XP)
• Scrum
• Lean Development
• Feature-Driven Development (FDD), etc.

Each of these methods has its focus, but they all embody the core values of Agile. The most common Agile methods are XP and Scrum. Since XP focuses on actual programming practices while Scrum emphasizes management and organizational practices, they are often used together in practice. Each Agile method contains a series of practices; for example, XP includes 13 core practices such as pair programming, test-driven development (TDD), and continuous integration (CI).

Unlike other software development methods, Agile has two important characteristics:

1. Agile is "adaptive" rather than "prescriptive."
2. Agile is "people-oriented" rather than "process-oriented."

The "adaptability" of Agile methods is designed to address the inherent difficulty of "variability" in software development. Compared to traditional engineering fields like civil engineering, one challenge in software development is the instability of requirements, which directly leads to unpredictability in the software process. Traditional development methods attempt to create a detailed plan for a software project over a long time span and then develop according to that plan. The obvious drawback of this approach is its difficulty in adapting to changes. Agile significantly reduces the impact of changes through methods like "incremental requirements" and "rapid iterations," thus achieving "responding to change."

In traditional software development, the focus of team project work allocation is on clearly defining roles, with individuals adapting to their roles based on their abilities, and the definition of roles is meant to ensure the implementation of processes. Agile methods, on the other hand, aim to leverage human characteristics and fully utilize human creativity in software development. Agile requires all members of a project team to participate in software development, allowing developers to make independent technical decisions. Additionally, Agile development places particular emphasis on information exchange within the project team, believing that face-to-face communication is more effective than written documentation.

2. Misconceptions in Agile Practices

With the popularity of Agile methods, more and more teams are attempting to incorporate Agile thinking into their project development processes, but the results are often not ideal, leading some to question Agile methods. Indeed, Agile is not a "silver bullet"; it has its own set of problems and unsuitable scenarios. However, a significant reason for project failures is not Agile itself, but rather falling into misconceptions when using Agile.

Misconception 1: Agile for the sake of being Agile

As Agile has become a widely accepted development model, more teams are attempting to transition to Agile. However, there are many cases of failure, often stemming from misunderstandings about Agile, where teams merely adopt the formal "coat" of Agile without truly understanding its principles. Many failed cases simply force-fit Agile practices, and when problems arise, they may revert to their previous development models, leading to inefficiency.

When deciding whether to use Agile as a guiding model for development, we should advocate for pragmatism. While Agile is beneficial, it needs to be carefully chosen based on actual circumstances. If Agile cannot solve the current problems, then forcibly applying Agile will only lead to greater troubles. Additionally, when deciding to use Agile, it is crucial not to apply it rigidly; a superficial adoption of Agile will not solve any problems.

Misconception 2: Agile is anti-documentation

The Agile Manifesto states, "Working software is valued over comprehensive documentation," which often leads to a misunderstanding that documentation holds an unimportant position in the Agile process, and that documentation does not need to be overly detailed. Some even believe that Agile is against documentation and resist it in Agile practices.

In this regard, we must first clarify that the essence of documentation is to make knowledge explicit. For implicit knowledge in a project that requires some communication cost, making it explicit through documentation is meant to facilitate effective communication. Traditional views often overlook the costs associated with documentation, especially the costs of updating and synchronizing documents. In situations where certain knowledge can be effectively communicated verbally, relying on documentation can seem inefficient. However, completely abandoning documentation is also unreasonable; it would require explaining concepts to different people verbally each time, which would inevitably lead to some loss of information, making it an even less efficient approach.

Therefore, concerning Agile, we must first identify what types of knowledge in the project need to be made explicit and what can remain implicit. For the former, documenting important knowledge ensures that it does not suffer excessive loss during transmission and communication, while avoiding unnecessary repetitive exchanges. For the latter, verbal communication can reduce the costs of writing and understanding documents.

Misconception 3: "Embracing change" means requirements can change arbitrarily

As stated in "The Mythical Man-Month," "The only constant in software development is change itself," which is why the Agile Manifesto emphasizes, "Responding to change over following a plan." However, this does not mean that Agile weakens the control over changes in requirements or allows arbitrary changes.

Agile still requires a review of the quality of requirements. If a team makes a directional mistake in requirements from the outset, it can be extremely detrimental, rendering subsequent work meaningless. Therefore, careful evaluation of requirements to ensure their basic quality remains very important in Agile. During an Agile iteration cycle, it is also advisable to avoid frequent changes in requirements, as frequent responses to changes can leave a team without a clear direction and goals, diminishing team members' sense of achievement. Furthermore, Agile emphasizes business value orientation; if a change in requirements does not hold significant business value, the team does not need to rush to respond to that change.

Misconception 4: "Rapid iteration" means frequent updates

Some believe that "rapid iteration" means software versions are updated frequently, with extreme cases seeing a new version released every day. Excessive updates can lead to three problems:

1. From the user's perspective, software updates are time-consuming and troublesome, and they need to consider issues like data usage costs.
2. From the project process perspective, this can disrupt downstream tasks like releases, as software releases may require a certain amount of review time.
3. From the product's perspective, frequent updates imply that the product has not undergone sufficient testing, which may lead to instability and various bugs in actual use.

The "rapid iteration" advocated by Agile methods should focus on iterating requirements. In the internet age, user needs are often vague and uncertain; Agile aims to quickly respond to potential needs, rapidly launch products, validate requirements through user feedback, and gather new requirements before starting the next iteration. From this perspective, "rapid iteration" does relate to version updates, but the updated versions should introduce new features and user experiences rather than just minor bug fixes. In reality, many teams first release a version, quickly identify bugs, and then release a new version, with each iteration cycle being too short and focused solely on bug fixes rather than centering on requirements. The recommendation is that for less severe bugs, feedback should be collected and revised collectively before releasing a new version, with each iteration still focusing on requirements.

Misconception 5: Agile is free and unrestrained

Agile emphasizes self-organization within teams, creating an illusion of freedom and lack of constraints. The twelve principles of the Agile Manifesto state, "Motivate individuals to work on projects, provide them with the environment and support they need, and trust them to get the job done." This actually entails two underlying requirements:

1. Team members' self-discipline or professional ethics (not just working for a paycheck). Any team member's lack of engagement can severely undermine the enthusiasm of the entire team.
2. The project leader's ability to lead the team rather than manage it. Agile seeks to eliminate "bureaucracy," emphasizing leadership skills over management skills. A significant distinction between the two is that leadership inspires team members to willingly follow rather than being managed coercively.

Thus, Agile is not entirely free and unrestrained; it requires self-discipline from team members and correct leadership from leaders. Many teams fail to achieve these two points, so encountering problems while using Agile is understandable. Therefore, when attempting to use Agile, these two factors should be fully considered. For some teams, strict management may still be necessary, and Agile may not be suitable for them.

3. Suitable and Unsuitable Scenarios for Agile Methods

From the Product Perspective

Agile methods are suitable for products where requirements are likely to change, but unsuitable for systems with high demands for reliability, safety, and complex large systems.

In the software development process, issues related to requirements mainly fall into three categories:

1. Key information is lost during the long process of transferring requirements from users to programmers.
2. Users are unclear about their own needs, and their descriptions of requirements are inaccurate, leading to a widening gap between developers' understanding of requirements and the actual vision, resulting in serious deviations in development.
3. In the context of the rapid development of mobile internet, requirements may change daily; even if users accurately describe their needs and communication is flawless, by the time the software product is delivered, the requirements may have changed significantly.

Agile methods can effectively address the above requirement issues through "incremental requirements" and "rapid iterations," but this development model also determines that Agile methods have low fault tolerance for software. Therefore, for systems with high demands for reliability and safety, Agile methods may not be appropriate. For example, lunar landing program control software requires extremely high safety and reliability, and its requirements are relatively stable; Agile methods do not provide much assistance in such projects.

Additionally, for large systems, various factors need to be considered, such as system characteristics, communication and integration with other systems, and the needs of different user groups. As a result, large systems often come with high complexity and also have high requirements for safety and reliability. In the face of such systems, each iteration in Agile requires rigorous validation, and the issues to be addressed are quite complex, making it difficult to achieve "speed" while meeting requirements, which contradicts the original intent of Agile.

From the Organizational Structure Perspective

Agile methods are suitable for small, smoothly communicating, and mutually trusting development teams, but are difficult to apply to large development teams.

Agile methods aim to unleash the maximum potential of each member through the operation of Agile teams, achieving the development of software products with maximum synergy. In such a self-organizing team, each member can contribute fully to the project, and every member is a stakeholder in the project, with ample technical exchange and communication sharing among members. This technical exchange and communication sharing process can enhance the abilities and levels of developers, maximizing individual capabilities.

In team building, one conceivable issue is that an increase in team size can lead to higher communication costs, requiring more unrelated personnel to intervene in management, which undoubtedly reduces team members' enthusiasm and sense of participation. Therefore, in an Agile project that heavily relies on individual initiative, having too many people can be disastrous. Moreover, Agile methods like Extreme Programming are only suitable for small teams of 7 to 10 people; splitting a large team into smaller teams can exponentially increase communication costs and severely overspend on management costs. Thus, Agile methods are better suited for small, agile development teams; at least for now, adopting Agile methods in large development teams is inappropriate.

From the Urgency of Software Projects Perspective

Agile methods are suitable for projects with aggressive time constraints; if a project does not have strict time limits and supervision, there may be no need to use Agile methods.

The development and maintenance cycles of systems are also factors to consider when choosing software development methods. Typically, large systems require long development and maintenance times, and attention must be paid to potential changes and redesigns, and they may be required to deliver different versions. If a project does not have a high level of urgency, considering other factors, Agile methods may not be suitable for that project, as some practices in Agile, such as time-box management, exist to maintain tension and focus during the project development process. If the project itself lacks urgency, there may be no need to use Agile methods.

4. Conclusion

Agile methods are a practical tool, and the characteristics of software development in the internet age provide a platform for Agile methods to shine. However, having the tool is not enough; we must also learn how to use it correctly, as incorrect usage may backfire. Moreover, Agile methods are not a "silver bullet"; they also have unsuitable scenarios and problems they cannot solve. Understanding how to use Agile correctly, when to use it, and when not to use it is essential for truly benefiting from the advantages Agile methods offer.