Understanding Agile Spikes

In agile product development, teams often encounter uncertainties that can hinder progress. Whether it’s grappling with new technologies, estimating complex tasks, or exploring innovative solutions, these challenges require a strategic approach. Enter the agile spike—a tool designed to reduce uncertainty and risk through focused experimentation.

What Is an Agile Spike?

Originating from Extreme Programming (XP), a methodology developed by Kent Beck [1], the concept of a spike serves as a means to manage uncertainty in product development and projects. The term “spike” draws inspiration from rock climbing, where climbers drive a spike (piton) into the rock face to support future progress. In product development, a spike is a short, time-boxed investigation aimed at answering specific questions or exploring particular approaches.

Ward Cunningham, one of the pioneers of XP, offers a different angle to understand spike solutions. He likens a spike to a depth-first approach—going “from the top to the bottom, then closing the loop” [3]. This means creating an end-to-end, but very thin, slice of functionality, much like driving a spike through a log. The goal is to focus on the core of the problem without getting bogged down by complexities.

He often asked, “What is the simplest thing we can program that will convince us we are on the right track?” This question encapsulates the essence of a spike: stepping outside the immediate difficulties to find simpler and more compelling solutions.

Key Characteristics of a Spike

1. Time-Boxed: Spikes are limited in duration, usually spanning a few hours to a few days.
2. Focused on Depth: They delve deep into a specific aspect to gain a thorough understanding.
3. Experimental: Involves creating prototypes or conducting targeted research. (See also Rapid Experiments vs. Big Upfront Planning)
4. Independent of Existing Mechanisms: Developed separately to avoid added complexity.
5. Non-Deliverable Output: The primary output is knowledge, not a finished product.

When to Use a Spike

Spikes are particularly useful in situations where the team faces high uncertainty or lacks knowledge in a specific area. Common scenarios include:

• Estimating Complex Tasks More Accurately: When the team is unsure about the effort required.
• Evaluating New Technologies or Methods: Assessing how new tools fit into existing systems.
• Reducing Risk in High-Uncertainty Areas: Gaining insights to make informed decisions.
• Experimenting with Innovative Implementation Techniques: Testing out new ideas before full-scale implementation.

Identifying the Need for a Spike

A good indication that a spike is necessary is when the team feels too unsure to provide rough estimates for a user story, or when estimates are inflated due to uncertainty. As an experienced practitioner notes:

“If your team tries to estimate a task and they either feel too unsure to give rough estimates, or their estimates are very high because of the uncertainty, a spike can help to reduce uncertainty and get to a more useful estimate. Also, the knowledge acquired during a spike is very valuable in later implementation.”

Good Practices for Conducting Spikes

1. Time-Box the Spike

Spikes differ from regular user stories as they don’t deliver direct business value. To prevent spikes from becoming long-running tasks with questionable value, it’s crucial to set a strict time limit.

• Set Clear Time Limits: Typically a few hours to a couple of days.
• Accept Incomplete Certainty: The goal is to reduce uncertainty enough to proceed confidently, not to have perfect certainty.

“We cannot use the same criteria to determine when a spike is ‘done.’ So how do we ensure that spikes don’t spiral into long-running tasks? A common and proven method is to time-box them.”

2. Focus on the Core Problem

Emulate Cunningham’s approach by concentrating on the essence of the requirement. Write the smallest possible code or create a simple prototype to understand the core functionality.

• Depth-First Exploration: Dive deep into a specific aspect to gain clarity.
• Simplify the Problem: Strip away complexities to focus on what’s essential.

“Compounded complexity within existing code can distract from the essence of a requirement. Therefore, write the smallest possible code that could be said to perform a function independent of existing mechanisms.” — Ward Cunningham

3. Avoid Overcomplicating Objectives

While it’s important to have a goal, avoid creating extensive acceptance criteria. The aim is to reduce uncertainty, not to achieve complete understanding.

• Define Key Questions: Outline what you hope to learn.
• Stay Flexible: Be open to discovering unexpected insights.

“Sometimes it can also help to define questions we want to be able to answer when the spike is done. But refrain from creating ‘acceptance criteria’ in the form of a list of deliverables; that’s usually not helpful for spikes.”

4. Share Knowledge Effectively

Spikes are most beneficial when their insights are shared promptly and efficiently.

• Face-to-Face Communication: Enhances understanding and retention.
• Collaborative Efforts: Pair or mob programming distributes knowledge across the team.
• Alternative Documentation: Use screencasts or demos instead of lengthy reports.

“Try to pair or mob around spikes instead of having them done by a single team member. That shares the knowledge, improves the team’s resilience, and reduces the need for written documentation.”

5. Avoid Overuse and Misuse

Over-reliance on spikes can undermine agile principles and revert the team to less adaptive methodologies.

• Spikes Are the Exception: They should not dominate your backlog.
• Beware of Waterfall Patterns: Adding an “analysis spike” to every task can mimic waterfall processes.

“If half of your backlog is full of spikes, you’re doing it wrong! Spikes are the exception, not the rule.”

Caution in SAFe Environments

In the Scaled Agile Framework (SAFe), spikes are considered a subtype of enablers—work items that support future business functionality by providing exploration, architecture, infrastructure, and compliance [4]. While enablers have a prominent role in SAFe, it’s crucial to avoid overusing them.

• Risk of Waterfall Tendencies: Excessive focus on enablers and spikes can lead teams back to waterfall methodologies.
  ​

  “In SAFe, a spike is a subtype of enablers, and enablers have a more prominent role in SAFe than in other agile approaches. So especially in a SAFe context, I would caution against too many spikes or enablers in general. It’s an easy escape for those wanting to preserve the waterfall.”

• Balance is Key: Ensure that spikes and enablers are used judiciously to support, not overshadow, value delivery.

Real-World Applications of Spikes

Experimenting with New Technologies or Methods

Implementing unfamiliar technologies or methods can introduce significant uncertainty.

“Spikes helped our teams especially in cases when we used new technology or a new process. We built a small prototype utilizing the new method. That way, we gained insights to estimate the effort properly and acquired the knowledge to implement features with the new approach.”

Reducing Risk and Uncertainty

Spikes play a crucial role in risk management by enabling teams to make informed decisions.

• Confidence in Estimates: Improved accuracy in planning.
• Avoiding Wrong Turns: Early detection of unsuitable technologies or methods.

“Spikes reduce uncertainty about future tasks. By allowing us to more confidently estimate those tasks, we reduce uncertainty in delivery dates.”

Accelerating Development

By resolving debates over implementation approaches through experimentation, spikes can expedite the development process.

“Spikes can also accelerate development because they can essentially shortcut long debates about how we should do things. Instead of ‘this approach might be better than this,’ we can simply say, ‘let’s just try both and see.’”

Beyond Software Development

While spikes originated in software development, they are equally valuable in other fields of product development.

• “If you’re building a rocket and want to change how you inject the propellant, you usually want to experiment with this new injection mechanism first before you build the entire engine.”
• “In the finance industry, I saw how spikes increased the team’s knowledge about new regulations, so that they could make quicker and better decisions when it came to implementing product features around this regulation.”
• Manufacturing: Testing new materials or production methods to improve product quality or efficiency.

Advice for New Agile Teams

Introducing spikes to new agile teams requires caution to prevent misunderstandings and misuse.

• Don’t Overemphasize Spikes: They should be introduced as needed.
• Educate on Proper Use: Ensure the team understands the purpose and limitations.
• Emphasize Time-Boxing: Always set strict time limits.

“I usually don’t mention spikes to new agile teams. The risk of them being misunderstood and inadvertently abused is real. Only once we encounter a situation where our knowledge isn’t enough to give any kind of rough estimate do I introduce the concept of a spike. And then I always, always, ask them to time-box a spike!”

Conclusion

Agile spikes are a powerful tool for navigating uncertainty and mitigating risks in product development. By focusing on the core problem, time-boxing the effort, and sharing knowledge effectively, teams can leverage spikes to enhance their agile processes. Spikes are a means to explore the essence of a problem quickly and efficiently. Spikes can sometimes help when your team really struggles with estimating a user story.

However, it’s essential to use spikes judiciously to avoid potential pitfalls like overuse or misalignment with agile principles. When applied correctly, spikes can be a catalyst for innovation and efficiency across various domains of product development.

You can find my poster describing spikes here – it’s free to download.

References

1. Beck, K., & Andres, C. (2004). Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley Professional.
2. Jeffries, R. (2004). Extreme Programming Adventures in C#. Microsoft Press.
3. Cunningham, W. (2010). Spike Solution​
4. Scaled Agile, Inc. (n.d.). Spikes and Enablers in SAFe. Retrieved from Scaled Agile Framework​

Thank you for reading The Agile Compass. I’m Matthias, here to help you help those around you become agile.

To get more, consider upgrading to a paid subscription. You’ll join our premium Discord community and get access to all past articles.

​