Thank you for the question.

This is clearly an intake problem.

Let’s start by making one thing clear: We can’t (and shouldn’t) eliminate all unplanned work.

In many environments, production incidents, regulatory changes, security vulnerabilities, and critical customer issues are legitimate realities.

And therefore…

Our aim should be to make the “intake” of such work visible and governed by clear trade-off decisions.

As a Scrum Master, you can help your team.

How?

By investigating the system through which “such” work enters your Sprints.

Let’s walk through how to do this properly.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Q: “Hi Vibhor, could you write about the metrics an RTE can use in an Agile Release Train? What should an RTE actually focus on to enable better planning, improve execution, and steer the ART toward value?”

Thank you for the question.

This is one of the most common questions I get from Scrum Masters stepping into or supporting an RTE.

The honest answer is this:

Most RTEs either measure too little (and fly blind into PI Planning) or measure too much (and drown their teams in dashboards). The goal is the right data, used the right way.

Here’s how I’d break it down.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Thank you for the question.

That observation you shared tells me that you are already doing the harder part of the job.

Most Scrum Masters I speak with are focused on getting their teams to follow the practices. You are asking whether the practices deserve to be followed at all. That is a more honest question, and it is the right one to be asking in 2026.

The problem you are standing in right now looks like this:

• Daily Scrum at 9:15 a.m.,

• Sprint Planning every other Monday,

• a groomed backlog,

• a velocity chart on the wall,

• a retrospective every two weeks,

• a Sprint Review on the last Friday.

• Attendance is good.

• The events run on time.

And yet the team’s decisions take longer than they should.

Nobody questions the events, because the EVENTS are Agile, and Agile is how the team does things.

Scrum Masters are no longer introducing Agile to teams for the first time.

They are inheriting Agile.

A team that inherited its practices is a different problem than a team that failed to apply them. The inherited team looks fine from the outside. The events are followed and accounted for. What is missing is the reason the events were designed in the first place.

And because these practices have been around for years, people stopped asking the necessary question:

Is this still helping us learn, or are we just preserving the shape of Agile?

SO let’s ask this question and see what happens.

Subscribe now

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Agile is old enough to need inspection

Agile is no longer new.

In the early days, many teams were trying to move away from heavy upfront planning and rigid delivery models like “waterfall”. Agile practices felt fresh at that time because they challenged the old system.

Now, in many organizations, Agile itself has become the system.

That means some teams are no longer rebelling against bureaucracy. They are navigating Agile bureaucracy.

Has Agile failed?

No!

Agile practices have been institutionalized.

And anything that becomes institutionalized needs inspection… periodically!

Separate the principle from the habit

The Agile Manifesto comprises four values and twelve principles.

Velocity appears nowhere in it, nor do story points.

The Manifesto does not define the structure of a Sprint Review or specify how long a Daily Scrum should last.

Those practices came later, mostly through Scrum and XP, and then through the organizations that implemented them, modified them, scaled them, and handed them down as received wisdom.

The distinction between Principles and Practices matters enormously, and getting it clear is the first step for an audit.

#1: Principle

A Principle is a durable belief about how good software work gets done.

These beliefs have held up across a wide range of contexts over 25 years.

#2: Practice

A Practice is a specific “action” designed to express a principle in a real working environment.

An institutionalized habit occurs when a Practice disconnects from its “original” purpose and continues to run on social inertia.

The team does it because the team has always done it, because the tooling was built around it, because the certification said so, because the manager can see it on a report, because removing it would require a conversation nobody wanted to have.

phew…

I know.

Why teams protect Practices that no longer work

The error that keeps Agile organizations stuck is treating Practices and Principles as equivalent, then using the authority of the Principles to shield the Practices from scrutiny.

Read that again…

Here’s an example:

When someone questions whether “velocity” tracking is actually helping the team make better decisions, they are examining a specific “action” used to express a Principle. That examination is completely legitimate. The Principle survives the question. The Practice has to answer it.

A 2025 systematic mapping study that analyzed 97 studies published throughout the Agile era found that

• communication,

• adaptability,

• leadership, and

• collaborative sense-making

are the qualities that recur most consistently across healthy Agile teams, regardless of which framework they use.

These are human capabilities with a 25-year evidence base behind them.

The practices that still support these capabilities deserve to stay. The practices that have gradually displaced them deserve inspection.

4 tests every Practice needs to pass

Before deciding what to keep, retire, or rebuild, you need a consistent basis for evaluation.

The audit I recommend applies 4 tests to every Practice.

Test 1: Feedback:

Does this practice shorten the time between action and learning?

Apply this test by asking whether the gap between doing and understanding actually shrinks because of the practice, or whether it would shrink just as fast without it.

Test 2: Customer Contact:

Does this practice move the team closer to real user evidence?

Product specifications written on behalf of end-users are a different category of evidence than observations of actual end-users interacting with working software.

Apply this test by asking whether the practice narrows that distance, or whether it substitutes one layer of interpretation for another.

Test 3: Team Learning

Does this practice produce adaptation, or only attendance?

Showing up to a Scrum event is compliance.

Changing behaviour as a result of that Scrum event is learning.

Apply this test by asking what the team does differently after the practice runs. If the honest answer is nothing in particular, the practice is generating compliance.

Test 4: Decision Quality

Does this practice help the right people make better decisions sooner?

Visibility that informs senior leadership without changing what the delivery team does next is a different outcome than a practice that puts decision-making authority with the people who have the best information.

Apply this test by asking who makes a better decision because of this practice, and whether that person is actually the one closest to the work.

These four tests are not a published academic framework. Apply them as you see fit.

What matters is applying the same questions to every practice, including the ones you personally favour.

Practices that still hold up

Some “inherited” Agile practices hold up well.

They earned their place because they still do real work, and that work is clear when you apply the four tests.

#1: Small Increments of Deliverable Work

Short cycles create feedback. Long cycles create assumptions.

This passes all four tests with little difficulty.

Breaking large uncertain problems into small deliverables, then getting real feedback on each slice, is as sound in 2026 as it was in 2001.

#2: Honest retrospection

This passes the feedback, learning, and decision-quality tests, but only when done well.

The word “honest” is carrying weight in that sentence.

A retrospective that produces a list of observations and no behavioural change is no good.

When a team uses a regular cadence to examine what is slowing them down, what is creating confusion, and which experiments they are willing to run, they are applying the twelfth principle of the Manifesto directly. That principle has not lost anything over 25 years.

#3: Direct Customer Contact

This practice holds up for the same reasons it always did.

Teams with regular, structured access to real users improve their understanding of what to build.

The teams that build things people actually use tend to be the teams that spend time watching people use things.

#4: Visible Work

This practice passes the decision-quality test when visibility genuinely serves the team.

What these practices share is that they quickly connect the team’s work to real consequences.

They create conditions under which learning happens continuously, without waiting for a scheduled reporting cycle.

Be careful with metrics that become targets

Metrics deserve special attention.

Historical delivery data can be useful. It can help teams forecast, understand flow, and have more honest conversations about capacity and trade-offs.

But when a metric becomes a target, behavior changes.

Velocity is the obvious example.

When velocity is treated as a forecasting signal, it may help. When it becomes a performance target, it can damage the system. Teams may inflate estimates, avoid risky work, split items artificially, or focus on points instead of outcomes.

That is why Scrum Masters need to inspect not only the metric, but the behaviour the metric creates.

A metric is not neutral once people are judged by it.

Rebuild before you retire the practice

Some practices should not be kept as they are. But they should not be thrown away either.

They need to be rebuilt.

This is an important category because it avoids the false choice between defending the old process and rejecting everything.

Sprint Planning is a good example.

The need behind Sprint Planning is still real. Teams need to decide what matters next, understand the goal, see the risks, and align around a short-term plan.

But in many teams, Sprint Planning becomes a capacity-filling exercise. The discussion focuses less on the outcomes and more on the capacity planning to fit.

Rebuild refinement around evidence

Backlog refinement needs the same treatment.

The original purpose is valuable: create enough shared understanding before work begins.

But refinement can easily become backlog farming.

Items are added, described, estimated, tagged, split, and stored. The backlog grows. The team feels productive. But the organization is simply creating a larger inventory of untested ideas.

Sometimes the best refinement outcome is not a better ticket. It is deleting the ticket.

Rebuild Sprint Reviews around decisions

Sprint Reviews are another common place where Agile becomes a routine.

Here’s what happens:

The team demonstrates work. Stakeholders nod. Someone says, “Looks good.” A few comments are made. Then everyone leaves, and the backlog continues mostly unchanged.

That is a demo. It may be useful, but it is not enough.

When the Sprint Review becomes a decision-making and learning event, it regains its power.

The goal is to inspect the product and adapt the future.

Look for customer contact

One of the strongest tests of an Agile practice is whether it brings the team closer to users.

Many teams are internally Agile but externally disconnected.

They collaborate well with each other. But real user evidence remains distant.

That is dangerous because the team can become very efficient at delivering assumptions.

Agile without customer contact easily becomes internal bureaucracy.

Compliance is not the same as adaptation

• A compliant team holds the retrospective. An adaptive team changes how it works as a result of the retrospective

• A compliant team updates the board. An adaptive team uses the board to notice bottlenecks and rebalance work

• A compliant team attends the Sprint Review. An adaptive team changes product direction based on what the review reveals

From a distance, compliance and adaptation can look similar.

Inside the team, they feel very different.

Compliance feels like going through the motions. Adaptation feels like a process that helps the team think, decide, and improve.

Scrum Masters should care less about whether the activity happened and more about whether it changed anything important.

Use decision quality as the main lens

If there is one lens that ties the whole audit together, it is decision quality.

A practice is valuable when it helps the right people make better decisions sooner.

For Example:

• Daily Scrum should help the team decide how to coordinate today.

• Sprint Planning should help the team decide what matters most in the next short cycle.

• Sprint Review should help the team and stakeholders decide what to continue, change, stop, or investigate.

• Retrospective should help the team decide what to improve next.

• Refinement should help the team and Product Owner decide what is clear enough, valuable enough, small enough, and evidence-backed enough to move forward.

This lens is powerful because it connects Agile practices to the actual outcomes.

The last thing teams need is more events. What they need is better moments of alignment, learning, and adjustment.

How to run the Audit

The 4-test audit works best when teams run it on themselves.

Start here:

Have the team list every recurring Agile practice it currently follows: events, artifacts, reporting habits, and meetings that exist because the framework says so. Get the list visible before evaluating anything.

Then:

take each practice and run it through the 4 tests.

Aim for an honest conversation.

Then:

Mark each practice as Keep, Retire, or Rebuild.

• A practice that passes all 4 tests earns Keep.

• One that the team cannot honestly defend against even a single test earns Retire.

• One that still addresses a real problem but whose current form is no longer serving that purpose earns Rebuild.

Then:

• Pick one practice from the Retire list and stop doing it for the next four weeks.

• Pick one from the Rebuild list and commit to redesigning it around the original problem it was meant to solve.

Run both experiments simultaneously.

Then:

After a month, review:

• whether learning improved,

• whether decisions got faster, and

• whether the team has better evidence about what their users actually need.

Use that review to decide whether to continue, extend, or revise.

The most Agile thing you can do

The most Agile thing a team can do in 2026 may be to stop protecting its practices from inspection.

Agile is twenty-five years old.

That is long enough for an idea to move from radical proposal to industry standard to inherited assumption.

The Manifesto’s authors were trying to articulate principles that would help teams navigate uncertainty and learn from their work. The practices that grew up around those principles deserve the same scrutiny that Agile teams are supposed to apply to everything else.

The question to bring back to your team is this:

If we were designing how we work today, with everything we now know, would we choose this practice?

If the answer is yes, keep it. If the answer is no, or if nobody can remember why the answer should be yes, that is the conversation the team has been waiting 25 years to have.

Show your support

Every post on Winning Strategy takes 1 full day of writing. You can show your support with small gestures.

Liked this post? Make sure to 💙 click the like button.

Feedback or addition? Make sure to 💬 comment.

Know someone who would find this helpful? Make sure to recommend it.

I strongly advise that you download and use the Substack app. This will allow you to access our community chat, where you can get your questions answered and your doubts cleared promptly.

Further Reading

• Hybrid Agile. What is it?

• Parameters for Agile Assessment (With Data Collection Template)

• First Principles of Scaling Agile Practices

Connect With Me

• Product Output Vs Product Outcome

• Coaching Model and Mindset For Scrum Masters. What to Coach and How to Coach your Team?

• I spotted 9 problems in a healthy-looking team backlog

Winning Strategy provides insights from my experiences at Twitter, Amazon, and my current role as an Executive Product Coach at one of North America’s largest banks.

Leave a comment

Thank you for the question.

Most Communities of Practice start with good intent. Someone creates a calendar invite, names the group, adds the Scrum Masters and POs, and then waits for learning to emerge.

But this is NOT enough to make a Community of Practice valuable.

A Community of Practice becomes valuable when people feel (not think, FEEL):

• “This helps me solve MY problems.”

• “I learn things here that I can use immediately.”

• “People like me are dealing with similar challenges.”

• “This is not another status meeting.”

• “I have a reason to come back.”

If your Scrum Master and PO CoP have irregular attendance and low-impact conversations, the issue is usually how you have designed that CoP.

The community has probably become a meeting rather than a community.

I have a few suggestions on how to fix this.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Thank you for the question.

I have seen this happening, especially in technically strong teams.

Many teams are good at building things, but not always connected to why those things matter. The team is not engaging with the product problem or the business outcome.

In such teams, Sprint Planning is a conversation about workload. There is no discussion about the purpose.

This is not uncommon. And it does not mean the team does not care.

Often, it means the team has been operating in an organization where delivery is more visible than value. They have been trained (unintentionally, of course) to focus on output rather than outcome.

How do you help such a team?

Let’s find out.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Thanks for the question.

This is one of those situations where a Scrum Master can either become a process police or a coach.

Because on the surface, the problem looks simple:

The team said they would finish something, and they did not.

But if you stay only at that surface level, the conversation quickly becomes unhelpful.

The Product Owner starts thinking, “The team is not serious about its commitments.”

The team starts thinking, “The PO does not understand how complex this work is.”

Leadership starts thinking, “Maybe this team is not performing.”

And the Scrum Master gets pulled into the middle, trying to defend the team while also acknowledging that, yes, the problem is real.

This is exactly the moment where your coaching stance matters.

Repeated carryover is a delivery problem. It is ALSO a signal, “Something about the way we plan, refine, slice, collaborate, or communicate is not working.”

The Scrum Master’s job is not to protect the team from accountability. It is also not to allow accountability to become blame.

The job is to help the team and stakeholders inspect the practices being followed honestly, learn from them, and make the next Sprint a little better than the last one.

Let’s look at where the real issues may be hiding.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

“Not everything that counts can be counted, and not everything that can be counted counts.”
— Albert Einstein

Agile teams often have no shortage of metrics.

The problem is not the lack of data. Agie teams have a lot of it.

The problem is that they have too much disconnected data, which means they don’t have a shared understanding of what that data is supposed to help them decide.

This is the situation where metrics are discussed without changing anything. Everyone can point to numbers, but nobody is sure what they mean or whether they matter.

That is metrics overwhelm.

And it is dangerous because it makes a team “appear” data-informed without helping the team make better decisions.

Subscribe now

If you’re new, then you missed the posts below in the past few weeks

1. The Retrospective that never lead to action

2. How to deal with part time team members with split allocations

3. 11 things Atlassian Rovo AI can't do yet

4. How to define TRULY URGENT mid sprint interruptions?

5. 9 built in ROVO AI Agents for Scrum Masters

6. How would I help my team resolve impediments without depending on me?

Here’s what “metrics overwhelm” looks like

You may recognize metrics overwhelm when conversations sound like this:

“Our velocity is down, but our throughput is up. Is that good or bad?”

“The dashboard says we’re behind, but the team says things are fine.”

“Leadership wants more predictability, so we added more metrics.”

“We measure customer adoption, but nobody knows what adoption level would count as success.”

“We report sprint completion every two weeks, but the product decisions are still based on stakeholder pressure.”

“We have all the data in Jira, Productboard, GA, Amplitude, Power BI, and spreadsheets, but we still don’t know what to prioritize.”

The team collects metrics at three different levels, but mixes them together:

1. Delivery metrics: Are we delivering work smoothly?

2. Product metrics: Are users benefiting from what we deliver?

3. Business metrics: Is the organization getting the result it needs?

Each category is useful. But confusion begins when people expect one type of metric to answer another type of question.

Velocity cannot tell you whether customers value the feature. NPS cannot tell you whether the team has too much work in progress. Revenue cannot tell you whether your refinement process is healthy. Cycle time cannot tell you whether you are solving the right problem.

When the metric is used to answer the wrong question, the conversation becomes confusing.

Why does metric overwhelm happen?

The simple answer is UNCERTAINTY.

Metrics overwhelm usually happens because people are trying to reduce uncertainty.

Product work is uncertain. Agile delivery is uncertain. Stakeholder expectations are uncertain. Customer behaviour is uncertain. Technical complexity is uncertain.

So teams and leaders reach for measurement to build confidence.

The problem is that metrics are often added one by one, without removing old ones and without clarifying their purpose.

None of these is bad on its own. In fact, most are reasonable.

But together, without a decision framework, they become cognitive clutter.

The team ends up with a pile of signals but no way to separate what is important from what is merely available.

What’s a useful metric?

A useful metric should help someone make a better decision.

If a metric does not influence a decision or trigger a conversation, or guide an experiment, it is probably not a useful metric.

One of the questions you can ask your team is:

“What decision will this metric help us make?”

If nobody can answer, pause before adding it to the dashboard.

For example:

The metric itself is not the insight.

Insight comes from interpreting the metric in context and deciding what to do next.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

How Product Owners get overwhelmed by metrics

For Product Owners, metrics overwhelm often shows up as pressure to prove progress.

Stakeholders want to know:

In response, the Product Owner collects more and more data to defend decisions.

But a Product Owner’s job is not to produce enough metrics to satisfy everyone’s curiosity.

Their job is to create enough CLARITY to support better product decisions. That means the Product Owner needs to distinguish between:

The above metrics are not the same.

A roadmap confidence metric may help stakeholders understand uncertainty.

A feature usage metric may help the Product Owner decide whether to iterate.

A cost-of-delay discussion may help prioritize backlog items.

A cycle time metric may help the team improve flow.

A business outcome metric may help evaluate whether the product strategy is working.

When all of these are thrown into a single dashboard without a clear intent, the Product Owner becomes a curator of confusion.

How Scrum Masters get overwhelmed by metrics

For Scrum Masters, the danger is slightly different.

Scrum Masters often care deeply about team health, flow, continuous improvement, and transparency.

Metrics can be very helpful here. But they can also become a subtle form of micro management if not handled properly.

The above metrics can support excellent conversations.

But if the team feels the numbers are being used to judge them, they will adapt their behaviour to protect themselves.

Metrics should help the team see the system more clearly, not make individuals feel watched.

So how do we handle this?

The best way to escape Metrics Overwhelm is by clarifying the decisions first.

Start by asking:

“What decisions do we need to make on a regular basis?”

Then, for each decision, identify the smallest set of metrics that can inform it.

For example:

This approach prevents the collection of metrics that serve no purpose.

Use a “metrics menu,” not a metrics buffet

If you have ever had a buffet at a restaurant, you know that buffets encourage people to pile everything onto the plate.

A menu forces choice.

Agile teams benefit from a small, intentional set of active metrics, which may change depending on the team’s focus.

Below are a few examples of such a metric menu:

#1: If the team is struggling with the delivery flow

Those may still matter, but they are not the immediate requirement for improvement.

#2: If the team is validating a new product idea

The goal is learning, not maximizing output.

#3: If the team is improving quality

The goal is to restore confidence and reduce the demand for failure.

#4: If the team is scaling stakeholder trust

The goal is better alignment and expectation management.

Separate health metrics from target metrics

One distinction is between target metrics and health metrics.

Target metrics

These represent what you are actively trying to improve.

Examples:

• Increase activation

• Reduce onboarding drop-off

• Improve cycle time

• Reduce escaped defects

• Increase trial-to-paid conversion

• Reduce customer churn

• Improve forecast reliability

These metrics are central to business decisions.

Health metrics

These are things you monitor to make sure improvement in one area is not damaging another.

Examples:

• Team morale

• Defect rate

• Customer complaints

• System performance

• Support volume

• Work in progress

• Technical debt indicators

For instance, if the target is to increase feature delivery speed, quality and team health should become health metrics. Otherwise, the team may “succeed” by burning people out.

If the goal is to reduce defects, delivery throughput may become a health metric; otherwise, quality improvement could lead to paralysis.

This distinction helps reduce overwhelm because not every metric gets equal attention.

Some metrics are steering the current change.

Others are guardrails.

How to turn metric data into insight

Data becomes insight when it changes understanding.

Here’s how the progression happens:

1. Data: “Cycle time increased from 4 days to 9 days.”

2. Information: “The increase is mostly in review and testing.”

3. Insight: “Stories are reaching review before acceptance criteria are clear, causing rework.”

4. Action: “We will improve refinement and add earlier QA/product review.”

5. Learning: “If review wait time drops, we know the change helped.”

Many teams stop at step 1 or 2.

They display the data and assume the insight is obvious.

It usually is not.

Here, Scrum Masters can add huge value by facilitating the movement from number to meaning to action.

Qualitative insights

Metrics tell you what is happening.

They often cannot tell you why it is happening.

For instance, if activation is low, analytics may show where users drop off. But they may not explain whether users are confused, unconvinced, distracted, lack permissions, are using a workaround, or are solving a different problem entirely.

If the velocity drops, Jira may show fewer story points completed. But it may not explain whether the issue is unclear priorities, technical debt, dependencies, interruptions, illness, low morale, or overcommitment.

This is why metrics should be paired with qualitative learning:

You don’t have to choose between data and conversation. Use data to focus the conversation.

Metric Purpose Statement

For each metric you keep, define a short purpose statement.

Use this format:

We measure [metric] to understand [question] so that we can decide [decision/action].

Examples:

We measure cycle time to understand where work slows down so that we can improve flow and reduce unnecessary waiting.

We measure feature adoption to understand whether users are trying the new capability so that we can decide whether to improve, promote, or stop investing in it.

We measure escaped defects to understand where quality issues reach customers so that we can strengthen our feedback and testing practices.

We measure team sentiment to understand whether our way of working is sustainable so that we can address overload before it becomes burnout.

We measure activation rate to understand whether new users reach first value so that we can improve onboarding and early product experience.

If you cannot complete the statement, the metric probably does not belong in the active dashboard.

How to clean the current metrics you have

Conduct a workshop with the Product Owner, Scrum Master, Developers, and relevant stakeholders.

Step 1: List your current metrics

Collect every recurring metric the team reports, reviews, or tracks.

Do not debate yet. Just list them.

Step 2: Mark the audience

For each metric, identify who it is for:

• Team

• Product Owner

• Scrum Master

• Stakeholders

• Leadership

• Customers

• Operations

• Other teams

Step 3: Identify the decision

Ask:

“What decision does this metric support?”

If the answer is unclear, mark it.

Step 4: Identify the action threshold

Ask:

“When would this metric cause us to act?”

For example:

• If the cycle time increases for three consecutive weeks, we investigate blockers

• If adoption is below target after launch, we run customer interviews

• If the number of escaped defects increases, we review testing and release practices

• If WIP exceeds the agreed limit, we stop starting and start finishing

Step 5: Remove, archive, or downgrade

Not every metric needs to be deleted.

Some can be:

• If a metric helps us make a current decision, keep it active (Eg: Cycle Time)

• If it protects us from side effects, make it a guardrail (Eg. Escaped defects)

• If it helps us investigate a problem, keep it diagnostic (Eg. Time spent in each workflow status)

• If it has historical value, archive it (Eg. Adoption of last quarter’s reporting feature)

• If nobody uses it, remove it (Eg. Number of backlog items created)

This gives the team permission to stop carrying old measurement habits forever.

The honest truth about Metrics

Data doesn’t create clarity on its own. Clarity comes from a team that has agreed on what they’re trying to achieve, identified the signals that show whether they’re getting there, and built the habit of talking honestly about what those signals mean.

Metrics are a language.

And like any language, they only communicate something when both parties understand the words.

Further Reading

1. How to Discuss Metrics With Your Team?

2. Metrics for Project Based Scrum Teams

3. How Agile Metrics are Getting Misused - And How Can You Fix It?

Connect With Me

• Connect on LinkedIn (I publish every Monday, Tuesday, Wednesday, Thursday and Friday)

• Connect on Community Chat

• Connect on YouTube

Winning Strategy provides insights from my experiences at Twitter, Amazon, and my current role as an Executive Product Coach at one of North America’s largest banks.

Leave a comment

I want to start with something I hear constantly when I talk to Scrum Masters and product teams about mid-sprint interruptions.

They describe the problem as a process problem by saying:

“We need a better intake system.”

“We need stakeholders to stop bypassing the backlog.”

“We need leadership to respect the sprint.”

Those things might be true but…

I’ve come to believe the deeper problem is almost always a failure to build shared understanding about the outcome the team is trying to protect and what it genuinely costs to interrupt the team.

When teams have that shared understanding (something that they can clearly show the stakeholders), urgency becomes a tractable question.

When they don’t, it becomes a power struggle. And power struggles are exhausting… believe me!

In this post, we will learn how YOU can help your team build that shared understanding.

Let’s get started.

Read more

I’ve been collecting notes on those teams on what they actually use, the exact queries they type, and where they’ve hit roadblocks.

This post is that list.

Here’s what I am hoping you learn from this post:

1. What ROVO’s out-of-the-box agents can do for a Scrum Master

2. Examples of how people are using those agents

There’s a lot to cover, so let’s get started.

If you are new to Winning Strategy, here’s what you missed in the past few weeks

1. How to write Acceptance Criteria using AI

2. 13 AI prompts your Product Owner needs

3. How the POs in my teams use AI to turn vague complaints Into Backlog Items.

4. We created guardrails for AI assisted backlog refinement

5. AI Summaries for Leadership

6. How to prioritize defects when everything Is critical?

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Which ROVO Agents are Scrum Masters successfully using?

They are using ROVO to handle repetitive, administrative tasks. The list below is based on what's genuinely in use across the team.

These are out-of-the-box ROVO agents.

So if your team is on Atlassian Cloud with ROVO enabled, you have access to all of them today.

#1. Backlog Refinement: Theme Analyzer

This is being used to scan the backlog and then group issues into recurring themes.

You open it in Rovo Chat and describe what you want in plain language:

• point it at a project,

• paste in a search results page link, or

• give it a JQL query if you want to narrow the scope

The agent handles the rest.

The simplest way to start using it is with the built-in conversation starter prompt below:

“Help me group the issues in project X into themes.”

It will ask clarifying questions if it needs more context.

Scrum Masters are using this agent to map what stories have been piling up and have not been worked on.

One thing to keep in mind:

This agent was built primarily for analyzing customer feedback and support tickets at scale. Scrum Masters are applying this to backlog refinement, and it works well for that, but the output reflects what’s written in the tickets.

Incomplete descriptions and unclear titles lead to incorrect groupings.

#2. Backlog Refinement: Work Item Organizer

This out-of-the-box agent is being used to handle the backlog cleanup.

Things like

• moving items into sprints,

• linking loose tickets to epics,

• flagging old written issues that haven't been touched in months

It's typically run as a pre-refinement tidy-up.

Like the Theme Analyzer agent, you interact with it conversationally in Rovo Chat. You describe what you need, it proposes the actions, and you confirm your decision before anything changes.

Queries Scrum Masters are using:

• “Find all backlog items not linked to an epic and suggest which epics they belong to”

• “Identify issues in the backlog that haven’t been updated in over 90 days”

• “Move all story-type items estimated under 5 points into the next sprint”

Two things you must know before you use it:

First: bulk actions are capped at 20 work items at a time. If you ask it to move or update a large batch of issues, it will stop at 20. On an active backlog, this limit comes up quickly, so expect to run it in multiple passes rather than in a single sweep.

Second: the agent proposes actions and waits for your confirmation. It doesn’t make changes without your approval. That’s useful to know before you set expectations with the team about what it can do automatically.

#3. Sprint Planning: Readiness Checker

Once configured, the Readiness Checker runs automatically when a new ticket is created.

It checks whether the ticket has:

• acceptance criteria,

• a definition of done, and

• enough context to actually be planned

If something's missing, it adds a comment to the issue flagging what's needed before they reach planning.

You can use it either:

1. manually through Rovo Chat (you open the agent, point it at a ticket, and get feedback immediately) or

2. automatically via a Jira automation rule

The automation approach is more scalable, but Scrum Masters also use it manually to spot-check individual tickets before planning.

Note: We will discuss each agent in more detail in future posts.

#4. Sprint Planning: Breaking down work

ROVO chat

Through Rovo Chat, you can take a large Epic and ask ROVO to break it into smaller, more actionable User Stories.

It pulls context from linked Confluence pages, linked issues, and any attached documents to inform the breakdown. We have a Confluence page set up with WAHZUR and T-Split techniques that it usually refers to.

Queries Scrum Masters are typing:

• “Break this epic into individual user stories with acceptance criteria.”

• “This ticket is too large for one sprint. Suggest how to split it into smaller pieces.”

• “Draft a description and acceptance criteria for this ticket based on the Confluence spec linked to it.”

Work Item Planner

This out-of-the-box agent’s main use is to break large items into actionable steps.

Note: “Work item Planner” is separate from “Work item Organizer.”

The Organizer tidies up what already exists, the Planner creates what doesn't yet exist.

You can use “Work item Planner” in two ways:

The first is directly inside Jira, without going through chat at all.

1. Open an epic, go to the child work items area, select Add, then Suggest

2. Rovo generates a list of proposed child items based on the epic's description

3. You review them, remove what doesn't fit, and create the ones you want. This is the simpler starting point for most teams

The second is through the Work Item Planner agent in Rovo Chat. The agent's built-in conversation starters are:

• "Break down my epic into subtasks" and

• "Turn this page into a list of tasks."

One thing that applies to both methods: the quality of the output depends directly on what's written in the epic description.

Rovo only reads what you give it. If you want to use a Confluence page with instructions, you need to link that page in the epic description first. It doesn't automatically scan linked issues or attached documents.

#5. Retrospectives: Transcript Insights Reporter

The Transcript Insights Reporter is a general transcript analysis agent.

It was built primarily for customer interviews and user research transcripts. Scrum Masters are applying it to retrospective meetings, and it works for that purpose, but it is important to know that this agent wasn't purpose-built for retros.

Here’s the workflow:

1. After the retro, you open the agent in Rovo Chat, paste in the transcript (if you have one recorded), and ask it to identify themes and suggest next steps

2. It returns a structured text response (1) with themes from the discussion, observations, and (2) recommended actions

3. That output then needs to be reviewed and turned into Jira tickets manually, or via a follow-up prompt.

4. The agent surfaces the analysis; it doesn't create tickets on its own

The agent’s built-in conversation starter is:

“Extract insights from this transcript and turn them into interview notes.”

For retros, Scrum Masters use the following prompts:

• “Here is the transcript from our sprint retrospective. Identify the main themes and suggest action items for each”

• “What were the main concerns raised in this meeting, and what actions did the team agree to?”

One thing worth knowing:

There is a separate agent called the Meeting Insights Reporter, launched more recently.

That one doesn't work from a pasted transcript. It searches meeting notes and transcripts already saved in Confluence via Loom. If your team doesn't use Loom, this agent isn't available to you. The Transcript Insights Reporter is the one that works with a transcript you paste in directly.

Also…

The silent retro caveat applies here:

If your retrospective runs mostly on written stickies and dot voting without verbal discussion, there's no transcript to work with.

The Transcript Insights Reporter agent is useful for teams where the retro is conversation-based, and someone is capturing or recording it.

#6. Blocker Detection: Blocker Checker

Blocker Checker is an automation agent.

It is designed for use inside a Jira automation rule. It doesn't run on its own, and it's not something you interact with through chat.

Here's how they have it set up:

1. You build a Jira automation rule with three parts: a trigger, the Blocker Checker agent, and a follow-up action

2. The trigger watches for a condition, typically a comment added to an issue containing language that suggests the work is stuck

3. The rule calls the Blocker Checker, which reads the context and evaluates whether the issue is blocked

4. A second action in the rule then uses the agent's response to transition the issue status to Blocked

The agent signals; the automation executes.

This means the setup requires more than just enabling the agent. You need to:

1. configure the automation rule,

2. write the trigger conditions,

3. connect the agent, and

4. add the status-change action

Example of how it runs:

Let’s say a developer comments at 2 pm that they're waiting on sign-off from another team. The rule fires, and the issue status changes to Blocked the same afternoon. The Scrum Master sees it on the board without waiting for the next standup.

That difference in response time (same day versus next morning) is where this automation is useful, particularly for teams where impediments tend to surface late.

#7. Sprint Reporting

There is no dedicated out-of-the-box agent for sprint reporting. What Scrum Masters are doing here falls into two approaches.

1. First is using Rovo Chat directly for end-of-sprint summaries

You ask it to pull data from the completed sprint and return a structured summary:

• completed items,

• carryover,

• current blockers, etc.

For single-sprint summaries, this works reasonably well, though the quality varies depending on how consistently the team has been maintaining their Jira data.

Prompts that work:

• “Summarize Sprint 12 results: how many points were planned, how many were completed, and what carried over”

• “List all incomplete items from the last sprint and their current status”

• "Generate a sprint report for stakeholders covering blockers and carryover items"

Avoid this:

Avoid asking Rovo for velocity. Velocity requires comparing completed work across multiple sprints, and as covered in the limitations section below, cross-sprint reasoning is where Rovo produces unreliable results. Velocity is better pulled from Jira's built-in Velocity Report than asked of Rovo Chat.

2. Second approach

This is used by Scrum Masters who need consistent output every sprint. They are building a custom agent in Rovo Studio specifically for sprint reporting.

You define what the agent should pull, how to structure the output, and connect it to the relevant Jira project. It takes some setup, but once configured, it produces the same format each sprint without requiring a new prompt each time.

Note: We will learn how to build these in future posts.

#8. Finding issues without writing JQL

Scrum Masters who don't write JQL regularly use ROVO Chat to convert plain-language questions into Jira queries and run them.

Queries Scrum Masters are typing:

• “Show me all bugs assigned to the backend team that have been in progress for more than five days”

• “Find all tickets with no assignee that were moved into the current sprint”

• “List all issues flagged as high priority that haven’t been updated in the last week”

#9. Cross-Team Dependencies: Deep Research

In environments where multiple teams share dependencies, ROVO's Deep Research mode helps SMs identify cross-team dependencies.

Deep Research scans across:

• projects,

• Confluence pages and

• connected tools

and compiles a report on the current status and dependencies. Before using it, there are two things to know:

1. It takes up to 15 minutes. This is not a quick process

2. It will first ask you clarifying questions. The current version of Deep Research doesn't run immediately on submission. It first asks targeted questions to confirm the scope

Prompts used:

• “Research the current status of shared dependencies between Team A and Team B for the Q4 release”

• “Summarize progress on the platform migration across all teams involved”

• "What Jira issues are blocking work across more than one team right now?"

One limitation:

Deep Research only surfaces what was recorded.

If dependencies between teams exist but were never documented in Jira or Confluence, tracked informally, discussed in meetings that weren't captured, or managed through direct messages, they won't appear in the report. The output reflects the quality of what your teams have been writing down.

Bottom line on ROVO output quality

Before wrapping up, there is one thing that cuts across every agent on this list.

Rovo works with what’s in the system.

The Theme Analyzer groups what’s written in your tickets. The Readiness Checker evaluates what’s been filled in. Deep Research synthesizes what’s been documented. Every one of these agents is only as useful as the data behind it.

The Scrum Masters I work with who get consistent, reliable outputs from Rovo are the ones on teams where Jira is reasonably well-maintained, i.e. tickets have descriptions, epics are linked, and sprints are closed properly.

If your team is earlier in that journey, the agents still work, but you’ll notice the gaps quickly. In some ways, that’s useful information. Rovo makes bad data hygiene visible faster.

Next Post…

In the next post, we will explore what can't be done with ROVO yet and when it makes sense to stop waiting for Atlassian to close the gap and start building CUSTOM AGENTS of your own.

Show your support

Every post on Winning Strategy takes a few days of research and 1 full day of writing. You can show your support with small gestures.

If today’s post gave you a new perspective or a “lightbulb moment,” here are three quick ways to help it reach more people:

• Enjoyed the read? Hit the 💙. It tells the algorithm this matters.

• Have a thought? Drop a 💬. I read every single one.

• Know a friend? Forward this to them.

Don’t forget: The best way to stay connected is on the Substack app. It’s where our real-time chat happens and where I can answer your questions directly.

Connect With Me

• Connect on LinkedIn (I publish every Monday, Tuesday, Wednesday, Thursday and Friday)

• Connect on Community Chat

• Connect on YouTube

Winning Strategy provides insights from my experiences at Twitter, Amazon, and my current role as an Executive Product Coach at one of North America’s largest banks.

Leave a comment

This week’s recipe will help you put those prompts to work as a team.

You will use this recipe when a story is clearly too big, and you need it ready for the sprint before the next planning session.

Note: It won’t replace your regular backlog refinement.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

We have all manually split user stories.

A few years ago, I introduced WAHZUR and T-Split techniques to split large user stories into smaller ones. If you don’t know what they are, watch the video below, where I talk about these two techniques.

Now imagine doing all of that in a fraction of the time, right inside Jira, using Atlassian’s own ROVO AI.

What is ROVO?

ROVO is Atlassian's built-in AI assistant. You access it via the chat icon in the top-right corner of Jira. You can also access it by typing /rovo in any issue description or comment field.

This post walks you through the simplest way to split user stories with ROVO AI utilizing WAHZUR and T-Split techniques.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

It is one of those Product activities that everyone knows matters, yet it's routinely underdeveloped or written too late.

Your team has probably already used AI to write acceptance criteria.

You typed in a user story, got a list of conditions back, and pasted it straight into Jira. Fast. Easy. Done.

Here’s the thing, though. This speed comes at a cost.

I have seen that this acceptance criteria is usually vague, with missing edge cases and conditions that can’t actually be verified.

AI can write AC. That part is easy. Writing AC your team can trust is a different story.

Used without a clear process, AI gives you false confidence. The gaps are still there, they're just harder to spot.

In today’s post, I am going to give you a simple 3-phased process my teams use to write acceptance criteria every sprint.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Last Thursday, we learned how the POs use AI to turn vague signals into refinement-ready backlog items.

When this is done right, your PO can start the refinement sessions with:

1. a categorized signal,

2. a one-paragraph problem statement, and

3. a selected user story

But what if it is NOT done right?

Then what?

If you are a Scrum Master, how would you help the PO with this process and ensure it happens consistently?

Before we get into the mechanics of this, let me clarify something:

Your job as a Scrum Master IS NOT to run the AI prompts. Your job is to ensure the prompts are run by the PO and the team on a regular basis.

And to do that, you need to know:

• exactly which prompts exist,

• what each prompt produces, and most importantly

• when to push the PO to use the ones they’re skipping

That’s what this post gives you.

There are 13 prompts in this AI discovery toolkit, organized across various groups.

For each prompt, I’ve added a single pre-refinement question that you can use as a trigger to nudge your PO in the right direction. You don’t need to become a prompt engineer. All you need is to know the right question to ask.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Every refinement session has a tax.

The first 20-30 minutes where the team figures out what they’re actually building, why it matters, and whether it’s even the right thing.

That tax is paid because, usually, the PO walks into the Refinement sessions with a vague signal and not a prepared backlog item.

On Thursday, I showed you how some of the POs I work with have started using AI to turn such vague complaints/signals into refinement-ready backlog items in three steps, before the team ever gathers for the Refinement session.

Today, you get the template and facilitation guide to actually run it.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Here’s a typical refinement session:

Someone complains: "Users are saying they can't find anything in the app. Sales is losing deals over it."

The team comes together, one team member reads the complaint (or a vague requirement, or an idea, etc.) aloud, and for the next 40 minutes, the team tries to figure out:

• WHAT the problem actually is,

• WHOSE problem is it, and

• whether this is even the RIGHT THING to build

The effort is usually parked or rushed because “the thing” is not yet ready for refinement, and other items are lined up in the queue.

This happens because that complaint or requirement is a vague signal, and the work of turning it into a clear backlog item usually falls onto the PO.

Most POs do this in their heads between meetings, which results in a vague complaint becoming a vague user story.

This is now being changed thanks to AI.

Some POs I work with are using AI to quickly and easily turn these kinds of vague ideas into refinement-ready Backlog Items.

Here’s how they are doing it.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Welcome to the Sunday edition of the Winning Strategy Weekly Recipe.

This past Thursday, we discussed the risks my team encountered when using AI for backlog refinement.

Specifically, we looked at how AI behaves and how it confidently generates acceptance criteria, system behaviours, and trade-offs that look completely plausible but are often fabricated.

I shared the 7 essential guardrails my teams use to ensure the tool accelerates the work without taking decision-making ownership away from the team.

But understanding the guardrails and operationalizing them in an Agile environment are two different things.

To benefit from AI without the rework, we need a system that translates those principles into steps during our live sprint cadence.

This post bridges that gap.

You receive an artifact pack that you can add to your backlog workflow today, enabling your team to use AI for refinements with minimal risk.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

YES, we did.

And it went great… for about two weeks.

Then we spotted a problem.

The risk of using AI for refinement is that it sometimes produces bad output that is indistinguishable from good output.

I've spent a lot of time thinking about where AI helps in product discovery and delivery, and where it introduces risk that teams don't see until it's too late. In this post, I share 7 guardrails that are essential for any team using AI for backlog refinement.

I have conducted these tests across various refinement sessions, with multiple teams iteratively enhancing them.

Before I walk through each one, I want to name the one underlying principle that forms the basis of every guardrail.

Here it is. 👇

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

There’s a famous Bezos-ism that I think about a lot:

“Your margin is my opportunity.”

The idea is that wherever there’s inefficiency in a system, there’s an opportunity for someone to build something better.

Status meetings are exactly that. They are pure margin. They’re organizational overhead. And the opportunity is to replace them entirely with something that is async, consistent, and… let me say it… AI-generated.

In this post, I will share the system I've seen some Scrum Masters using to generate AI-powered reports for executives and stakeholders.

And let me tell you… they’ve been quite successful doing it.

But…

Before we kill the STATUS meetings and replace them with AI, we have to understand what the STATUS meetings are actually used for.

What’s their purpose?

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

The idea was simple:

Spikes shrink estimates ONLY WHEN they shrink the solution space, not when they confirm a solution exists.

What we need is a “Decision Spike.”

A Decision Spike is a session where we focus on clearly defining what will be built, what won't be, and the clarity needed to keep the timeline on track.

In today’s weekly recipe, I’m sharing the template to help you run a workshop for a Decision Spike.

It’s a 60-minute (time may vary) session that takes a stakeholder request (often a vague request) and turns it into confirmed increments with estimates, assumptions, and (most importantly) clear NOT-TO-DOs.

Note: This is more of a scoping session than a sprint planning. It's a great opportunity to get a clear understanding of a request as it first comes in, ideally before anyone starts writing anything.

Let’s get started.

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

Read more

Robert Falcon Scott’s team did extensive preparation. They studied the terrain, tested their equipment, and built confidence that the journey was technically feasible. They knew they could make it.

Roald Amundsen’s team did something different. They picked a specific route. They pre-positioned supply depots at exact coordinates and decided, before they left, what they would NOT bring with them, what they would NOT attempt, and what had to be true at each checkpoint for the expedition to continue.

Scott’s team proved the trip was possible. Amundsen’s team defined the trip they were actually going to take.

Scott’s team didn’t succeed.

Why?

What happened?

If you are new to Winning Strategy, check out the posts below:

1. Weekly Recipe: The PO Q&A System

2. How to deal with the one difficult team member in your scrum team?

3. Weekly Recipe: How to deal with the one difficult team member in your scrum team?

4. How to prioritize defects when everything is critical?

5. Weekly Recipe: Prioritize defects when everything Is critical

Got an urgent question?

Get a quick answer by joining the subscriber chat below.

The Spike for estimation problem is a Strategy problem

We all know about spikes?

If you haven't encountered it (lucky you, in some ways), here's the short version:

When a team faces a major technical unknown, they timebox a few days to build a quick prototype and determine whether the scary thing is actually scary.

If you want to explore the topic further, I have written a post on Spikes that you can check out at the link below:

It's a genuinely good idea.

It comes from the Extreme Programming tradition, it's battle-tested, and mature teams use it all the time.

Here's the thing, though. Spikes don’t always deliver something!

A team runs a spike. The spike succeeds, BUT the estimate doesn't change.

The prototype works.

But when the team sits down to estimate the actual time, the range is just as wide as before. The team is still unsure about the requested time commitment.

How is that possible?

It’s possible because most spikes (in general) solve for the Technical uncertainty. And that distinction between the uncertainty you think you have and the uncertainty you actually have turns out to be one of the most interesting (and underappreciated) dynamics in product development.

Let me show you what I mean.

Story of a mature team

A product team I worked with needed to ship enterprise SSO. Azure AD integration (to be precise) for a big B2B prospect.

Leadership wanted an answer to:

“Can we commit to delivering this in 6 weeks? Yes… or No?”

The team did what any experienced team would.

(1) They timeboxed a spike (3 days). (2) They picked the riskiest technical area (OIDC/SAML. (3) They aimed to “remove uncertainty.”

Which uncertainty? The technical one.

This was the output (what they delivered):

• a prototype that successfully authenticated against Azure AD in a sandbox

• a short document stating: “We can use OIDC; libraries exist; token validation is straightforward”

• a diagram of the auth flow

The spike worked. But the estimates?

They had no estimates.

They still had NO answer for how much time this will actually take to be production-ready.

Stakeholders were confused: “But you proved it works, why can’t you estimate it now?”

What went wrong?

The spike answered one question:

Can we technically authenticate against Azure AD?

Yes. Definitely yes.

But that question was never driving the estimates.

The estimates depended on a certain kind of uncertainty. The kind that involved a long list of decisions and dependencies.

Things like:

• Do we need just-in-time user provisioning or SCIM or manual invites?

• How do Azure AD groups map to internal permissions?

• How do we handle multiple enterprise tenants and domain discovery?

• What does the security team require before anything goes live?

• What happens with disabled accounts, renamed emails, and duplicate identities?

• Can we roll out per customer behind a feature flag?

• What does the support team need for the inevitable Monday morning "SSO is broken" fire drill?

None of these is a technical feasibility question. They're decision questions. Policy questions. Cross-team coordination and dependency questions.

Technical uncertainty is the easiest uncertainty to identify, so it is the uncertainty that gets most teams Spike. But estimation uncertainty is usually dominated by scope ambiguity, unknown decisions, and external dependencies.

A spike on Technical uncertainty does approximately nothing for the Estimation uncertainty.

Understand it like this:

Let’s say you're trying to estimate how long it'll take to drive from New York to Los Angeles.

The unknown in your mind is "can my car handle the Rocky Mountains?"

So you test the car on a steep hill. It handles it fine. Great.

Can you estimate your trip?

No!

Why?

The reason you can't estimate the trip is because you haven't yet decided whether you're stopping in Chicago, you don't know if the highway through Kansas is closed, and your friend who's supposed to meet you in Denver hasn't confirmed dates.

The car works. But the trip is still unplannable.

Why did the experienced team get this wrong?

This is the part I find genuinely fascinating, because each reason below corresponds to a different type of cognitive bias.

#1: Knowledge Without Decisions

The team above came back from the spike saying, “We can do OIDC. We could map groups later. We might need SCIM depending on customer requirements.”

Could. Might. Depending.

There's a concept in strategy called "real options," which is the idea that maintaining flexibility has value. And it does! In certain contexts.

But when you're trying to estimate something, optionality is the last thing you want.

Every open option multiplies the “solution space.”

The team were estimating a decision tree. And the expected value of a decision tree with seven unresolved branches is... wide.

#2: No Constraints

The team didn’t decide the Scope.

No one locked down statements like:

• “Phase 1 will not support group-based roles”

• “Phase 1 supports one tenant per customer”

• “Phase 1 is invite-only; no SCIM until Phase 2”

Without constraints, every “maybe” stayed inside the estimate.

As a result, estimates remained vague because, for stakeholders, “done” still meant “everything enterprise customers might want.”

#3: Ignored Dependencies

In most teams, the most likely thing to blow up the timeline is WAITING.

Waiting for:

• security review lead time

• infra changes (redirect URIs, secret storage, cert rotation)

• customer IT coordination (metadata, app registration, testing windows)

The spike the team performed didn’t secure any commitments or timelines from those dependencies.

As a result, the team still couldn’t forecast dates because the critical path was dependent on 3 different teams… Security, Infrastructure and IT.

How did they fix it later?

The team re-ran the spike.

They now had a clear goal in mind for the technical spike:

"By Friday, the spike will help us know exactly what we are shipping in Phase 1, what we are clearly not shipping in Phase 1, and what external conditions must be true for the timeline to hold."

Here’s what the Spike looked like:

Timebox: 2 days

Inputs from: Product + Delivery Lead + Security rep + one engineer

Decision to make by the end of the spike: What is “in scope” and what is “out of scope” for a 6-week timeframe (Phase 1)?

For example:

Scope:

• OIDC only (no SAML in Phase 1)

• No SCIM in Phase 1

• Role mapping is manual (group sync deferred)

• Per-customer feature flag rollout

• "Done" = audit logging + v1 support runbook

Evidence required:

• thin vertical slice in staging with auth, feature flag and logs

• lightweight security sign-off via threat model checklist

• written non-goals included in the epic

Phase 1 (6 weeks) backlog change required. Splitting the epic into 3 increments with separate estimates:

• Increment 1 (2 weeks): SSO login behind flag, single tenant

• Increment 2 (2 weeks): Admin config, audit logs, runbook

• Increment 3 (2 weeks): Role mapping and SCIM (later)

Estimating this was straightforward.

🎯 Weekly Recipe

Stay tuned for Sunday’s Weekly Recipe, where I’ll share a step-by-step guide for running decision spikes, including templates and facilitation tips.

What did the team learn?

Let’s zoom out for a second

In strategy, there’s a distinction between learning and committing.

Both are valuable. But they serve different functions, and confusing them leads to a very specific kind of organizational frustration, the feeling that you’re doing all the right things and somehow not making progress.

A traditional spike is a learning tool. It says, "We don't know if this is technically feasible, so let's find out." It works great when technical feasibility is the binding constraint.

But on most mature teams, building the thing is rarely the constraint. The binding constraint in such teams is deciding what to build, coordinating external dependencies, and defining what "done" means in a way everyone agrees on.

So the team needs a technical spike that can help them make these decisions.

A Decision Spike.

A decision spike is a commitment tool. It says,

"We have enough information to decide. The decision will help us reshape the backlog."

So, how do you tell if your spike will actually help?

Here's a simple test.

Spikes increase estimate accuracy ONLY when they shrink the solution space. Which means it should produce at least one of the following:

1. A decision (we chose X, not Y)

2. Constraints/non-goals (we will not do Z in this release)

3. The smallest shippable slice (this is what ships first)

4. Dependency commitments (security/infra/customer IT: who, what, when)

5. A rewritten backlog (split stories/epics that reflect the new decision)

If your spike does not end with the above, then that's research. Research is valuable! But it won't result in an accurate estimate.

Next time your team raises an estimate spike

Even experienced teams can fall into the trap of running spikes that feel productive but fail to answer the required question.

Next time someone proposes a spike for estimates, ask this:

“What uncertainty will this spike remove, and how will that reduce the range of our estimate?”

If the answer is clear, run the spike.

If the answer is vague, i.e. “we’ll learn more about the problem space,” that’s fine, but set expectations accordingly. You probably won’t be able to provide estimates.

The bottom line: spikes help in estimates when they shrink the solution space, not when they confirm a solution exists. The best teams don’t just spike the technical risk. They spike the ambiguity. And those are almost never the same thing.

That’s it for today!

So…Why Scott’s Team Didn’t Succeed

Let’s close the loop on the story we started with.

Why didn’t Robert Falcon Scott’s team succeed in their race to the South Pole, despite knowing the journey was technically possible?

Because Scott’s team ran a Technical Spike, while Amundsen’s team ran a Decision Spike.

Scott’s preparation proved that the equipment worked and the terrain could be crossed. But just like the engineering team that proved OIDC authentication worked in a sandbox, Scott’s team gathered Knowledge Without Decisions.

Amundsen, on the other hand, shrank his solution space. He committed to one method of transport (dogs). He defined strict non-goals (leaving behind anything that wasn't absolutely essential for survival and speed). He secured his dependencies by laying depots at exact, mathematically calculated coordinates before setting out

In short, Scott confirmed the expedition could work. Amundsen defined exactly how it would work.

The result is what we all know.

Show your support

Every post on Winning Strategy takes a few days of research and 1 full day of writing. You can show your support with small gestures.

Liked this post? Make sure to 💙 click the like button.

Feedback or addition? Make sure to 💬 comment.

Know someone who would find this helpful? Make sure to recommend it.

I strongly advise that you download and use the Substack app. This will allow you to access our community chat, where you can get your questions answered and your doubts cleared promptly.

Further Reading

1. SPIKES - 101

2. Non Functional Requirements are Ignored

3. I spotted 9 problems in a healthy looking team backlog

Connect With Me

• Connect on LinkedIn (I publish every Monday, Tuesday, Wednesday, Thursday and Friday)

• Connect on Community Chat

• Connect on YouTube

Winning Strategy provides insights from my experiences at Twitter, Amazon, and my current role as an Executive Product Coach at one of North America’s largest banks.

Leave a comment