A developer's guide to taking action: Evidence vs. Instinct

Acting based on evidence is slow and stable, acting on gut instincts is rapid and risky – every organization needs a mix of both, but how do you know which to use and when?

Every situation will ultimately be based on some amount of incomplete information, otherwise, there wouldn’t be so much of a choice as a straightforward direction. This article will explore the general strategy for making a value call when managing projects based on the available information at hand.

TL;DR

Different people will have different approaches and dispositions, acting according to a sound process or based on experience. Each method, by itself, will have shortcomings that the other will make up for; so there must be an avenue for communication between them. One route to attain this is by clearly defining project values that can be placed in a hierarchy. Doing this will make for better communication and result in a higher rate of project success.

The Iron Triangle of project management decision making

It may take a bit of soul searching, but everyone, whether in sales, development, marketing, or administration, can relate to this common scenario. The scenario involves two sides to a coin. Flipping the coin, you get one of the following:

Side one - Time and resources
Time and resources are available and allow the collection of necessary decision-making information to better improve the project scope and the likelihood of a successful project outcome.
Side two - Deadlines and limited resources
A deadline or limited resources pressures a snap decision to be made with some major pieces of information still missing for the project scope. A successful project outcome is not guaranteed and scope may change.

This type of scenario, or a variation of it, is common in project management and to make a decision we can use something like the Iron Triangle model of project management constraints. This model is useful when you have three important value metrics for a project but can only act on two of them. The values chosen will result in different outcomes.

In this scenario, the values might be Time, Resources and Scope. Depending on the two values selected in a hierarchy determined to be most important to the project, the outcome of the project can be drastically different.

Refining values important to a project

Anything so abstract and generic as the above diagram is of little use in a real-life situation without knowing what the important values for consideration are. When refining the values, if we think too specifically, it may cause disagreement on the details. But, if we think too broadly, no one will know what details are actually being considered. We need a way to determine our values and, like Mama Bear’s porridge, they need to be just right.

The following examples will help us understand how to come to the right set of values.

Example 1: Running the algorithm — An analogy for gathering evidence and using experience to determine values

When do we have the information we need to make a decision on values for our scenario? The algorithm, or set of rules we will follow, we can use to determine this is as follows:

Get better information
Determine if we can make a decision
If yes, make a decision. If no, repeat from step 1

To help us understand how this works, let's imagine a pendulum – a string with one end fastened to a fixed point and the other attached to a swinging weight. In this scenario, our goal is to guess where the pendulum weight will rest when it’s still. Easy, right? The weight is always the same distance from the fixed point and it will always come to rest at the bottom. It isn’t a stretch to assume that you will have the good instinct to where that will be.

Running our algorithm, our understanding (instinct based on experience) of the situation gives us enough confidence and we can make a decision quickly.

Now consider this twist on the situation: we place a strong magnet at some random point underneath the pendulum weight. Its final rest position will now be somewhere between the natural rest point at the bottom and the pulling influence of the magnet. In the beginning, before the pendulum swings, you will be unsure where the weight will end up and a decision can’t be made. Watching carefully as the pendulum swings, every time the weight passes over the magnet you see it acting erratically. A handful of passes later, a pattern emerges and you are better able to derive a more accurate guess of its endpoint.

Running our algorithm again, initially, we couldn’t make a decision because of the unknown influence of the magnet and the erratic movement of the weight as it passes it. However, as time goes on and we become more familiar, a pattern emerges that gives us a better understanding of where the weight will end up. We don’t know for sure, but we have enough of an understanding (based on evidence) to allow for a decision to be made.

A common value for estimating projects is time, which is a challenge on its own

Anyone who has estimated the time a task or project should take knows how easily the estimate can come in over or under budget. Like the pendulum, only familiarity and experience with a particular situation makes accurate estimates more likely (typically by those closest to the situation – the development team, product owner, etc.). It takes attentive observation over time to be able to suss out the reasons why estimates are off-target and, like in the pendulum analogy, there can be environmental circumstances that have a measurable effect on an estimate.

When estimating time, making snap decision estimates are rarely accurate, but investing resources to investigate and construct a perfect scope and estimate (i.e. waiting for a pattern to emerge) has diminishing returns for the vendor and possibly real consequences for the client (i.e. increased cost, pushed deadlines, etc.). Plus, it’s also an ongoing challenge to have effective processes in place to generate over time the information needed for making a decision. So, when is the algorithm that is generating useful value information no longer valuable to run? That’s a good question and there isn’t always going to be a good answer.

Example 2: Late night snacks — An analogy for how values affect scope

For this next example, let’s pretend we’re going into the kitchen for a late-night snack. It isn’t much of an adventure, but it will serve to make a point. When you get that late-night craving for some sweet and salties after a marathon of Netflix, an interesting thing happens. While you are opening the cupboard, your brain is firing on hyperdrive, focused on the action. However, it's doing this in a counter-intuitive way. Your brain knows that you set out with a highly-valued, narrowly scoped goal in mind, “get snacks,” but, when you enter the kitchen you didn’t notice the small pile of crumbs on the stove, or that there were dirty dishes in the sink, or that the light was on before you came in. The vast majority of our sensory stimulus is filtered out by our brain so that we can focus on what matters, the snacks (the scope). It may not seem profound, but consider this in terms of software development.

I have personally had the experience of beginning a task with a clear scope in mind: go into the code (the kitchen) and fix the bug (get snacks!). While doing this task, I entirely fail to notice the bad function name, the comment explaining why something was architected this way, and other code constructs not meeting code standards (the crumbs, the dishes, the light). It’s not even that I saw it and chose to ignore it – I literally didn’t see it at all! This means that I am potentially missing valuable information.

Instead of being hyper-focused on scope, I should have instead set up my goal as: go into the code and fix the thing while ALSO improving existing code while I’m at it. Keeping a tight scope on what I was working on had the short-term benefit of taking less time and staying within scope, but with the potential future cost of being able re-orienting to the code again later or provide valuable information for the next estimate. The values we establish have an effect on the things we do and, depending on the project or task, we need to consider this effect.

The danger of not considering all values when decision making

Referring back to our Iron Triangle explanation from earlier, we can begin to illustrate how the values determine to be important can have consequences when not considered fully. Here’s the situation: Jan and Nico are in charge of making an update to their surveillance electronics website. The ongoing values they have identified that are important for their project are Conversion Rate, Site Visit Time and Traffic Growth. Right now they have a budget for two of three possible actions that support these values:

Throw a sales event
Influence sales through a discount or promotion to support conversion rate and traffic growth.
Refine the checkout user experience
Make it faster for customers to make a purchase to support conversion rate and site visit time.
Do some outreach marketing
Drive traffic to the website to support traffic growth and site visit time.

The Iron Triangle for this project and scenario would look something like this:

Let’s assume that Jan has NOT considered the underlying values involved. She simply wants to upgrade the user experience of the site and drive traffic, because this is what she values. After all, refining the user experience is something they have wanted to do now for several months. If this is done and we get more traffic, sales will increase. Nico has taken a vacation and therefore can’t offer input, so, Jan makes a snap decision to invest here. What Jan will painfully learn in this scenario is that her focus on the values important to her, without considering all options, will leave her unaware of the real opportunity.

When Nico returns, he asks Jan what prompted the decision to work on the user experience. “We need to prioritize how quickly our customers can complete checkout!” Jan exclaims. Her reasoning is that each cart abandoned to a lengthy checkout is an opportunity missed. She’s right, but with Black Friday right around the corner, Nico points out that the primary values to act from should have included conversion through a competitive sales event since a huge number of sales are typically made during this major shopping event. Focusing on this value along with outreach marketing would give them more leverage to ride the wave that Black Friday is. Without competitive pricing at this time, the financial impact will be felt as customers shop for the best deals out there.

Tip: Know your team and assign tasks that match their values

Clarifying your values ahead of time has many benefits. This is mostly because the problems that arise from not having clear values, as we saw in the example above, will continue to be problematic across many different scenarios. By defining your values and then sticking to them you are better able to make conscious adaptations to different situations.

Consider another example of a team that values, and is successful at, promptly shipping new features. The team, as a result of their perceived values, might have them unwilling to consider taking on a new coding standards process, even if it is relatively low-hanging fruit and straightforward to implement. The values of this team place importance of functionality at the cost of quality. Foreknowledge of this and clarity of the values important for a project lends to a more strategic approach when distributing the work to the team or individual team members. A team with values that match the goal of the task should get the task for the best result.

If the wrong team is given for the goal of the task, a conversation about their blind spots may arise as a result. If no one has considered how a hierarchy of values affects setting priorities for a task, this conversation could come off as critical to the team, even mean, and be counterproductive.

Taking action with confidence

“We have enough information, let’s make a decision.”

Bringing this back to the start, it takes experience and many iterations of estimating to mature confidence in your intuition, or a great deal of relevant information and data processing tools to have rock-solid evidence. In either case, that knowledge is hard-come-by and often costly (in time, money and energy) which leads to everyone believing that they have the best solution (because they have worked hard to come to their solution). Having a consistent, well-defined framework of values when encountering each new situation eliminates much of the noise and personal conflict that can come from a lack of information in a complex situation.

To help give understanding to this through one last example, here’s a simple statement that can be read a number of different ways:

8,000 people visit a website per day, on average for less than one minute each, with a visitor growth of 5% per month, and having a net conversion rate of 1%, of which 7.5% are return customers.

You can take a lot from that information, but what’s important? Individuals that look at the world as chaotic potential have difficulty communicating with those that only see hard facts. But knowledge not shared is knowledge wasted. A hierarchy of values here would bridge some of those gaps so that knowledge can move back and forth between people, letting them arrive at a viable value-based solution.

If everyone on the team understands that we, as a group, value growth over conversion rate, and conversion rate over visit timed, our values are clearly established and understanding the importance of the statement becomes simpler. Now other forces may challenge our hierarchy of values, but by establishing our values we are creating a healthier balance between evidence and instinct to produce meaningful and cooperative action.