Larger companies often have trouble introducing an agile way of working. Not occasionally multiple causes are identified. But it all comes back to one big cause: the feedback loop is too long. And not only between organization and end-user (or customers): also between teams, teams and management and within teams. Continuously validating whether the feedback loop might be too long, helps to understand the real problem. And thinking up ways to shorten the loop, gives insights how to overcome these challenges. Also published on linkedin.
But why would we want to have a short feedback loop in the first place?
Imagine a business were you think of a new feature, and instantly, magically it’s there. Metrics and all. You can see immediately what the effect of this new feature is by these metrics. Therefore you can determine whether it’s an actual improvement. If not, imagine it away again, or tweak around a bit until it improves; or ditch it after all. With the speed of this genie-magic it’s very easy to figure out which paths lead to viable business.
This is not realistic of course. In the world of IT this is especially hard. Knowing what to build, building and designing it, market it, and come up how to measure and assemble these metrics… Especially with complex applications or services this is not done by a single person. Let alone imagining the feature and it’s there instantly. You require different people with multiple area’s of expertise. Ideal is to throw them into a single room, and have immediate contact with users/ customers. This might work in a startup. Larger organizations however have several challenges to get the feedback loop as short as possible.
Disclaimer: I think of myself as a developer, this article is written from a software development point of view.
Challenges shortening the feedback loop
I have listed several pitfalls and obstacles for introducing agile within larger companies below. Looking at the feedback loop caused by these challenges helps to understand how to overcome them (although this is easier said then done: most of them are of a cultural nature and requires people to actually change their mindset; hardly trivial).
Stories are too big
User stories are often used to describe to (development) teams what the feature should do. It’s a way to give developers a feeling how the product or service ought to be used. Stories that take longer than two days often require rework. With smaller stories the inaccuracy of what is actually required can never be big; with valid metrics the best path can be salvaged.
Slicing as if it’s waterfall
When coming up with a new, exciting feature, often this is too big to handle within a day or 2. Do not slice it up in technical chunks though. Because then you require all these loose chunks before you can start measuring things in production. Better to trim the feature down completely to the bone, so only the core of the feature remains. Use concepts from design sprint methodologies or as I like to call it the ‘mechanical Turk‘ to substitute for not-yet-developed pieces of automation. Implement that within a single story through all layers of a system (apps, services, data) so after the story you’ve got the feature completely working (although in a minified version). This way the feature is directly shippable and feedback can be gained immediately.
Introducing spotify model
When somewhere in the top of an organization it is decided to start working agile, often the spotify model is adopted. But creating chapters, guilds, tribes already creates distance between teams and team members by pigeonholing them. Don’t get me wrong, I think a spotify model could work for other organizations as well. However, spotify grew into this model and apparently it suited them well; that doesn’t mean it works automagically for other organizations as well.
The spotify model is to overcome challenges of sharing knowledge when scaling up. The famous matrix makes sure the lines between disciplines and roles are as short as possible for organizations of a certain size. But it can also work against you when this matrix is artificially put into place, where informal lines of communication would have worked better. Instead, every piece of information is crammed into this matrix model, and information is not flowing as freely as it would within an organization with a lesser urge to use a specific model.
Same goes for other introductions of tools, methods or frameworks supposedly helping to make an organization more agile. It could help, but only when used properly. Otherwise, it’s just obfuscating the feedback loop.
When budgets are too big , extended planning, look ahead, forecasting, roadmapping, overengineering without any restrictions are made way too easy. Nobody feels the pain when we all relax down a bit and do nothing too important for a while.
However, it gets more interesting as a deadline or budget rounds come closer. All of a sudden results seem to matter after all. But no intermediary results are available, so launching the new service or product is going to be a big gamble. It all might turn out ok, but it’s still a gamble. As long as there’s nothing going online and gets tested with actual users or customers, there’s no feedback on what to expect when you go live with the complete envisioned service. Big budgets make organizations lazy. Better to use smaller budgets (per epic or feature) or to simulate such an environment, where (within Scrum) product owners have to back up their wanted features with actual metrics.
A nice side-effect: when having too much budget, managers tend to hire extra hands to speed up the development process. This only works if you have enough projects that run more or less isolated from each other; but the minimal requirement is to make sure you can find a corner of code in a repository where nobody is working on at that time. Aligning what team is building what when, merging code bases, contradictive requirements shattered across teams, requirements on backwards compatibility without even using a public API, different opinions about code quality… just to name a few challenges you have to solve when it’s getting too crowded. All this is slowing down development, stretching the feedback loop. Better to find another project or just to not hire those extra developers.
User research before building
Before building something, you need to have an idea whether a customer actually want the product or service. Design sprints can be helpful in that, but don’t overdo it. Users don’t always provide you with good information on what to build, especially if you’re building something new. Better to use (more) objective metrics for that by putting something out in the market (or in production), or some nice ab-testing. This will give you tangible results, it shortens the feedback loop. Design sprints and other UX sessions just provides the customer or a small group of users better insight, but that doesn’t necessarily mean that the rest of your target audience will think or behave in the same lines.
When coming up with a solution before go-live, you never exactly know whether the user will go for this solution, wants something slightly better (which doesn’t necessarily fit in your beautiful design) or you need to pivot away. So coming up with a generic solution, which fits more cases then required at that moment, is only a good thing when it doesn’t cost you any extra. You might end up throwing it away. If your initial experimental (I would even say hacky) solution proves to be spot on, you can always go back to your grand generic design. But at least then you know it’s worth the extra mile.
Things like an automated delivery pipeline can help tremendously, just like automated testing, design patterns, (coding) styleguides, branching models, moving stuff to commons libraries, etc. But when this is done/set up too premature you lose your agility (with a remark that it later might make that return; but you don’t know for sure). Everything that needs to be implemented requires a learning curve, governance, additional maintenance, setup of your local environment. This is only worth the effort when working with multiple teams in the same code base (which I’m not a fan of: please see ‘Large budgets‘). Basically it’s the same reasoning as the introduction of the spotify model: whenever you get or start getting into growing pains, use technological tooling or methods to help scaling up, but not earlier.
New technology because of… new technology!
Developers are developers because they like technology, new fancy shiny thingy-tangy stuff (I’m completely biased). So naturally, introducing as quickly as possible new technologies, so other cool new stuff can be build, is completely on top of their agenda. But how important is new technology really? Yes, regular updates are important. Yes, ending up with systems nobody dares to touch because they’re so legacy, and use dated technology, is not a good situation to find oneself in.
But being an early adaptor also requires to reinvent the wheel several times. Or just waiting until some new release of the software arrives, which finally supports some feature urgently required. Or no other colleagues understand this new technology (yet). This causes delays in putting products or services quickly in production, stretching the feedback loop. I’m pretty sure the end user of most products or services don’t care that much about the technology being used under the hood.
If you really must so adopt (prematurely) new technologies, try to come up with metrics that matter to the successrate of the product or service, like increased performance (which might lead to a better NPS for instance) or quicker development in the long-term. Determine what to measure and when it’s successfull for a next step in adopting, establish a baseline, experiment with the technology in an isolated scope, and start measuring its succes. That will discard all the guesswork on whether to adopt a technology and makes it a no-brainer.
Completely rely on subject matter experts
Subjetc matter experts have gained expertise from the past. New products need to work in the future as well. And since users are getting used to new services, technologies and UX in an astonishing rate, the opinion of a subject matter expert might matter less than their experience might make you think. Subject matter experts tend to overengineer the system because they can rely heavily on assumptions gathered in the past.
Subject matter experts are great for setting up experiments, coming up with hypotheses on where to go to. Even better: measure these afterwards, and as quickly as possible, so more probing towards the right direction (preferably done with ab-testing) can be done.
Keep developers as far away from normal humans as possible
Whatever project management methodology your using, keep the persons who are steering the what-to-build close to the persons that are actually building it. UX, business developers, Product Owners in Scrum: make sure you’re available for the persons who are building the products or services you requested. When you’re not involved, the developers might deviate into the wrong direction when the requirements are ambiguous or some technicalities are not taken into account, even for a tiny bit (and believe me: this is almost always the case). The longer the silence takes, the bigger the deviation might be.
Also, when you’re not available, developers might even end up not bothering you at all. Then you’re losing mandate to steer the team. So try to shorten the feedback loop within your team as well.
Product is not good enough for feedback yet
What do you have to lose bringing software to production which isn’t fully operational yet, but it does says so (make sure it does)? And only the people you tell to (read: controlled environment), know about it? They might have an opinion about it and provide you with early feedback; they might find bugs; they might get enthusiased by the idea and are going to spread the word for you.
Ofcourse, when starting a (expensive) marketing campaign you want to offer a complete product or service. But some additional testers never hurt. So bring the product or service live asap.
Stories for the coming 6 months
Planning ahead is good: then you at least know where to head for. But don’t be afraid to pivot away again when all signs (read: metrics) indicate you’re on the wrong track. If you have six months worth of stories, described in the greatest detail possible, all that needs to be discarded. The time requested from developers to give some technological feedback on these far far away stories is by the way also thrown away. Also, when planning in detail over such a long period, it makes slicing the elephant (or better, drawing the Mona Lisa) much harder because you have to make loads of assumptions. Stories are likely to represent only a part of the solution that can be put into production. Stretching the feedback loop further.
So, plan ahead, but only vague and not in detail. This gives you time to think a bit more on what path to take in the near future, which can actually be accomplished fairly quickly.
Swap around team members a lot
Implementation of software takes up pretty much time. To get software out to production as fast as possible to gain feedback as soon as possible, it’s of the utmost importance to optimize development time. In order to optimize development, focus on development is important. This means no disturbances, team stability so everybody knows what to expect from each other and build up some domain expertise, and reduction of the amount of meetings are required. Swapping team members around often introduces loss of knowledge.
It can help to freshen up the team with new knowledge, new perspectives, new lines of communication, and changes in team dynamics. However, when done too often, you’re basically creating a new team. And a complete new team requires time to get up to speed, stretching the feedback loop.
Software delivered, mission accomplished
The goal of putting something out to production should be to gather feedback whether the feature has the intended result (validated learning). If features are just pushed out without checking what the succes of it is, you end up in the best case with a very large application full of occasionally used functionality, but high maintenance costs and very complex or a spaghetti-like design. Also, users can get confused by all this functionality.
Try to create a dashboard with both technical metrics (bugs, performance, traffic) as well as business metrics (KPI’s, NPS) on it, preferably realtime. In case of emergencies, the team (or anyone else who is committed to those results) can immediately act on it. Let the team talk to users to get some qualitative feedback and gain some deeper insights than metrics can tell. In short: development is not done when it’s in production.
That’s not my job
Cutting software development into various areas of expertise seem to make it easier to scale up: a single person can serve multiple teams. When something needs to be done in the frontend, get a frontend engineer. When something is wrong in the database, get a database engineer. But when this subject matter expert is not available, you leave the problem dangling. Or even waste some time on it yourself because you have no clue where to start, or communicating with other people who don’t have a clue either, further wasting time. Also, the expert is not always involved in the development of some piece of software, so handover requires knowledge transfer. All this is stretching development time, making the feedback loop longer.
Having people around involved in every step of the development process, even for a tiny bit (like a frontend engineer knowing from where to fetch specific data), can already help figuring out the problem. Especially when the most ideal candidate is not available.
Big organizations often have long processes and everything cut up like a pre-Lean era with small subtasks per employee (waterfall for example). Before software can go to production, it requires to pass some hoops first, knowledge needs to be transferred, loads of planning… Introducing agile basically means breaking these processes, the culture and unlearning all this.
In an organization you always have to deal with politics, even in very flat-structured ones. What I’m referring to is the opaque kind. Best to start on this is top-down, and some layers of management are likely to be collateral in the process. This is also why it’s hard to succeed.
From a developer role you normally would not have enough mandate; setting a good example with your team is a start: have an open atmosphere (no compromises on this) and from there try influencing other colleagues. But this kind of preaching usually ends at the department boundaries. And it is definitely a long-term project
Conclusion: shorten the feedback loop helps introducing agile
Large, incumbent organizations often have trouble introducing agile. Employees and managers require to change their mindset. However, striving for the shortest feedback loop possible within the restrictions of an organization already greatly improves adapting to an agile mindset. Discuss optimizations within your team. Lots of agile principles follow naturally after this.