Until recently, I was a true believer in only having functional user stories. In fact, I once had a very heated backlog refinement with a BA friend of mine over him adding a technical note to a user story about a form I had to build. My understanding of Agile was that you never stranglehold the implementation process with implemenation requirements, as it’s up to the developer to decide on the best approach to a problem. And, as anyone who has spent money on learning that stuff would do, I was damn sure sticking with what I learned when I got certified.
That all ended last week, when a team of business, project, and dev folks got together for a two-day workout in California to discuss the status of our project. The result was a sudden and eye-opening realization that if left to their own implementations, our devs will do things the way they know how, not necessarily how the architecture requires they be done.
Let’s go through the general process of requirements gathering, shall we?
- The BA will work with the product owner to discover the high level requirements
- The team will get together in refinement to ask questions and poke holes in feature requests
- The BA will get more details from the PO, and ultimately come back with well-defined requirements
- The UX then makes a workflow wireframe (and if time a visual mock-up) and run it by the dev team
- The dev team will give it the OK, and it will be presented to the PO, who will eventually give it the go-ahead as well
- The dev team builds to the requirements as they’re defined, hopefully making the product match the UX design as best they can
At what point in that process do we define whether or not the implementation matches the overall structure of the application? I’ll give you a hint, we don’t. Ideally it should be happening at implementation. But that makes the assumption that each dev team has insight into the overall structure and architecture, which – on a project that spans multiple dev teams – is kind of hard to maintain.
A misunderstanding of archtecture
If you’re running a project that has one dev team, it’s very easy for everyone to get a solid understanding of how the pieces come together. All concerned parties get to hear about what’s being done in standup, they review each other’s code in pull requests, and in an ideal setting they’re all sitting right next to each other. That’s a lot of connection points to make sure everyone is up to speed on how the architecture is evolving.
On a larger project that might have four or five dev teams, it’s quite a bit harder to get everyone on the same page. Architecture tends to evolve much more quickly on larger projects, and it’s much harder to disseminate information to all of the devs in time to effectively inform their implementation decisions. It’s even worse when some of that falls through the cracks and doesn’t get noticed until much later, when other pieces have been built around a feature that’s out of touch with the evolving architecture. Said other pieces are then also out of touch, leading to a mess of technical debt.
In a real-world example, this is something that I’m finding to be very common in an offline-first app, as built by four dev pods, soon to be five. As none of us have done this before, the information exchange between client and server alone is enough to keep those of us in the architects’ chairs on our toes. Ideally we would have gotten ahead of the project. But without dictating implementation to the devs, our aim was to explain the architecture, evolve it as we needed to, let them implement as they saw fit, and then try to match integration with implementation.
It seems ass-backwards, and by Sprint 4 we learned that it is. The only way to solve it was to build technical requirements into the user stories to prevent technical debt from building up.
After the first day of the workout mentioned above, the team realized that the only way to get through the gaps was to have the architects in on the requirements gathering to think through implementation options, and add technical notes about considerations that had to be accounted for. In some cases, that ended up with dictating implementation points to certain features.
At first I felt dirty about it. Who was I to tell devs how to do their job? And when I sat down with a pod to do just that, I expected to have my head bitten off.
The opposite happened. They were excitedly smiling the entire meeting, because for the first time they saw how their addition to the iteration was going to integrate into the larger applicaiton. One even said that up until that point, they felt like they were just tacking on functionality to a proverbial ball of string that was moving in a mess of diffferent directions. REST calls where there should be data caching, or an assumed happy-path where there should be an offline fallback. As a team, we all felt like we finally figured out a way for everyone to feel confident that we were moving forward without creating technical debt. And that’s a really big deal.
As those who draw boxes generally do, we on the architecture side of this project have a pretty good map of how things are supposed to fit into the pattern. And, as is natural, that’s all based on a high-level view of the application, not a code-level view. Conversely, the devs are all looking implementation from a line-by-line perspective, doing what they have to in order to hit their sprint goals. Ideally they’ll have architecture in mind, but (let’s be honest) they’re realistically only doing the best they can with what they know.
As architects, it’s up to us to us to sit down with them and make sure they understand how data needs to flow, and what other pieces of the app their code needs to interface with. Once they gain that perspective, they tend to see the value they’re adding more than if they were left alone to their own devices.
On a project level, it also adds to the team’s overall benefit, as instead of building several individual modules, the team as a whole is contributing to a larger applicaiton. So far, that’s lead to better team-building and more interest across pods in what the other folks are doing.
Once a plan has been established – a mesh between the high-level view and ideas that will drive the line-by-line view – a team can confidently add technical requirements to user stories so that as they’re worked on, technical considerations are documented and ultimately accounted for. So far, this hasn’t yet happened on the individual work items as much as the epics or portfolio items. But in the few examples that I’ve seen, it works, and works well.
Moving forward, technical planning meetings between the architects and the pods will ideally happen before refinement, so that we still only dictate the high-level aspects as they apply to high-level requirements, rather than how they’ll fit into the individual line-items. Hense why we generally apply tech requirements to epics. It also gives the architects time to build anything that needs to be in place for the dev team to get started, so that they’re not adding technical debt with mock interfaces or hard-coded JSON.
This will add another meeting to my team’s calendars, I’ll admit. But in true Agile fashion, I’ll assert that another hour spent in a conference room is more valuable than having my fellow devs go off in the wrong direction, creating technical debt without a clear understanding of how their feature is supposed to integrate. Ultimately, as far as I know, continuous attention to technical excellence and good design enhances agility.
I was surprised by how much the devs liked gaining insight into the bigger picture of the architecture, and working through questions that arose out of implementation ideas. We were able to reach common ground faster than I though we would, matching arrows and boxes on a whiteboard to the bullet points in their lower-level acceptance criteria. While I’m not sure this is entirely necessary for smaller teams, this might prove essential to making sure that a larger team is moving at a decent speed with the most efficient progress in mind.
Last, I’ll add that recently I’ve been reading a lot of angry blogs about how Scrum (and sometimes Agile as a whole) isn’t working, or has too many failing points. In this case, my team identified a failing point that might not necessarily be a failing point of anything Agile, but instead a failing point in the way we chose to communicate.