We Don’t Need No Frickin Architects

“We Don’t Need No Frickin Architects”

I’ve lost count the number of times that I have heard this statement (or some kind of derivative) during my career when mixing with Agilistas. The reasoning behind these statements is that architects are either defunct in the agile world, or that they need to roll their sleeves up and relearn to code alongside others in a delivery team. It makes no sense to over-egg the pudding.

I understand the reasoning, but as a Systems Thinker it is my belief that there is a need for architects.

Lets listen to what Dr Russell Ackoff, a Systems Thinking pioneer and organisational theorist, has to say on the subject.

A system has 3 properties:

  1. Each part of the system can affect the behaviour or the properties of the whole. They don’t necessarily do it all the time but they can.
  2. The way each part affects the whole, depends on at least what one other part is doing. The parts aren’t independent, they interact, they are connected. No part of the system has an independent effect.
  3. If you group the parts of a system into sub-groups i.e. sub-systems, they have the same properties the parts do. Every subsystem, or collections of parts, can affect the whole, and none has an interdependent effect.

The performance of a system is never a sum of the performance of its parts taken separately, its the product of its interactions. The performance of a system is based upon how the parts interact and fit, never on how they act separately.

There are many places where improving the performance of the part, will make the performance of the whole worse.

Architects knows this intimately:

  • An Architect draws the house first; the whole, then he adds the rooms; the parts.
  • He only improves a room, in a way that improves the house (the whole).
  • Sometimes a part needs to be made worse to make the whole better. If he can make the room worse, but make the house better, an Architect will do it.
  • The objective is to build the best house, not the best rooms.

If we have a system of improvement that’s directed at improving the parts taken separately, you can be absolutely sure the performance of the whole will not be improved.

The doctrine of the Western world is to think “divide and conquer”, if every separate part of the organisation is managed well, then the whole system will improve. This is absolutely false!

This is completely counter intuitive, we are committed to managing the actions of the parts, not their interactions. Most improvement programmes look to improve the parts separately not the whole. The whole is what should be the focus for continuous improvement.

In the above quote Ackoff is referring to the Organisation as a System, he is not referring to technology based systems. However the same thinking can be applied to technology based systems too.

We are encouraged in Agile to break work down into the smallest possible value adding chunks (often called either Epics, or Stories, or Features, or MMFs, or MVPs etc). Breaking work down makes sense. With the least amount of effort we can not only validate learning (by answering questions and testing hypothesis early, for example whether customers will use or buy our output), but also to minimise risk and possibly realise value earlier.

There are some thought leaders that even say that projects will soon become defunct, to be replaced instead by a constant flow of smaller chunks of work flowing through the delivery pipeline, with projects just becoming containers for these chunks of value adding increments.

As an Agile practitioner this is all good by my book, and makes sense to me.

But the Systems Thinker in me thinks differently. Lets listen again to what Ackoff is saying

“There are many places where improving the performance of the part, will make the performance of the whole worse

and that

“Architects knows this intimately”

So with a constant flow of small chunks of work, who is thinking about the whole? Who is concerned about the interactions between the parts? Who has the end game in mind?

I have spoken before about the difference between efficiency and effectiveness. Where efficiency is doing things right, but effectiveness is doing the right thing. My observation around Agile, Kanban and the like, is its all about efficiency, whereas we want to be more effective. I like the way a colleague of mine once put it

“We run the risk of rowing really fast as a high performing agile team, but in completely the wrong direction”.

As Peter Drucker says

“There is nothing so useless as doing efficiently, that which should not be done at all”.

So, we are at risk of building a whole load of well engineered parts that do nothing as a whole, or could in fact make things worse. Here is Ackoff again

When you try and put the best parts together you think you will get the best whole. This is wrong!

As an example, if you took all of the different cars on the market today, and asked a group of engineers which had the best engine, which had the best transmission, which had the best alternator etc and they took these best parts and tried to put them together, you would not get an automobile; the parts wouldn’t fit, let alone give you the best car in the world. It’s the working together that is the main contribution to systemic thinking.

Our entire culture is built on analytical thinking. The product of analysis is how things work, never why they work the way they do.
Analysis takes you inside the system, and how it works. It provides knowledge but not understanding.

By using a process of analysis you can reveal structure and say how something works, but not why something works in the way that it does.

Analysis has three steps:

  1. You take the thing you want to understand apart.
  2. You explain the behaviour of each part taken separately.
  3. Lastly, you reaggregate your explanation of the parts into an understanding of the whole.

Analysis became the dominant mode of thought in the Western world for hundreds of years. Today we still use analysis and thinking as synonymous terms. This thinking permeates all of our institutions, corporations, etc.

To reiterate, the process of analysis it to separate the whole into parts and study each part individually. Over to Ackoff

You will see this happening at an early age, if you give a child something they have never seen before, for example a puzzle, and they are confronted with a need to understand it, the first thing they will do is take it apart, the second thing they will do is to try to understand each part taken separately, then they will reassemble it to try and get an understanding of the whole.

This explains why inherently we feel that breaking work into parts to put them together again seems like a good thing.

To understand “why” questions, you need to use a process of synthesis. Synthesis is the opposite process to analysis and consists of three steps:

  1. You take the thing you want to understand as a part of a larger whole.
  2. You explain the behaviour of the containing whole.
  3. Lastly you disaggregate the understanding of the containing whole, by identifying the role or function of the parts.

Synthesis is an another way of thinking, it’s about making sense of the whole, which provides explanations of the behaviour of a system. You take the thing you want to understand and study it as a part of the larger whole.

If you want to understand how something works you use analysis, if you want to understand why it works you use synthesis. To be effective we should be using a combination of both.

The essential property of a system (whole) is that it cannot be divided into independent parts. If you apply analysis to a system, you take it apart and it loses all its essential properties, and so do its parts. This is easily demonstrated by Ackoff:

An example is a car, its a product of its parts. When a car is taken apart the system looses its essential property; moving from A to B. When we take a car apart we have cut its interactions. Not only that, we remove the essential properties of the parts. The motor cannot move from A to B on its own.

Here is another example, you can write, your hand can’t write, its easy to demonstrate by cutting your hand off and asking it to write, it wont do it!

As a sub note, I like the notion of Consumer Driven Contracts as an example of concern regarding interaction between parts:

A consumer-driven contract is a representation of a service provider’s obligations to all its current consumers. Consumer-driven contracts are created when each consumer communicates to the provider its specific expectations. The obligations created on the provider’s side establish a consumer-driven contract. As part of adopting a consumer-driven contract, a provider is free to evolve and innovate its service offering just so long as it continues to satisfy its existing obligations.
Source: http://www.infoq.com/news/2008/07/consumer-driven-contracts

However I do wonder if this is merely concerned on how two parts interact with each other, rather than understanding the whole.

What we do need is architects, but….

What I am not advocating is the style of architect that contemplates their navel for months on end, or sits atop an ivory tower dictating policy, standards, monolithic specifications, and the “one right way” to all. I’m also not suggesting that we introduce inspection regimes, or to build gold plated enterprise solutions that we realise are either out of date by the time they are released, or that more nimble competition have beaten us to it.

What I am suggesting however, is the need for someone who understands the whole, with the ability to use synthesis in addition to analysis. Someone whom is viewed as an additional stakeholder by all of those involved in the creation of the parts.

The modern architect gets visibility of, and has interactions with, the parts, but also has an appreciation of the whole. Agile brings transparency to the development process that can enable an architect to see what’s really going on within the parts, that allows them to react based on their objective of building the best whole, one that meets the customer’s needs.

There is a balance. The objective is to build the best house, not the best rooms.

About these ads

19 thoughts on “We Don’t Need No Frickin Architects

  1. Interesting, but article suggests that teams without architects are not capable of doing right things, drawing the house, looking at the whole, and so on. Also, what makes one person as an architect more capable than a whole team with always much more (undiscovered) knowledge.
    In my experience, architecture becomes much better when agile teams are “doing” the architecture. There is still an architect, but his main role is facilitation of the architecture process.

    • If you are working on a truly isolated project, that is in itself a whole and not a part, then I agree the team can suffice.

      If however you are working on a project that is a part of a whole, an example would be a programme of work, then team capability isn’t enough. Are they really viewing things as a whole or just concentrating on their part (which is natural). If the former then they are showing very high signs of maturity, as are their stakeholders.

      There is nothing wrong in my view of individual team’s facilitating local level architecture to then be implemented by craftsmen working on their craft, but you only have to see the slew of articles on the web from agile teams who are frustrated at rising levels of technical debt, and frustration that stakeholders aren’t buying into the quality argument, to understand that the team has limited influence, and that they are hindered by non systemic (whole) thinking.

      A role (call if architect for now) that has an understanding of the whole, understands quality, can vocalise the 7 Deadly Sins of Quality Management etc will provide great benefit to the individual teams, and more importantly give rise to a whole that is more likely to meet customer need.

      • The project I’m talking is part of a whole and consists of 3 teams. I’m actually fulfilling the role of the lead architect. Although the project is quite large, half of the time, I’m just participating in the teams and programming. The other half I’m making sure that everyone is concerned about quality, periodically hears about the big picture or aspects such as security, business process, integration with other systems and so on. I’m mostly not the one who is telling this, but “domain experts” (who are also part of the teams). All decisions are made together. Therefore, everyone understands pretty much everything what needs to be known to make a decision.

        So, I don’t deny the need for an architect as a leader or facilitator, but he/she is not the one making the best architecture. Quality is something which is wired in the mindset of every team member and understanding customer needs is even more important. With some facilitation everyone is capable of looking at the whole.

        I think that technical debt is lack of concern for quality and architecture and not the lack of an architect. It should not be solved by introducing traditional architect, but by changing the mindset of team members.

  2. Neglected design is expensive design.

    I didn’t count the number of times that I have said this statement during my career as architect and developer. I think we need architects, people inside a team who care about the whole design, avoid sub-optimisations and inconsistency. But what we don’t need are ivory tower theorists and shiny titles…. they are waste!

    Many greetings from an agilist in Sweden! :)

  3. Hi David,

    I have had the interesting experience of holding a senior architecture role just at the moment when this kind of thinking took hold in my organisation (and it came as much from the top as much as it did from within teams) but to be honest I didn’t much mind, and it wasn’t bad for my career either!

    Yes, architecture very important and you’re right to approach it from a Systems Thinking angle, but still I find it hard to defend the stance that it can only be done by someone in a specialist role. Ultimately, managers (and for business systems, process owners too) need a proper understanding of the systems within and around their remit, and although it may be efficient to delegate the activity, some meaningful responsibility must remain. One could even draw a parallel with Deming’s “Appreciation of a system” (part of the of System of Profound Knowledge).

    I consider it a priviledge to have worked with great managers who are also strong architects, even if this is mostly (and quite efficiently) expressed through an uncanny knack of asking the awkward question at just the right time. Perhaps your article doesn’t _quite_ rule this out, but I’m tempted to think that it might actually be the ideal setup for many organisations.

    Mike

  4. I am not so sure I understood your point.

    I am an advocate of Fred Brooks’ style small teams, and a true believer that people in a project need to spend more time understanding the big picture rather than just building a bag of features and calling it “system”.

    That said, the points you make are way too similar to what I’ve seemed being used by hand-waving enterprise/systems architects trying to justify their role. I think we all agree that developing software is very different from building a house and that a developer is very different from a bricklayer so I am not sure how useful that quote is in this industry.

    Everyone in a project should understand the whole, because decisions will be made at all levels. If a leader and/or visionaire happens to be the person who understands better or even the one who defines what the whole is it is probably ok. That’s very different, though, to the architect role –especially if you compare this role to what an architect does in a civil engineering process.

    • I’m certainly not advocating the need for hand-waving or justifying any particular role. What I am advocating is understanding of the whole, rather than individual parts, the latter of which is ingrained in analysis. This can be seen outside of civil engineering, it’s not uncommon to hear statements in corporate corridors such as “It’s their fault” or “let’s get our own house in order”.

  5. I really liked this post, David. It certainly challenges practitioners of Agile methods to find a way to fit the design and architectural tasks (and the driving, intuitive, creative vision) into processes that are good at creating well-engineered components efficiently, but perhaps not so good at getting a wonderful, innovative whole out of those parts. I’m pretty sure this aspect of software design and systems thinking remains largely un-addressed by the Agile community in particular and software development professionals in general. It is here that the practices of artists can teach much, I think. Watch for a future blog post on tropicaltheartist.wordpress.com for more. You’ve inspired me to write an article where I attempt to tie the two disciplines together and assert that the artistic method can apply. I hope to get that done before the weekend.

  6. Nice piece! Always great to see Ackoff’s (genius) ideas promoted to a wider audience.

    But I fundamentally disagree on one point: I believe that what we need is *architecture* (the discipline, the skill-set) not *architects* (the job-title, the specialist). By perpetuating the idea of (narrow) specialists, we fall into another “analytic thinking” rabbit hole.

    But your central point, that we benefit immensely when we consider the whole in harmony with and as essential context for considering the parts – that I agree with entirely.

    As one of those folks who “say that projects will soon become defunct, to be replaced instead by a constant flow of smaller chunks of work flowing through the delivery pipeline” (a.k.a. FlowChain), my mental model is one of constant evolution of a whole (e.g. a “whole product” c.f. http://en.wikipedia.org/wiki/Whole_product ) , where each additional small chunk augments and enhances the relationships within the whole. FlowChain also draws on Christopher Alexander and his ideas about the evolution of the “built environment”, as well as Stewart Brand’s most excellent book “How Buildings Learn (What happens after they’re built)”.

    Of course, for FlowChain to deliver real benefits, folks have to have an working appreciation of the need to take the holistic (synergistic) view to which you and Ackoff refer. That’s one reason I don’t talk/write/tweet about FlowChain much – the world of work is a long way from widely appreciating the Synergistic mindset.

    I have to thank you for reminding me that the FlowChain idea carries the risk of catastrophic misinterpretation when viewed through the lens of the “Analytic” mindset (the most prevailing and imo pernicious mindset, and common to most organisations today). If FlowChain were used just to churn out small chunks in isolation, then indeed it would be pointless, dysfunctional and deleterious.

    Cheers,
    - Bob

  7. Great post.

    The architect in our team put it nicely last week: “A [systems] architect is just an engineer who works at a higher level of abstraction”. What you call them is not important but you undoubtedly need people who consider the whole system in terms of its interactions.

  8. Pingback: Where the Magic Happens | Creative Ideas for Starving Artists

  9. We need a balance so we don’t have systems held together with sticky tape or systems that are designed by astronaught architects that look great on paper but cannot be implemented.  The best architects I have worked with are those that can also help implement them.

  10. This is such a dumb debate. The only difference between an architect and a developer is scale, that is the breadth and level of the components they deal with.

    The scale determines the factors they have to take into account. Developers don’t have to worry about aligning projects with company strategy and architects don’t have to worry about test cases. You can’t understand a forest by concentrating on the veins in the leaves and you can’t understand a leaf by studying a forest.

    Agile is fine as the only methodology only in an enterprise where all systems are mature and the systems are under the direct control of the enterprise and there is no major change at the enterprise level going on, otherwise it is a recipe for a huge expensive mess. This is because it is only in such a stable controlled environment that a developer would have a broad enough veiw to see everything necessary to make the right decisions, and they wouldn’t have enough time left over to do any coding.

    The enterprises I have worked for always have lots of large enterprise level changes going on, have outsourced large chunks of their systems, and have in the thousands of systems. Try to get a developer to make sense of that in an epic.

    Regards

    • @Mark Kortink
      I agree with what you say, it is a matter of scale and breadth of concern. I was thinking the same thing as I read this interesting post!

      As an Enterprise Architect, I am managing the big picture view of the future enterprise architecture and defining a roadmap of change initiatives which can easily include a huge number (1000′s) of individual building blocks, let alone IT applications. Building blocks can be any element that is Business or IT, ie. a Business Function, a Business process, A Product, an Organisation Unit, as well as an Application Service, Application Component or part of the infrastructure.

      My focus is on a broad and shallow view of the future architecture across the whole enterprise i.e. both Business and IT aspects, looking how all the future building blocks will need to work together and satisfy the business strategies and less in terms of the detail inside each building block.

      The focus of a solution architect will not be on the future but on the here and now within a programme or project to realise a single building block as a solution. They would do this with a smaller scope of a single solution but in greater greater detail. The solution can still contain both business and IT changes.

      An IT solution is likely to be composed of many individual application, service and data components and the developers will work in an agile fashion to develop the even smaller scope of these single IT components and understand them in great depth in excruciating detail as they write the code and test cases.

      To think that agile developers can possibly also have the bandwidth to be concerned with the Solution Architecture or the Enterprise Architecture would indeed be heroic.

      Each job is extremely important in its own right but no-one can possibly do everything.

  11. The problem is there is no official “diploma” for software architecture. I’m not sure there *should be* one, but the fact is there is no school, no cursus, no classes, nothing that is somehow recognized by everyone from the computer software field or from outside. I know many many so-called “architects” that architect nothing but a bunch C# classes in a Visual Studio project, and think they built the new Taj Mahal because they used some trendy patterns (cqrs, event sourcing, ioc, di, …) that every “serious architect” should use… but, wait, who am I to say that? They don’t have to prove they are architects, I don’t have to prove it either. So if we don’t know what/who is an architect, how could make sure we need them?

  12. Thanks for the great discussion here.

    I agree with the architecture as a skill/badge as opposed to architect as a role/title reference. In fact I think this applies to other skills on projects as well e.g. analysis, testing, “managing”.

    Also, if you haven’t already heard of it, you might find Simon Brown’s blog and book on the topic interesting -
    http://www.codingthearchitecture.com/blogentries/1.html

  13. Pingback: Weekly – CW21, CW22 and CW23 | Zsolt Fabók's Page

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s