Moving from Project to Team based Funding in Agile

When we move to Agile we typically form our teams and then happily keep our waterfall project based funding structure in place.

We do this for a few reasons:

  1. We think it’s the only way to show the cost of the work that we are asking to be delivered.
  2. Projects are easily understood from an operational standpoint as they have a defined start and end date which is tightly aligned with the cost of the project.
  3. It’s the way we’ve always done it.

As part of our project based funding model we have an annual project funding request process, where we  spend weeks/months identifying the things that we think are important (note I’m not using the word valuable here) so that we can obtain funding.  Things move above or below the approval line and when the dust settles we have a book of work that is committed to, with freshly minted project plans and a cadre of project managers to manage the money we just gave to these projects.

An annual project funding request process also means that we ask for what we need and then everything else we may or may not need.  We ask for a Ferrari when perhaps all we need is a good dependable family sedan.

There is power in money, who has it and who controls it typically drives what gets funded and what doesn’t.  It’s not uncommon for Sr. Leadership to commit to work even though their team doesn’t have any experience in the product solely to get the money to keep their teams funded and employed.

If you see a lot of potential waste here then you are correct.  We see waste just in the amount of people and money needed to manage our project money.  If a money market fund had as much overhead associated with it, I and everyone else would leave because that overhead just cuts into profitability.

Next let’s more waste related to developing the stuff we didn’t need and may not actually use.  All of this extra stuff we develop has an expense associated with it and this is a long term expense.  We have it built into our architecture negatively impacting our systems scalability, performance and security.  We have to test it every time we build new stuff.  Bet you didn’t take that into account when you did your Cost/Benefit analysis for the project.

When we move to Agile we have an ability to really simplify our IT funding function.  It’s really quite simple – it’s the cost of your team.

In most organizations this cost typically hovers around ~$40k per sprint or over ~2 million annually.  So as a financial manager trying to manage things like cash flow, depreciation and the like, this makes financial reporting much simpler.  Each team becomes a fixed line item cost on my balance sheet and operating statements.  I don’t have to worry about cost overruns from project funding since the team is a fixed cost.  Product Owners ensure our teams work on the most valuable things in a consistent manner and at the end of the year we should/can be able to gauge the value of that work to some accuracy.

So here’s a challenge that we face, how do we actually assess value?  How do we value things like reducing technical debt to make applications technically better?  How do we value writing an input validation security framework that makes our application safer for the user and ourselves?  How do we value things that our customers aren’t asking for, but in the end benefit from?  That is the core element of software development, there is significant value in the things that the customer never sees yet we place little effort or priority in delivering these elements.  Instead we focus on the visual functionality and throw quality and architecture down the drain in favor of meeting project timelines.

If you get the fact that you have a fixed development cost it actually should foster better conversations regarding what is really valuable for the entire organization to be working on.  Not your pet projects, not the projects you agree to only to get funding to keep your people employed, that’s not how real value and efficiency happens and it’s time we stop thinking that it does.

Agile highlights very quickly that an organizations planning and funding functions are broken. It also typically becomes clear that we don’t have a real grasp on what our real value streams are and finding them often means removing political barriers that have built up over years.  Agile requires that we redefine what value is and organize our delivery across these streams over organizational silos.

The traditional PMO also goes under a dramatic shift, moving from managing projects and funding to ensuring that programs align to the organizations value streams, are well understood and organizational impediments are removed for the teams.

So if you are looking to move your organization to Agile you must understand that your funding and planning functions must change to align to a new mindset and paradigm.  Funding is easy in Agile, really it is (remember it’s the cost of your team), uncovering your value streams and maximizing the work that flows to your teams that delivers that value, now that’s harder (but not impossible).

Advertisements

Launching SoundAgile Consulting

I’ve been involved with Agile with many different organizations now for over 12 years.

In these years I’ve primarily been involved with being a contributing individual over a being an Agile coach.

The business of Agile has grown to a significant size and has now become a product that is sold to businesses who want to move their organization to Agile.  The very people who started Agile off as a movement have splintered off into several factions, each having their own opinion or approach in how to help organizations adopt Agile as a capability within their organization.  We now have Scrum, SAFe, DAD and LeSS to name a few in our acronym vocabulary.

Agile can indeed bring about valuable changes to an organizations ability to deliver software product more quickly.  These areas of Agile are fairly thought out, User Stories, Continuous Integration, Automation and Scrum.  You can move your development teams to a faster pace with some focus on specific team and development techniques that require some time to learn with some level of ease.

What Agile is struggling with is at the organizational level.  The Agile manifesto is specifically focused on building software better with a goal of delivering high value and quality software to our organizations.  A noble cause for sure and one that was sorely needed, given the changes in our software capabilities over the past 20 years.

Sr. Leadership however hasn’t changed much, still managing in a large up front analysis budgeting process which creates a painful friction between fast moving product delivery teams and slow moving hierarchical management structures .

For those organizations who are being sold Agile as a product that will deliver ‘x’ benefits know this about what is occurring.  These organizations are finding people who have done ‘some’ to ‘no’ real Agile, meaning they haven’t actually worked on an Agile team. Getting people who have the ‘right’ certifications doesn’t provide those people with the ability to coach teams in the reality of Agile, only the theory of Agile and their current frameworks.

They are also focused only on the product development area of your business, letting you believe that you will receive huge benefits from moving to Agile without the corresponding changes necessary throughout your entire organization to support a fast paced product delivery teams.

Agile is not a small change management effort, rather it is a multi-year impact to your organization, that if done well will lead you to great success.  If done poorly will provide you with significant pain without any corresponding benefits.

I’ve spent many years thinking about what I might offer from an Agile consulting perspective and I’ve come to the conclusion that any Agile ‘consulting’ work that I would want to engage in must include both Sr. Leadership down and the team level.

Another thing I have concluded is that successful organizations that want to become Agile, must do so with a much smaller footprint of coaching.  You don’t need full-time coaches for a long period of time.  In relying on full time coaches you are asking them to be your organizational Agile cop over owning the change within your organization.  The most successful Agile organizations I’ve worked in never had an Agile coach. Let me repeat that, never had an Agile a coach.  Instead they owned the move to Agile from the top down.  They provided the opportunity for teams to be empowered and fail and were not afraid to change organizational processes when they became impediments to improving Agilty.

SoundAgile will provide two levels of support and coaching for your organization.

  1. Team Level – Coaching and training will be accomplished through a combination of online training videos, 1:1 coaching and targeted onsite sessions for specific techniques such as Discovery/User Story Mapping, User Story Writing and Behavior Driven Development.
  2. Management Level – This will cover every management level in your organization, especially focusing in on your most impacted people, your technology managers.  Coaching and training will again be accomplished utilizing videos, 1:1 coaching and probably most importantly, targeted 1-2 day sessions that will continue for a multi-year time period. These sessions will provide for a longer term inspect and adapt change management process.

I’m really excited to be launching SoundAgile and am looking forward to working with people and organizations as they engage and encounter this thing called Agile.

SoundAgile will be live within the next two weeks.  I look forward to working with people who are motivated to move to Agile and make it work for them and their organizations.

Currency for Change – Transformation Needs and Roadblocks

change-1-1563676-640x480

Business leaders, those who run our organizations, continually look for strategies that deliver growth, synergy, profit and increased market share, to name a few.  They are judged and compensated based upon their ability to deliver results around financial or operational focal points.  Those leaders who manage a publicly traded company take on the added burden of providing predictable quarterly results year after year in order ensure a stable and growing stock price.

Many times new strategies require changes to your organization.  When attempting transformative organizational change, our focus is on changing the way that an organization operates at an organic level.  However our Leadership is rarely focused at this level, rather their focus is on the expected benefits of the desired change. They often fail to address the real organic element necessary for change, which are the very people who help manage and run their organization.  People will determine the final success or failure of any particular change management effort.

Change, the type that transforms an organization is often done so out of perceived need or stress event, such as new competitor or competitive products or disruptive technology.  Though the stress/threat may be very real to the survival of the organization.  Though the threat may be real the people working for the organization may not necessarily be motivated by changing how they work in order to respond to the perceived threat.  The reasons for this can be:

  • We may not be connected to the threat in a real way, we don’t see how the threat impacts our job.
  • We may not agree that the threat is real.
  • We may not agree with or believe that the requested changes are the right approach or strategy.
  • We simply may not care.

We are ultimately are creatures of habit, what worked in the past should work for us in the future, we come to expect outcomes based upon these past experiences.

Our natural world provides us with examples of how change is handled when stress is applied.  In nature change is a natural state and happens without negotiation, let me repeat that:

 Change happens without negotiation.

Trees don’t talk to hills to see if they are ok that more trees are grown, the change happens naturally based upon the need of the environment not the want of the trees.  Organic change happens in reaction to a stress event and then the system responds by initiating change that provides the appropriate reaction in order to bring the system back into a steady state.  In this example there is no currency for change between actors in the system as the system operates in a manner which brings the system back into a static or healthy state without applying change management techniques to encourage adoption of the change.

Large human systems are unlike our natural counterpart on multiple levels, primarily due to the people who are the actors of the system.  Natural systems form a comprehensive whole were all of the sub systems work in synergy on a grand scale.  Human systems however don’t share this synergistic behavior and as such operate independently of each other and the stress of one organization may not have any association or perceived dependency with another organization.

When human organizations inject change into their system in response to a perceived threat they trigger a broad set of activities at impact people in that system.  Change in both our natural world and our organizational world has the primary goal of keeping the system healthy and strong, but whereas the natural system accepts change without negotiation the human system involves potentially significant negotiation which has the negative effect of diluting the positive impacts the desired changes are expected to deliver.

Why is this?  Much of it surrounds not taking the time to communicate WHY the change is necessary and understanding the currency of our organization to accept the change.

What is Currency for ChangeIt is the perceived value that an individual will derive by participating in change.

Human systems require people to participate in change.  However in order to get them to fully engage in the change process we need to communicate WIIFME or What’s In It For ME?

Change requires that the people in your organization do some of the following:

  1. Learn new things (software, processes, tools, etc..)
  2. Take on new roles (Project Manager to Scrum Master)
  3. Report to new people
  4. Change the way that they manage
  5. Change the way that the project manage
  6. Change the way that you plan
  7. Change the way they are compensated

Currency then is what an organization is willing to ‘pay’ people in their currency in order get them to actively engage in change.  Currency is individual and ultimately relates to how an individual perceives their place, influence and power within the organization, this will drive what their specific currency will be.

Currency for change relates closely with the motivational needs of employees.  For example, though we may understand why we need to exercise and eat better for a longer life we may not be motivated sufficiently to do this consistently long term unless we identify the real currency we require to make the necessary changes.

There are many different needs based theories that can help define individual currency for change:

  1. Maslows’ hierarchy of needs:
    1. Physiological
    2. Safety
    3. Social
    4. Esteem
    5. Self-Actualization
  2. ERG Theory:
    1. Existence
    2. Relatedness
    3. Growth
  3. Acquired Needs Theory
    1. Need for Achievement
    2. Need for Affiliation
    3. Need for Power
  4. Three-Factor Theory for Employee Motivation
    1. Equity/Fairness
    2. Achievement
    3. Camaraderie

Parsing these different theories we come up with a few general themes:

  1. People need to feel safe
  2. People need to feel achievement
  3. People need to be acknowledged
  4. People need to feel connected to others
  5. People need to learn or challenged

When we begin to craft a change management plan for our organization we need to engage in conversation that explores the currency of the people who will be engaged in the change.

When beginning the process of change we must clearly identify the Why as part of understanding and leveraging an individuals’ currency for change.  If you can’t clearly identify the why people need to change you won’t be able to develop the What and the How in order to sufficiently engage people at their motivational level which we translate into currency.

Understanding what people require in order to be incented to change, translates into currency because change doesn’t come without investment and that relates to WIIFE, what am I going to receive if I change?  And unfortunately simply staying employed may not be enough, especially with highly skilled and sought after knowledge workers, you must engage them in a much different manner and their currency won’t be continued employment or more money (typically).

What does currency look like?

  1. Enagagement
    1. Allow more control and input with respect to the change to your entire organization, don’t make it a one way street with no negotiation. Unlike our natural world where change happens without negotiation, people in your organization are the source of successful change management.
    2. Benefit – It’s doubtful that your change management team has thought of everything that is required to make the change successful. Engaging your organization to participate in building the strategic direction of the change will create strong ownership of the change.
    3. Needs Met
      1. Need for Affiliation
      2. Social
      3. Esteem
      4. Camaraderie
  1. Failure
    1. We must understand that when we change our organization, the model by which we manage our organization also changes. Leaders and managers who have been successful in the organization are now faced with potentially dramatic changes with respect how they will manage and how they are perceived as successful.  Their very power base is threatened.  Encouraging a culture of failure as part of your change management efforts is essential for successful change.  Failure is not the goal, rather the mechanism that we use to encourage learning, because at its heart change is about learning and we all learn differently and we have different currency with respect to how we learn.
      1. Needs Met:
        1. Need to learn or be challenged.
        2. Safety
      2. Recognition
        1. There are people in your organization who have vast experience and domain knowledge which has been vital to the success of the organization. Though these people may be the most resistant to change, they can conversely be your biggest proponents for change if approached the correct way.  These individuals often want more recognition than material things such as more money.  They fall more along the needs matrix identified by Maslow, they are looking more for Social and Physiological needs to be met but also need to feel safe during the change.
          1. Needs Met:
            1. Safety
            2. Acknowledgement

As you think of taking on transformational change you need to start the conversation around the needs and currencies of the people who are going to make your change management successful.

Change is hard and when not engaged properly is destined to under deliver or worse fail completely.

Top Down or Bottom Up – Large Scale Agile Adoption

So I’ve worked in both large and small organizations where we have gone through an Agile adoption or the phrase of the day, Transformation.

Having seen both sides of the coin I started realizing that you have two paths to take when considering moving your organization to an Agile delivery methodology.

I use the phrase delivery, because I think at the end of the day that is what we are talking about.  Strip away the manifesto’s goals of conversation over documentation, accepting change, etc…What are are really talking about is moving an organization from a Project delivery methodology to one that is Product delivery oriented.

What is the difference?  From a Management perspective actually quite large.

  1. Project Delivery – These have very strong controls which move people to a new project.  Budgets are set up for the specific time period of the project and then up front requirements and design are completed in order to ensure that the project is fully ready to be engaged.  Project Managers manage all facets of the project via extensive project planning and plans.  Management receives up dates as to the project progress on a regular schedule, usually weekly. Resources are assigned either in full or in part, yet no one actually monitors nor can they really manage whether or not someone is working 25% on a particular project.  Projects tend to focus on reporting and there is high pressure to ensure that individual projects are green, which drives teams to deliver on the easiest and often less valuable parts of the project first and only at the end of the project is the hard work tackled, which reveals itself as budget overruns or timeline delays (or worse delivery of reduced scope).  Project reporting is elaborate and management receives project reports that are often sanitized.  Value is typically not delivered to the organization until the end of the project.
  2. Product Delivery – The work that is done for a Product is centered around a value stream it delivers and the work is ongoing.  Teams are funded as a whole and are kept together long-term in order to maximize their productivity.  Work is planned out in short increments called Release Plans that span anywhere from 6-12 weeks, with 2 week sprints.  Management receives regular updates (2 weeks) but can access information radiators at anytime, transparency is the key and goal with Agile Product Delivery.  Teams commit to work in 2 week sprints and their commitment is key to building trust with Management.  As time goes by management can trust that both a teams abilities and productivity can be counted on.  Teams are focused on delivering high quality code to production every two weeks which brings value to the organizations investment in them along with the increased value in terms of new sales, reduced costs, etc…Feedback loops via Product Demonstrations provides management the ability to assess where they are going with the product and deliver not what was asked for up front (Project Delivery) but rather what is actually needed (Product Delivery).

So what does the difference between Project and Product delivery approaches have to do with Agile adoption, well everything.

Most Agile adoptions begin at the bottom of the organization with the teams tasked with developing new software.  These efforts are borne, as the Agile manifesto was, out of frustration with how software was being developed in their respective organizations.  Often management is aware of the issues these teams face but are unable or unwilling to make any changes to how things are currently working and why would they?  You learn very early in your career that rocking the boat is not something that goes over well with organizations, my first boss told me I couldn’t by computers that weren’t from IBM because you don’t get fired if you buy IBM. The message is that if anything goes wrong you need to point to well-known names, processes, etc.. with which to blame or use as support.  To think that this doesn’t go on today is to place your head in the proverbial sand.

Though we can have great success with bottom up Agile adoptions with respect to improved productivity within small groups/teams, the overall Project oriented organization is typically still in place.  Management still wants to see project plans, have things ‘planned’ out for up to an entire year, they aren’t comfortable with the fuzzy feel of product roadmaps.  They want commitments, even false ones, so that when things fail they can point to the fact that they had all of their planning in place.

For Agile to really take hold, Sr. Leaders need to change the way that they manage both their people and the work.  It starts first I think in understanding that we have not learned how to speak to management very well yet from an Agile/Scrum perspective.

We need to understand what management is really concerned about and then center our product delivery efforts around that.  One of the problems that we face with some of our leaders is that they themselves don’t always know what to be focused on, they are looking at multiple balls in the air but at the end of the day as a Sr. Leader I think I have just a few things I should be focused on:

  1. Growth – This is often related to sales, market and revenue.
  2. Profits – Tightly aligned with the first item, our ability to make a consistent profit is what helps us continue to reinvest in our company.  Ever increasing revenue or sales without corresponding profits will eventually lead a company into bankruptcy, money isn’t free and it is not endlessly available, in spite of what we think we see with new technology organizations.
  3. Organizational Excellence – Because none of the above can happen unless you have a great organization.

Agile actually addresses all of the above, yet we spend more time talking about how we will improve our individual work efforts which causes us to  fail to tie this to the needs of management.  Management on the other hand often views the improvements that come from the bottom up approach as more of an anomaly rather than an organizational improvement worth adopting.

Trust is the missing component when it comes to conveying how Agile will make the entire organization better.

Agile isn’t easy and it requires skills that frankly many of our Sr. Leaders lack or don’t fully utilize and the politics of most organizations reward behavior that doesn’t align with Agile principles such as transparency, open honest conversation and openly questioning the status quo.  We have people in power who got there by way of the non-Agile status quo and changing that means they have to learn how the new game is played in order to stay on top, it’s much easier to keep things they way they are over learning how to navigate the new.

So how do we speak to our Sr. Leaders with respect to Agile?

  1. Better ROI – Talk to any Finance executive about what they look at when purchasing a piece of equipment that will deliver revenue and you will hear them talk about Net Present Value of the investment, Positive Cash Flow and Depreciation costs.
    1. We improve ROI in Agile due to our focus on only the most valuable items.  In non-Agile project work often are working on features/functionality that may be important to someone inside the organization yet will bring little or no value to the organization.
    2. When we are able to start talking about the value streams of our organization, be they revenue, cost reduction or improving our brand image we begin to be able to have a better ROI conversation with management.
    3. We also positively impact ROI via higher levels of productivity gained with dedicated teams.
  2. Flexibility – One of the most important elements that 21st century organizations require is the ability to be flexible enough to react to market forces or reactions.  Financing large projects far out into the future with the expectation of some level of return and no we don’t really have great track records of predicting future ROI out very far into the future.  With Agile we provide the framework to identify the most valuable work for the business in small planning windows.
    1. Sr. Management needs to understand that this flexibility comes with an obligation to have consistent short term review windows as the team progresses so that we deliver what is actually needed and not what we thought we wanted.  You may have thought you needed a Ferrari when in fact what you needed was a mini-van, we provide the framework to course correct via Sprint reviews every two weeks.  If you plan all of your project up front for the Ferrari, we’ll certainly try to make that happen, but in reality as the end of the project nears you will probably get the car but with a lawnmower engine and no brakes, it may look like a Ferrari but it won’t operate like one nor will it provide the value the organization really needed.
  3. Predictability – Another key element that we deliver with Agile is predictability and accountability.  Your teams will be much more accurate in planning and delivering in short-term 2 week sprints with a planning horizon of 6-8 weeks.  What management needs to look for is consistent delivery of the committed work that the team makes, commitment is everything.  What Wall Street analysts look for is a business that can provide a solid ROI, react responsibly to their competitors or even better be a market leaders and provide predictable results year in and year out.

So the question at hand is what is better a Top Down or Bottom Up adoption?  My money for long-term success is on the organization that can consume what Agile really means, not just to their development teams but the organization as a whole.  You can’t BE Agile if you don’t make the paradigm shift from command and control to one of collaborative and collective delivery.

Agile isn’t Easy

Over the years I have seen many teams and organizations who start on their journey towards Agile product delivery make the mistake of thinking that Agile is easy, promotes freely changing direction and worse will fix all of your issues and make everything better quickly and easily.  The truth couldn’t be further from the reality.

There is no such thing as a perfect software development delivery process.  Unlike production lines for things such as automobiles where you have the same pieces going to together each and every time and each piece has a specific functionality, tolerance and timing, software development is the exact opposite.

Software development is about meeting changing needs across the dynamic nature of business  For example – You wouldn’t see an automobile company add anew feature to a car on their assembly line over a 2-4 week time period.  They need time to design the entire process and ensure that the production line is capable of accepting the new change.

Traditional software development tends to align a bit more to the automobile example and in fact there are times where this type of rigid, pragmatic approach to product delivery is actually the correct process (Agile isn’t for everyone nor for every situation).

However for those that are going to move to Agile you need understand that the type of discipline that you might use in the automobile example actually needs to exist in your Agile processes, seriously you ask? Yes – You need to be able to deliver high quality, well-tested and fully functional pieces of software every two weeks. Now are you seeing how difficult Agile can be?

If you think Agile is easy, then you are already on the path to failure and unmet expectations.

It is very common for teams who are moving to Agile to take their interpretation of the Agile Manifesto to the unhealthy extremes, for example:

  • Working Software Over Comprehensive Documentation

Many teams I’ve worked with take this to mean NO documentation and that couldn’t be further from the truth.  We value working software over the need for documentation but I’ve never believed that you can have long-term success with your product without delivering some levels of documentation.  Without documentation your software knowledge becomes tribal and when your tribe leaves the team or your organization, well so does their knowledge.

There are way for teams to build documentation as part of their daily Sprint development work.  Using the combination of user stories and Behavior Driven Development (BDD) acceptance criteria as the foundational elements of your work you are creating your documentation as part of the work needed to deliver quickly.  The great value in BDD is that the acceptance criteria is written in English syntax and then translated into test automation.

This process of writing stories with BDD is supported by Gojko Adzics book, Specification by Example and allows us to deliver light weight documentation during our sprints.  I often see teams adding a story to a future sprint to handle their ‘documentation’ requirements of their previous work but I haven’t seen this work long-term.  Functional development, dealing with bugs, etc… will ultimately push these stories down into the backlog, never to be seen again.

The process described above is not easy, but it can be done and the teams that can get to this level of capability will succeed in driving value to your organization every two weeks.

One of the things that I consistently tell organizations moving to Agile is that it will highlight every current weakness of your product/software delivery methods.  Agile is a game changer, it requires a mind shift in how you look at your product, the work that supports it and how you see the visualize the value your product delivers.  If you are a leader who is going to be uncomfortable finding out truths about your organizations inefficient manner of product delivery then you need to think twice about moving to Agile.

Do you have TITL (To Important to Lose) people in your organization?  If you do, then you need to really look at how they influence any changes in your organization.  Do you need to get their approval, gain their support, etc….?  If so you will find more often than not that Agile will scare the hell out of them.  Successful Agile teams/organizations understand that self organizing teams take away the need for many of the day-to-day management decisions our middle management layer makes.  Agile speed comes when you remove the friction of management layers and provide teams with a clear vision of what you want them to deliver.

You must be prepared for the resistance that you will face from your product delivery teams, not everyone wants to go to Agile  We become comfortable with what we know and do in our daily work life and in that comfort comes stasis.  If you are not prepared to lose people, especially your TITL people, then you need to question if Agile is really for you.

I know it sounds heartless, but I’ve been working for over 30 years and one of the first things I learned right out of college was that technology was disruptive and if you weren’t on board with how it changes how you work you would be left behind. At one company I worked for many years ago, I saw Regional sales managers who had been with the company for over 30 years and when we rolled out a sales force tracking/marketing tool (which I led) there were several who refused to even turn on the computer, they were all let go within months.  As employees we have the obligation to continue to grow and learn and continue to make ourselves valuable to our organization and if we don’t, then you as a leader have an obligation to make tough decisions that ensure that the organization is continuing to grow and not stagnate with old processes.  It’s not personal it’s business.

As you move to Agile you also need to understand the investment that needs to be made in your technology stack.  Many organizations have decades old technology stacks which have been shoe horned into the future and though you get by you won’t be able to become Agile until you have a strong Continuous Integration framework, high levels of Unit, Integration and Functional test automation.  Getting to this will take time (and using the stories and BDD disciplines mentioned above will help you get there).  You simply can’t go fast if you don’t have the technology backbone that supports it.

Agile isn’t easy by any stretch of the imagination, it requires thoughtful introspection, focus on continuous improvement, disciplined delivery and a tenacity for value and quality that is never satisfied.

Baking In Quality with Agile

One of the things that I love about Agile and especially the related techniques of BDD and Test Automation is that if done correctly your teams are essentially baking Quality into your products AS they develop, not after.

Traditional SDLC (read waterfall) took a very linear approach to project delivery which in turn translated into a similar approach to how we developed and delivered our software products.

In our traditional delivery methods, quality was not ‘baked’ in but tested out and we never ever got to the point where we are able to test out all of the ‘defects’ that are found, instead we create a bug database where ‘bugs’ go to die.  Every organization typically has some form of a bug database with bugs that were ‘found’ sometimes years ago and were never fixed (of course begging the question were they ever bugs in the first place?)  In Agile we really don’t want to see a bug database because we should be instilling a zero defect policy for each sprint, meaning that we should never be introducing new tech debt into our product.

As I progressed on my Agile journey I came to realize that there are actually two types of ‘bugs’ that we encounter when we develop software:

  1. Bugs as Missed or Undefined Requirements
  2. Bugs as True Bugs

— Bugs as Missed or Undefined Requirements – I will argue (and in a book that I am starting to write about this topic) that a majority of the bugs that we find and document aren’t really bugs at all, but rather functionality that has been misinterpreted based upon the requirements definition that comes out of segregated development processes, ie Write Requirements, Develop Software, Test Software and Deploy Software.

Language is such an imprecise way of communicating that it almost virtually guarantees that if you have more than 1 person developing the software for your product you will get different interpretations of how to implement the requirement functionally.

Remember the old ‘The System Shall’ statement? that is commonly used in writing Business Requirements documents?  I always thought that this missed the scope of the requirement, what about what the System Shall Not Do?.  We focus so much on happy path for our functional development that we miss large segments of functionality based upon what an application shouldn’t be allowed to do.

Let’s take an example of an English word to convey what I’m meaning:

What do you think of when you see the word – BASS

Do you think of this:

Bass_Guitar

OR this?

Bass_Fish

These have the same spelling in the English language yet they have two entirely different meanings. Our life experiences become a prism for how we interpret what we read and how we react to it. (PS, I’m a musician so I think of the instrument before the fish)

Teams that rely on written requirements documents that are reviewed and worked on independently, meaning developers develop the code/functionality and then pass it on to testers will inherently have issues/bugs related to how something was interpreted and then developed.  In the above example the developers may have thought they were delivering a bass guitar, but the testers were expecting a bass fish (yeah extreme I know) but I believe this is the root of many of our issues when trying to deliver what the business expected in the first place.

— Bugs as True Bugs – I believe that true bugs are more technical than functional (or should be).  Bugs related to how integration happens are very common because although we can describe the behavior of our feature we can’t always anticipate issues related to how independent systems will work together.  Many ‘true’ bugs in Agile are caught in the moment and fixed before they ever make their way to production.  For the Finance people out there this is a tremendous cost saving that has been proven time and again.  ROI is greatly enhanced when you deal with tech debt up front rather over time.  And please don’t ever think really that it is more important to get ‘something’ to production over making sure that the product is operationally sound.

So what to do?

To address the  inherent limitations with our  communication, we need to abstract our thinking into more concrete descriptions of behavior over broad-based statements such as the System Shall.

User Stories and the corresponding BDD acceptance criteria are a great way to do this as a BDD example table clearly defines the behavior of our product functionality via outcome based upon inputs.  The ‘language’ of BDD is unique so that everyone can begin to have a shared understanding of what the story and behavior of the product(aka system) will do.  BDD abstracts our communication and removes individual interpretation.

In Agile we start by ensuring that the teams understand that they OWN the quality of their delivery. One of the things that I absolutely love about high performing Agile team is that there is no finger-pointing, if a Sprint fails to deliver what the team committed to then everyone shares the blame, not just an individual or functional group.

How we bake quality into our products is by understanding how to write User Stories that provide context without the ability to misinterpret the meaning of the requirements.

To do this we must first ensure that we have a well written user story, what does that look like?

A good user story needs to identify the What and then the Value statement.  Many teams that I have worked with start to write stories that only identify the actor and What but leave out the value statement, which is really the proof that what we are working on is of sufficient value to devote our resources to.  A good user story should never have any Creative or Technical design conveyed, I know that many people like to show their technical knowledge by writing stories that convey what they think the design or system will need to utilize, but all that does is start the team down a path before exploring all options.

Behavior over language interpretation is what you are striving for when writing contextually rich user stories.

BDD with its accompanied Example statements takes an otherwise basic user story and brings it to life.  Much like we do when we take basic ingredients for cookies and then bring them together in the right amounts to deliver awesomeness every time.

Software quality is much like baking, you need:

  • The right ingredients – Good individual team members, honest communication, commitment to quality
  • The right process – Write good user stories, add quality BDD acceptance criteria, code and test in parallel and then deliver, involve the entire team in writing BDD, involve the entire team in the estimation process.

By taking the time to build contextually rich user stories and define the story with BDD acceptance you move towards a shared understanding of the ‘behavior of your product’ over one that is driven by functional requirements.  Requirements tend to convey to little of behavior and focus more on the big win that is being conveyed to Sr. Management regarding what is being delivered.

Agile is a very disciplined delivery process and in order to bake the quality into your product you need to develop efficient processes that keep the User Story/BDD train running smoothly.  If you are entering a sprint and then writing your BDD then you are already behind, you need to develop a process by which teams are working on current sprint development AND building context for the next one.  It can be done and when it is you get what I call progressive regression with the automation that comes out of your BDD work.

Agile is very disciplined and to think that going Agile will make your current life easier, well guess again.  What Agile will do is highlight EVERY current weakness you have in your current product delivery process and then focus your attention on finding ways of improving on them.

For those of you looking for workshops regarding User Story/BDD techniques, please reach out to me at soundagile@gmail.com.

Software Engineering is Not Engineering (More Like Art)

Recently I had an interesting conversation regarding whether or not software developers were engineers or not. The most compelling argument that they could provide for why Software Engineering was truly an Engineering discipline centered around the fact that Universities and Colleges have their Computer Science group in the Engineering department.  That is not sufficient in my mind to by default call Software Engineers well, Engineers.

Why? Consider the following

Engineers who build skyscrapers, airplanes, bridges and products that require low tolerance for defects are often operating under the laws of science and as such require that they account for many scientific elements when building their products.

They perform a great amount of ‘testing’ both in theory and with product before something is released for use.  Their engineering practices are based upon scientific principles that go through extensive peer review and have been building in knowledge for hundreds if not thousands of years.

Now lets look at the definition of Engineering:

  1. The branch of science and technology concerned with the design, building, and use of engines, machines, and structures.
  2. The work done by, or the occupation of, an engineer.
  3. (Business / Professions) the profession of applying scientific principles to the design, construction, and maintenance of engines, cars, machines, etc. (mechanical engineering), buildings, bridges, roads, etc. (civil engineering), electrical machines and communication systems (electrical engineering), chemical plant and machinery (chemical engineering), or aircraft (aeronautical engineering)

Though of this I think can be applied to the science of computers, ie the hardware = Machines I don’t however see much if any of this being applied to Software Development.

A paper in 1996 (William Aspray, Reinhard Keil-Slawik, David L. Parnas) talks about some of the reasons that writing code isn’t associated with Engineering:

Computer science is often characterized as an engineering discipline with the systematic study and development of software as its principal subject matter. Software Engineering, however,although combining both key words, has not become a central discipline in most computer science departments.  In many respects, this discipline embodies the same ideosyncracies that can be observed within computer science as a whole such as:

• Highly innovative and rapidly changing field with no broadly recognised core of
material that every practitioner must know;
• Few results are supported by empirical or comparative studies;
• Work within the field older than 3–4 years is rarely acknowledged or referenced;
• Old problems are given new names and old solutions overlooked;
• Evolution of the discipline is tightly coupled to economic and societal demands;
• There is a need for interdisciplinary work comprising e.g. mathematics,
psychology, business or management science, … ;
• Continuing debate about whether there should be a discipline called software
engineering, and if so, whether this should be treated as another discipline among
the set of traditional engineering disciplines.”

As you read through this I think we can start to see a clear delineation between the disciplines required of ‘Engineering’ and the disciplines that Software Development requires (Note – This is not an article to criticize people who write software, but rather a way for us to better understand how to manage them and how to help them deliver even better software)

Go to almost any software development team and you will find a mix of people who have ended up going down the path of writing software, they can be people who were Computer Science majors all the way over to people with a Liberal Arts degrees.

The languages that allow us to write software today have matured over the years and as such have become easier to learn and work with.   This mix of skills and backgrounds is not bad, but it deviates from what an Engineering team building an airplane might look like.  Full transparency here, I have never worked in an organization that does true Engineering, but I know many people who do, and I suspect that you would not find people who are designing/engineering bridges or airplanes who don’t have a strong educational background in Engineering, but you will find these people in your software development teams.

So why is ‘Software Engineering’ more art than science?

I think that Software Development is more art than science because we must create software that we interact with on a more human basis.  Creativity abounds in web development as we continue to strive to bring intuitive and easy to use interfaces to the people who are our ‘users’.  The people who write software must be able to synthesize multiple layers of code, from Front End to Back end and everything in between. This talent is probably more aligned to writing a symphony in that you have to deal with an entire array of players, each with different instruments (API, Legacy, etc..) and when you get it right you get beautiful music, when you get it wrong you get noise.

When writing software we have the option to try multiple paths to an end, whereas when building a bridge our ability to deviate from proven engineering practices success is much more limiting and catastrophic when we do when we don’t.

One can also argue that the newer languages being created are designed to make writing software more accessible to more and more people who have not be through a computer science programs, we are striving to get our languages to ever more English like syntax.

Consider also that people who write software are not held to specific engineering practices.  Software, as it has evolved, has provided people who ‘develop’ software product with endless ways to implement it the underlying code.  The fact that there are not any set standards that every Software Engineer must use I think clearly shows that software is not a true engineering discipline, but rather a creative one.  Artists embellish, they innovate, the push boundaries into new ways of performing….I think this better describes many of our software developers more than engineering.

The beauty of current software development is that you have the ability to envision new ways of implementing code that delivers functionality and in doing so create new coding processes that others can use.  Software languages provide many different ways to write code to deliver a product functionality, that might be 3 lines of code or it could be 100, it depends on the background and experience of the individual writing the code.

I think when we start thinking about how to manage and support our Product Development teams in our organizations we would benefit from providing them the framework that encourages high quality through innovation, failing fast, and supports both the individual and groups ability to determine their success.  That is not to say that Sr. Leadership doesn’t play a part here, it certainly does, by providing clear vision of what is wanted (think of a symphony conductor, who must bring out greatness while providing their interpretation of a particular piece), an ability to accept failure as part of success and a fearless ability to have confidence that when you provide the other two components people will bring greatness as a by-product.

Artists operate in a very Agile manner, we listen and feed off of each other and in the process discover new ways to approach our art. Listen to jazz music and you see the elements of Agile and creativity evident as the small group (Scrum team) takes a song and through listening to each other can create new music, sounds or approaches to an old song.

As with Artists, Engineers don’t like to be told something can’t be done, that quest to make something happen that has never existed before is what an artist strives for, there is an emotional rush that comes with artistic creation and I think the same is true of software product creation.

Engineers who build skyscrapers, bridges and planes are no less artistic or innovative in their designs however they must take a much more rigorous approach to what they deliver, they simply don’t have the ability to go back and redo it.  If they get it wrong people die, though there is of course software that is also mission critical, but for the vast majority of us who develop software for a living, we are able to take a less rigorous approach when we create our products and we expect in Agile to move to a continuous delivery model so our work is never ‘done’.

Agile Gnostic

Introspection to Lean, Agile, DevOps and Project Management !

@tisquirrel

agile, software development, fun

Vasanthan

Agile Coach & Turnaround Specialist

#hypertextual

Organisations, Cultures, 21st Century

Liz Keogh, lunivore

you're different

Think Different

Shifting organisations to a better place

bowdenc

sql and primes, mostly

A servant leader's lessons

Mark Kilby's blog about agile, leadership, facilitation, mentoring, coaching and distributed teams.

Betsy Bites Back

Reflections on Business and Software

thetrainlineengineering.wordpress.com/

Software Development, DevOps, Testing, Performance, Agile

Dan North & Associates

faster organizations, faster software

The WordPress.com Blog

The latest news on WordPress.com and the WordPress community.