Using Agile Principles in Workflow Automation
Organizations that implement workflow automation solutions typically attempt to automate the entire end-to-end workflow in one big deployment. This results in long lead times, large up-front investment, delayed payback, and a high risk of project failure. This case study will demonstrate that Agile principles can be used to deliver complex workflow automation iteratively. Taking this approach delivers business value immediately, dramatically reduces project risk, cuts overall deployment costs, and ensures user adoption.
Key Terms
- Minimum Valuable Product: The minimum functionality that must be delivered with a product to create business value.
- Minimum Viable Product: The minimum functionality that must be delivered to make a product usable or useful. A product need not deliver business value to be considered viable.
- Latest Responsible Decision Time: The latest time a decision can be made about functionality to be delivered in upcoming sprints without jeopardizing sprint planning and story grooming.
- Agile: An iterative approach to project management and software development that is characterized by the division of tasks into short phases of work and frequent reassessment and adaptation of plans.
- Waterfall: A project management and software development methodology that emphasizes linear progression from the beginning to the end of the project. Projects are broken down into linear sequential phases, where the conclusion of one phase kicks off the next phase. This linear progression continues till the final phase of the project is reached and the software is delivered.
- Sprint: A fixed unit of time (typically between 2 to 4 weeks) during which the Agile team will work to deliver specific tasks, milestones, or deliverables. Sprint cycles are iterative and continue until all project deliverables are complete.
- Business Process Management Solution: A suite of software tools that help organizations create, manage, and optimize workflows that deliver valuable products and/or services to external or internal customers.
An Agile Approach to Implementing Workflow Automation
A solar roofing company is losing money and customers due to its inability to manage a lengthy and complex sales and installation cycle. An initial attempt to build a homegrown IT solution to the problem has been ineffective and has made things worse in some situations. The company has decided to implement a Business Process Management (BPM) Solution to stop the bleeding and position for profitability and growth. Instead of attempting to deliver one ‘final’ workflow automation solution that addresses all the problems simultaneously, Agile principles will be used to introduce automation in small increments. Customization will be kept to a minimum, especially during the initial deployments, and measurements will be taken at key steps in the process. Ultimately, this approach will help the team identify pain points to prioritize for automation, and user adoption and feedback will play a key role in developing the backlog. This strategy will reduce risk by pushing scope decisions to the latest responsible time without impacting project progress. Let’s take a closer look…
Step 1 in Workflow Automation: Defining Your Business Problem
The company is in the business of selling and installing solar roofs for homeowners. The sales cycle from lead generation through final installation is extremely complex and spans six to nine months on average. During this period, as many as fifteen different departments interact with the customer and each other, performing diverse tasks including, sales, credit check, financing, permitting, panel selection, technical design, sourcing, staffing, and eventually installation. In addition, each customer’s unique needs must be considered. Improper execution at any stage could result in the project being canceled or cost overruns that hurt profitability.
The company’s previous attempt to bring some semblance of order to its operations revolved around a custom database. The database was designed to track each project and provide custom reporting to keep teams in sync and track progress over time. However, this database has become unwieldy. It’s grown to over 3000 columns and there are relationships within the database that often change reports, and occasionally, even the data of unrelated projects. Simply put, it is almost impossible to know if the raw data coming out of the current system is correct and if the projections being made are accurate. Confronted with this reality, most teams are tracking projects manually using spreadsheets, email, electronic documents, and handwritten notes. This has resulted in a wide range of problems including missed commitments, poor handoffs, cost overruns, waste, and disappointing customer experience. The company needs to urgently bring its workflow under control to stop losing money and customers. To address this clearly defined business problem, the company decides to implement a Business Process Management (BPM) Solution.
Step 2 in Workflow Automation: Determining Your Implementation Philosophy
Now that the solution has been identified in the implementation of BPM software, it is crucial that all of the stakeholders come to an agreement on the implementation process. This philosophy will be the lens through which every decision will be made throughout the project. Thus, the following key resolutions are made:
- The solution is to be rolled out iteratively using Agile principles rather than one big release with all functionalities.
- Tangible business value will be delivered in each iteration, starting with the Minimum Valuable Product.
- Data driven decision-making will be used to prioritize targets for automation.
- Users will be encouraged to use the workflow automation software from day one. User feedback will play an active role in the rollout.
- User feedback is to be incorporated at every step of the process to determine subsequent automation steps.
- Customization will be kept to a bare minimum.
Step 3 in Workflow Automation: Executing Your Project Objectives
All projects, regardless of the development methodology used, must be guided by clearly defined business objectives with measurable financial outcomes. The objectives and targeted outcomes are the basis for determining the functionality that will be delivered at the conclusion of the project. All features that are delivered must tie back directly and contribute to achieving one or more of the project objectives.
ArgonDigital methodology uses the Business Objectives Model to capture the targeted business outcomes and functionality that will enable an organization to realize the profitability or cost reduction goals. A subset of the model for this project is shown below.
A complete discussion of the Business Objectives Model is outside the scope of this narrative. You can learn more about this model in the articles that are shown in the Appendix.
Workflow automation performed using Agile principles delivers functionality on an iterative cadence. The diagram below depicts the iterative process the team will use to roll out the functionality incrementally.
A brief description of each of the steps follows, along with the actions to be performed by the team.
Selecting Your Steps for Automation
Automation candidates are selected three or four steps at a time. These steps are selected based on their suitability to provide tangible gains in productivity by reducing the time needed to perform them. With this approach, the steps that yield the most returns to the business will be delivered early so that these gains can be realized immediately, even as the rest of the rollout continues. Tasks that provide minimal gains from automation will be taken up towards the end of the program, or never.
Next, Process Flows are created to document the work as it moved through different stages from lead generation to final installation. Key portions of the flow are measured to determine the bottlenecks to be removed with automation. At first blush, this might look like weeks of work, especially for complex processes. However, using this Agile approach allows the team to only do what is absolutely necessary.
It is important to note here: the team does NOT document and measure every workflow process step to a detailed level before beginning any work. Instead, the team only creates process flows and takes measurements to the level of specificity needed to build the first, or next set, of functionality.
Shown below is the Level 0 workflow for the project. The Key Performance Indicators (KPI) of the three longest steps in the workflow are shown in the diagram. Once the flow is documented and shown to the subject matter experts, they are immediately able to identify the steps that most frequently take the longest to complete.
Notice, the measurements are kept simple and only the total time to complete all the steps that comprise the task are captured. This level of detail is sufficient to select the first sub flow for automation. More granular measurements will be taken later, but only if needed.
Sequencing Your Automation
The sequence determines the order in which the selected steps are automated. Here again, value drives prioritization. The selected steps are stack ranked based on their respective Minimum Valuable Product scores – the maximum value delivered for the effort expended to realize the gains.
The inputs needed to determine the Minimum Valuable Product are:
- The potential gains in productivity that can be realized with automation. This is a simple calculation expressed in terms of the reduction in cycle time to perform a set of tasks once automation is implemented.
- Effort needed to deliver the targeted automation. T-shirt level sizing (Small, Medium, or High) was applied to capture the effort needed to automate each of the selected steps. The level of specificity was sufficient to make Minimum Valuable Product comparisons.
To perform the sizing, the team needs a more granular view of each of these process steps. This problem is easily solved by documenting Level 1 process flows, created by exploding the Level 0 step into the key tasks to be performed to accomplish it.
Here is a more detailed look at the Level 1 process flow of one of the key bottlenecks, Step 14: Dispatch Equipment and Schedule Job. You can see that there are several complex flows in the diagram. These flows have several steps within them to complete the higher-level task. The team works with subject matter experts who give a brief description of the steps involved in each complex flow to the developers performing the estimate. The conversations are sufficient to give the technical team the information needed to perform t-shirt level sizing. If the developers need to understand any one of these flows at a more granular level to give an accurate estimate, the team will create the corresponding Level 2 flow. This is how Agile principles are applied to all aspects of the project including documentation – create only when needed, not weeks or months in advance.
The table below shows the calculations performed to determine the Minimum Valuable Product score for each option.
Based on the minimum valuable product score, Steps 3 and 8 are prioritized for automation, and Step 13 is delayed to a later date. This decision is made because equipment dispatch takes place near the end of the workflow and there are Level 0 tasks already identified that will deliver value earlier in the process.
The pragmatic approach is common sense in action and a perfect representation of agility. If the team followed the stack ranked results ‘as is,’ functionality would be delivered before the workflow reaches that stage in the newly automated process. Instead, the team has avoided jeopardizing the delivery of the larger effort by building functionality with smaller returns and an earlier realization. This is a classic example of postponing decisions to the latest responsible time, as mentioned above.
Planning and Delivering Your Targeted Automation
Once the sequence of automation is finalized, product backlogs are created, and stories and acceptance criteria are defined
At this stage, additional models are created as needed in order to understand the problem space at a granular level and derive requirements from them. Some common models created at this stage are:
- Business Data Diagrams
- Ecosystem Maps
- Lower-Level Process Flows
- Decision Trees and/or Decision Tables
Here is a Level 2 Process Flow of the Create Quote step.
Here is a Business Data Diagram (BDD) showing the data relationships associated with a Quote.
Once the stories are created, groomed, and handed off, development takes place. At the end of the sprint, working automation software is delivered to be tested. Lastly, upon acceptance, the functionality is ready to be deployed to production.
Using Your Automation to Perform Tasks
A significant project benefit is gained during this phase by making users capture process data in the tool. The software is in use even before the first automations are rolled out. The simple act of entering data enables users to familiarize themselves with the out-of-the-box interface. As more functionality is added in subsequent stages, users build on this base of knowledge and learn to perform more complex steps without much difficulty.
This simple stratagem proves critical to foster user adoption and minimize customization. By getting users to master the standard interface in small, focused chunks as new features are delivered, customization is reserved for areas where true business value can be extracted. By almost eliminating interface customization, the time to realize business benefits is accelerated, and overall project risk is reduced.
Incorporating Feedback into Your Future Iterations
As users perform tasks on the platform, they provide meaningful feedback on usability, functionality, and areas for improvement. This is nothing more than Agile in action. Each iteration of delivery and feedback leads to better workflow automation software being built for the next increment of features.
Conclusion
As the case study demonstrates, not only is it possible to introduce workflow automation to an organization in an Agile manner, but it must also become the standard approach taken to all such initiatives. Big bang automation carries significant risk that is extremely difficult to mitigate due to the long cycle times and steep learning curves at the time of release. Most companies throw money at the problem and spend large amounts on user training and rework, significantly increasing the overall costs of projects that are eventually successful. These are wasteful expenditures that reduce the return on investment for gains that can be achieved at a much lower cost using Agile principles.
Appendix
To learn more about Business Object Model and other topics related to this case study, please visit the articles listed below.
- Visual Models – Agile Business Objectives Model
- Using the Business Objectives Model and Weighted Shortest Job First to Prioritize Features
- Why are Business Objectives Model(s) SO HARD?
- Business Process Automation Metrics
- 2-Minute Models: A Walk Through the Business Objectives Model
- ECM Project Requirements Models
- How a BOM (Business Objectives Model) Helps Project Managers
- Business Analysts, It’s Your Job to Understand Business Value
