Documenting Acceptance Criteria: some ways to do that using text
Different formats of documenting detailed requirements using text such as Given When Then, Event-Response, and so on.
This write-up is inspired by a recent post I read in Piotr Gorak’s blog on different formats of user stories. I am not going to focus here a lot on the user story technique by itself but rather on the specification of behavior that engineers are implementing as a part of a story. My main goal with this post is to outline different ways to write acceptance criteria, problems I had with them in the past, and what I did to overcome those problems.
In this post, I am NOT going to speak about how to build a comprehensive list of acceptance criteria that tests positive, and negative cases, performance, etc. Instead, I will focus more on the mechanics of documenting what the story writer was able to think of when writing a story and discussing it with the team.
Writing Stories
To me, a story by itself is an expression of intent. It explains (should, at least) a problem that exists that software is supposed to address to make the user’s life easier. It may or may not mention the context to which it applies (a particular task the user is performing, a type of the user engaged in the task). A story by itself does NOT tell how to verify the problem is solved and if it is solved correctly. It does not state how to leverage the solution.
By a story I assume a technique to specify a few components:
- summary (traditional story format)
- conversation (whatever details the author captured)
- acceptance criteria (meat of the story)
Most of the stories I wrote over the last 6 years were user-facing i.e. included interactions between a user and a user interface.
When I experimented with formats of story writing, sometimes I would squeeze in the intended workflow of the solution the story implements into Conversation. On one hand that helped, on the other if there were other stories related to a particular workflow either I had to replicate at least some steps in other stories or rewrite the flow entirely. A co-worker of mine followed a different path where he would prepare user scenarios with basic, alternative, and edge case flows and then build stories around specific scenarios or their parts. In that case, user scenarios could just be linked to individual stories.
The detailed flow by itself can be used as a test or an acceptance criterion explaining how to verify the implemented solution. One problem is that one flow likely represents just one test scenario and an implemented solution to a story often includes more than one test. Hence the questions: 1) how to document ways to test the implemented solution? 2) How to define what must be achieved for a solution to address the problem? 3) How to remember what has NOT been done i.e. what scenarios are not supported?
Enter Acceptance Criteria
Acceptance Criteria, IMO, are the most important part of a story that serves as solution requirements explaining what and how the solution is supposed to achieve in order to address the intent of the story.
Of course, there are many ways to write Acceptance Criteria. Furthermore, the format in which Acceptance Criteria is written likely depends on the type of story being implemented.
For instance, if the story deals with a number of configurations, some sort of a decision matrix will probably be the best method of communicating requirements. In other cases, where the story deals with the implementation of a particular flow with a number of decision points (Yes/No) a more appropriate format could be some sort of a flow diagram. If the writer is describing requirements for a data migration/conversion story or API then a likely format for specifying acceptance criteria is a field mapping document or a data spec.
In this post, I am going to focus on a more conventional text-based (not graphical) representation of Acceptance Criteria. I am also assuming that readers are familiar with concepts such as INVEST and requirements characteristics (from BABOK) such as Unambiguity, Cohesiveness, Completeness, and so on. I am also assuming that readers are at least attempting to maintain some level of quality of requirements due to the fact they are even attempting to define acceptance criteria.
Getting back to documentation of Acceptance Criteria, I know that some are doing that in the “system shall format” where each acceptance criterion looks like a traditional format of requirements writing e.g.” System shall accept non-negative integer values between 0 and 1000 as an input to the quantity field”
Depending on how much time to prepare documentation I had, how familiar the team was with the space in which we were working, and the complexity of the product I could then document acceptance criteria in a number of ways:
Test-case like
When I joined my first project in 2012, I was introduced to the format of requirements writing adopted by some of the teams. Each Acceptance Criterion was written as a test case basically containing a flow in itself. I am not really writing acceptance criteria in this way anymore because they tend to get quite verbose (even though they typically make every case really clear), sometimes with a lot of duplicate content and of course taking quite some time.
Statement-like.
Example 1 (an example comes from a story focused on an API capability):
- EmergencyIncident is capable of returning IDs and codes of all agencies (related records) associated with a particular discipline based on involvements between records and cad calls
- RelatedRecords will be available for any dispatch i.e. fire, ems, law (ideal! at least for law)
Example 2 (for a story where we wanted to stop covering the text of comments with notifications about new/unseen comments)
- at no point should a comment text be covered with a notification about a new comment
- we should not be showing a notification box when there are no notifications outside of the visible area
- ctrl l should skip to the first unread comment in cad and mobile
The benefit of using this format is that it is really easy and quick to write. These statements serve more like instructions or reminders to have a conversation about specific steps. If that conversation does not happen, the risk is that a lot of work is a subject of interpretation by an engineer. Then when testing starts more conversations are needed to get an understanding of what has actually been done. Essentially, this may turn into an avalanche of unidentified details and of course rework.
Event-Response
This format is similar to given when then in my opinion with “event” being “when” and “response” the “then” part of the other format. It just skips the “given” portion or the preconditions that could be written for a few scenarios if the preconditions are the same. If written well, this format can be quite straightforward and easy to read.
I liked and still like this format a lot as it has a very simple formula IF-THEN that even people with little experience can follow. The risks with this approach are standard — it is missing scenarios the writer has not thought of.
Given When Then (GWT)
At some point, I started experimenting with using the “Given When Then” format of writing requirements in the Gherkin language. Honestly, I do not remember what made me do that. Maybe it was that I read Gojko Adzic’s book “Specification by example” or there was some other reason. I believe TDD and BDD were popular concepts and being a no-coder I just wanted to look cool in front of coding brothers and sisters. I do not think I intended to write acceptance criteria that could then be implemented using tools like FitNesse. Regardless, let the reasons to try GWT remain a mystery.
The approach seems to work well when there is a need to describe specific interactions and system output especially when a user’s action results in a number of outputs. For example, a checkout/confirmation of paying for a product or a myriad of other examples.
In all honesty, this is just a format, as good as many others if done right.
When I first started experimenting with the format I realized it was difficult for me to understand what a particular criterion tested without having to read through its entirety.
Here is an example of how I wrote it in the beginning.
Because there can be a lot of “given” clauses and “then” clauses, the review was challenging so I added some sort of a header/summary for each such criterion. As a result, the format started looking something like this.
The team I worked with at the time was giving me good feedback on this format. One person who liked this a lot was the QA engineer because this work helped her prepare clearly-documented test cases that could then be automated. Writing a story in this manner of course was taking more time to spell everything out. I also had to keep in mind the format expectations (no OR, one condition per clause, specific examples, etc).
Until recently I was not familiar with the use of “But” as a part of GWT syntax and now I want to explore that however that experiment has not happened yet.
Given When Then as a feature documentation
To me, the technique of GWT was good for documenting very detailed requirements however it was not really intended to describe user interaction flows. For those, I preferred to rely on either use case narrative or interaction scenarios written as step-by-step instructions (see Alistair Cockburn’s book “Writing Effective Use Cases”).
Recently however I discovered that a few teams from a different department are in fact using Given When Then to document actual workflows in the system. Way more high level. I am not going to show examples in this doc because some of those flows are actually reflecting functionality not yet released but such flows tend to be a lot longer with easily 5–10 conditions if not more per clause. According to that team, UX designers really appreciated that documentation format. My take, however, is that designers really benefit from seeing interaction flows regardless of how they are documented. Nevertheless, it was an interesting adoption of Given When Then in practice.
My techniques of acceptance criteria documentation today
Currently, I work with two development teams. With each, the format of writing requirements is a bit different. In one of the teams, I take a lot more active role in actual spec writing and hence I tend to write requirements in a more detailed manner to simplify/reduce necessary communication. For that team, I am actually still writing acceptance criteria in a “given when then” format. It is a little different now as I am not really using a table-like format, primarily since the majority of the requirements I now write are in Azure Devops (and it is not the friendliest tool) while in the past I wrote them in Confluence, and linked to Jira.
Another, slightly more complex example with more conditions (each condition is written in a separate line)
The engineers working with these features appreciate the approach [if written properly] because there is typically clear scope for every scenario covered and it is easier to determine how to break the implementation down.
With another team, I am a little less involved with writing acceptance criteria on the story level as the approach is somewhat different. Instead of feeding the feature spec to the entire team, I engage a single person. When I prepare feature definitions and scenarios to be covered, I then go over them with the team lead who then takes scenarios and breaks them down either himself or with the rest of the team into bite-sized stories. He then goes ahead to prepare acceptance criteria (typically in a statement-like format) and it is my job to review them before they go to the team for estimation.
Despite the seeming lack of standardization, the different approaches seem to be working for both teams.
If you liked what you read here, please clap, comment, or start following me on Medium
