Write Independent Scenarios
- Every Scenario has to be completely independent of another scenario.
- If a Scenario is dependent on another, then it is incomplete.
- Dependent Scenarios cannot be executed in parallel.
Run Tests in Parallel
- Transform similar Scenarios into Scenario Outline.
- Break down Scenario Outline into different tags.
- Set up a Cukes Runner for every tag.
- Run every cukes runner simultaneously.
Do not wait unnecessarily
- No unnecessary waiting.
- Do not use sleep.
- Keep polling for every event that requires a wait.
Transforming Scenarios to Scenario Outline
Shown below are three Scenarios which upload a file, wait for the file to be processed, search and retrieve it.
Shown below are three Scenarios which upload a file, wait for the file to be processed, search and retrieve it.
### Scenario 1 ###
@upload_retrieve_file_1
Scenario: Upload and retrieve a file
Given I upload a file "file-1"
And I wait for the file to be processed
When I search for the file
Then I get the file
### Scenario 2 ###
@upload_retrieve_file_2
Scenario: Upload and retrieve a file
Given I upload a file "file-2"
And I wait for the file to be processed
When I search for the file
Then I get the file
### Scenario 3 ###
@upload_retrieve_file_3
Scenario: Upload and retrieve a file
Given I upload a file "file-3"
And I wait for the file to be processed
When I search for the file
Then I get the file
When you read through these Scenarios, you would see that every scenario is exactly the same, except that each deals with a different file name.
I see a pattern of similarity among these Scenarios, don't you?
"Whenever you see a pattern among Scenarios, they should be transformed into a Scenario Outline."What's a Scenario Outline?
A scenario outline is a Scenario with a table of test cases and the scenario is executed for every row of the table.
Advantages of transforming Scenarios to Scenario Outline:
- Reduce in number of lines
- Increases readability
- Re-use of step definitions
If we were to use Scenario, we would have nine separate Scenarios for every file, which is a lot of repetition and more number of lines.
### Transforming Scenarios to Scenario Outline ###
@upload_retrieve_files
Scenario Outline: Upload and retrieve files
Given I upload a file "<file name>"
And I wait for the file to be processed
When I search for the file
Then I get the file
Scenarios: List of files to upload and retrieve
| file name |
| file-1 |
| file-2 |
| file-3 |
| file-4 |
| file-5 |
| file-6 |
| file-7 |
| file-8 |
| file-9 |
Did I just compress nine scenarios into One Scenario Outline? This is interesting!
Let us assume that the Scenario Outline would upload the file, wait for a minute for the file to be processed, search for the file and retrieve it. Also assume that uploading, searching and retrieving are instantaneous that they run in no time.
We have nine files in the table, it would take one minute for every file to be processed and in total it would take nine minutes for the Scenario Outline to complete execution.
Hmmm.. Nine minutes for nine files, thousand minutes for thousand files? This doesn't seem quite right!
Breaking down of Scenario Outline into smaller Scenario Outlines
Any Scenario Outline could be broken into smaller Scenario Outlines, just by splitting the table.
Tag every smaller table with a unique tag name. With the use of tags you could execute a desired smaller portion of a huge scenario outline.
But wait, what's a tag?
- Anything that starts with a '@' is a tag.
## Break down of Scenario Outline into tags ##
@upload_retrieve_files
Scenario Outline: Upload and retrieve files
Given I upload a file "<file name>"
And I wait for the file to be processed
When I search for the file
When I search for the file
Then I get the file
@upload_retrieve_files_1_to_3
Scenarios: Files 1 to 3 to upload and retrieve
| file name |
| file-1 |
| file-2 |
| file-3 |
@upload_retrieve_files_4_to_6
Scenarios: Files 4 to 6 to upload and retrieve
| file name |
| file-4 |
| file-5 |
| file-6 |
@upload_retrieve_files_7_to_9
Scenarios: Files 7 to 9 to upload and retrieve
| file name |
| file-7 |
| file-8 |
| file-9 |
Now is that we have split the table of a Scenario Outline into smaller tables with tags. We could create individual cukes runners that will just execute the tags.
What's a Cukes Runner?
- It is the one that executes your feature files, Scenarios and Scenario Outlines.
On executing the Cukes Runner, it would go to the feature file you have mentioned, pick the Tag from it and execute just that.
Shown below is an example of a Cukes Runner, mention the path to the feature file and the tag name. You're all Set!
That was easy!
import org.junit.runner.RunWith;
import cucumber.api.junit.Cucumber;
@RunWith(Cucumber.class)
@Cucumber.Options( format = { "pretty", "html:target/results", "json:target/cucumber.json"
}, features = {"PATH-TO-FEATURE-FILE" },
tags = {"@TAG-NAME"})
public class RunCukesTest {
}
Now is that we have set up cukes runner for every tag. We have three tags, three cukes runner, each tag contains 3 files. If we run each tag at the same time, there would be three tests running in parallel and each test will deal with 3 files. Hence, for each test to complete it would take 3 minutes and in just 3 minutes we complete the execution of the whole test.
From 9 minutes to 3 minutes? This is awesome!
Yes it is... But wait, there is more!
"Breakdown huge Scenario Outlines into smaller Scenario Outlines by splitting their tables with tags"Most Efficient case:
If we split the nine rows of the table into nine individual rows, tag each row, set up its own cukes runner, then we would have nine tests for the same scenario outline.
If we execute all nine tests in parallel, in each test we would deal with only one file and that would take only one minute to execute. Since all of them are running in parallel, it would only take one minute for the whole test to be complete. Hence, we have dropped down the test run time from 9 minutes to 1 minute.
Did I just make my test run 9 times faster? I feel like a superhuman.
Transforming Scenarios into One Scenario using Data Table
Sometimes, there is always another way. Yes, there is another way to reduce the test run time without a Scenario Outline, without splitting tables, using tags and setting up cukes runners. That is by the use of data table.
Shown below is a Scenario with a table called as data table.
# Transforming Scenarios into Scenario using data table #
@upload_files_retrieve
Scenario: Upload, wait and retrieve files
Given I upload a file "<file name>"
| file name |
| file-1 |
| file-2 |
| file-3 |
| file-4 |
| file-5 |
| file-6 |
| file-7 |
| file-8 |
| file-9 |
And I wait for all the files to be processed
Then I search and retrieve the files
In the above Scenario, the Given statement will be executed repeatedly for every row in the table and only then it will proceed to the And and Then. In this case, all the nine files will be uploaded first and then finally we just wait for only 1 minute for all the files to be processed. That is it, we are done!
We just reduced the test run time to 1/9th of its original time, once again
Note that Data Table may not be useful at all cases, it depends on the functionality of your tests.
"The way you write your feature files defines your tests, code re-usability and test run time"
How to run tests in parallel?
We have broken down scenario outline into several tags, every tag has its own cukes runner. That's great!
I have 23 cukes runner now, how do I run them all at once? Don't tell me I have to run them one by one manually.
- Well, that is one way of doing it. But not to worry, we actually have a much better way.
To execute all the cukes runner at once, we are going to use Maven SureFire Plugin.
To implement Maven SureFire Plugin (MSFP) is pretty simple.
Add the plugin to your pom.xml
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
<useFile>false</useFile>
</configuration>
</plugin>
</plugins>
Since, we are dealing with uploading and retrieving files, name your cukes runners such a way they have a pattern.
Examples for naming Cukes Runners
Run_upload_retrieve_files_1.java
Run_upload_retrieve_files_2.java
Run_upload_retrieve_files_3.java
Run_upload_retrieve_files_4.java
Run_upload_retrieve_files_5.java
Now we need to create a profile for the set of cukes runner we want to execute in parallel.
Add this profile to your pom.xml
<profiles>
<profile>
<id>uploadAndRetrieveFile</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<version>2.11</version>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<testFailureIgnore>false</testFailureIgnore>
<includes>
<include>**/Run_upload_retrieve_files_*.java</include>
</includes>
<parallel>classes</parallel>
<threadCount>10</threadCount>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
In the above profile, things to note
<id>uploadAndRetrieveFile</id>
We could execute the cukes runner by using the id.
<include>**/Run_upload_retrieve_files_*.java</include>
This will run all the cukes runners from any package whose names starts with Run_upload_retrieve_files_ and that ends with .java
<parallel>classes</parallel>
Says that, run the classes in parallel.
<threadCount>10</threadCount>
Thread count mentions the number of thread.
More number of threads, more faster would be the execution.
Warning: If your code is not thread safe, you would run into concurrency modification issues for higher values of threads.
To start the execution,
In the terminal, go to the folder of your project and type
mvn clean -P PROFILE_ID test
mvn clean -P uploadAndRetrieveFile test
That's it! You will see that the tests have started running in parallel.
You could go check out the cucumber reports, once your executions of tests is complete.
Your comments and suggestions are much appreciated.
