We all know that automating the often tedious, lengthy, and expensive user acceptance testing (UAT) is going to result in a much faster process — but how much acceleration can you realistically expect? Today, I want to answer that question by walking you through the real-world results we achieved with a very large international investment bank.
First, I want to walk you through the actual processes that were being used (both the manual and the automated) and show step-by-step how long each process took manually and how much time the large investment bank saved by switching over to automated processes. Then, let’s have a look at the impact this change would have if you either package 500 applications at once or 20 applications per week, every week.
90% Time Savings By Automating A Labor-Intensive Manual UAT Process
Let’s start by having a look at the manual process:
- First, the application owner requests a virtual machine via ServiceNow (about 30 minutes).
- An engineer will then add on the prerequisites (another 30 minutes) and install the application manually (15 minutes).
- The UAT tester will then manually test the application (15 minutes), take screenshots of the application screen (15 minutes), and save the logs and errors (5 minutes).
- Finally, the engineer will reclaim the virtual machine (10 minutes).
Total Time Of Manual Test: 120 Minutes
In theory, this process takes 120 minutes, but since there will be additional wait times to consider, such as hand-off between the UAT tester and the engineer and back, in reality, this takes a lot longer. But for comparison’s sake, we will go with 120 minutes.
Now, if you automate the process with an automated application packaging and testing solution like Access Capture (and if you remember that you don’t want to just speed up a bad process, but instead want to optimize the process first before automating it to get maximum velocity) your process will look very different:
- First, the application owner self-service selects a virtual machine from a pool of ready VMs (1 minute, 97% time savings).
- Then, you select the prerequisite for your applications (1 minute, 97% time savings) and let Access Capture automatically install the application for you (5 minutes, 67% time savings).
- Next, the UAT tester performs the application test, which is the only manual part of this process (2 minutes, 87% time savings) while Access Capture automatically takes screenshots and video records the test in the background (1 minute, 93% time savings).
- All data is automatically stored in SQL/UI (1 minute, 80% time savings) and the virtual machine is returned to the pool upon completion (1 minute, 90% time savings).
Total Time Of Automated Test: 12 Minutes
Please note that multiple steps took mere seconds, but for the sake of this calculation, we rounded each step like this up to a minute. All in all, the process took less than 12 minutes (90% time savings), but since the process was completed by one person in one sitting, there weren’t any additional waiting times between steps. Also, I want to point out that since Capture logs, records, and screen captures everything automatically, you not only significantly improve the user experience of the tester but also ensure a standardized test outcome.
Save 50 Resource Days When User Acceptance Testing 500 Applications At Once
Saving 108 minutes while UAT testing sounds good, but how impactful is this really for your organizations? An average enterprise touches about 1500-3000 applications with a larger IT Transformation project, such as a Windows 10 migration. Now, let’s compare the impact if you just test 500 applications:
- 50 Days Per Resource Saved: if you have been using 2 testing resources for the manual process (62.5 Days). In other words, you can accelerate more than two months’ worth of work into less than half a month.
- 900 Total Hours Saved: if you assume 8-hour working days (1,000 hours for the manual testing)
- 54,000 Total Minutes Saved: if you are using the 60 minutes per hour metric (the manual process would take 60,000 minutes)
Reduce 5 Working Days Per Week To Half A Day By Automating Your User Acceptance Testing
If you are consistently testing your applications (in our example of 20 applications a week), you can significantly reduce the time and resources required by automating your User Acceptance Testing:
- 4.5 Resource Days Saved Per Week: Instead of 5 full working days, your tester now only spends half a day a week testing 20 apps.
- 36 Hours Saved Per Week: Spend 4 hours instead of 40 hours testing your apps.
- 1,260 Minutes Saved Per Week: Spend 240 minutes instead of 2,400 minutes a week testing your 20 applications.
As you can see from the numbers above, the time savings are astonishing. But this is only the time savings. There are also tremendous hard and soft savings as well as improvements in user experience to consider. In the coming weeks, we will take a look at the financial side of this use case and show you, based on industry averages, how many actual savings you can expect.
