Print:

Print this page

Share:

Send to friend

Download:

Automated_Testing.pdf

Automated Testing - White Paper

01 Jun, 2010

Business applications depend on many kinds of technologies, like databases, operating systems, and even compilers. When an application provides value to a company over a prolonged period of time, however, these technologies invariably become obsolete (as is the case, arguably, with IDMS and SAG-Natural today). When organizations make the decision to replace the technologies behind their applications, most of the time they can complete the exercise through a technical transformation of the application source code and meta-data, for instance, with a migration of IDMS/DB to Oracle or Natural to .NET.

For large, mission-critical applications, there can be many millions of lines of code that get transformed in the process. Two important concerns companies need to address before they put their transformed code into production and try to run their business with it are: “Does it all work?” and “How can we be sure?”
The traditional approach to these questions is to be sure by testing, and often companies enlist the help of users to test the new application because users are so familiar with the functionality they can test almost instinctively.
While many companies have some success with user tests, there are nevertheless some important drawbacks:

• It can be difficult to establish how much of the source code has actually been tested at any one moment in time;
• Human testers who rely on eyeball comparisons of two data-filled screens side by side can miss errors so the method is not 100% reliable;
• Users might have to repeat their full testing regimen (potentially more than once) depending on the number and severity of errors they find.

Beyond a doubt, the human factor is the weakest link in any testing exercise due to its high cost and its lack of accuracy. This is why the latest approaches to testing automate the process wherever possible in order to reduce human involvement.

Anubex has developed its own automated testing framework (ATF as it is known for short) that saves companies the inconvenience and cost of distracting users from their essential daily activities. At the same time, through automation it increases the speed, accuracy, and completeness of testing as a whole. This framework has proved to be very popular with our customers, who in some z/OS and BS2000/OSD migrations have been able to reduce the involvement of their users in testing by as much as 98%.

 The Promise of Automated Testing
Anubex’ ATF leverages the fact that users are constantly executing business transactions as part of their daily routine. Instead of asking them to explicitly document their interactions in the form of test scenarios, ATF just records all interactions during a certain period of time as they happen.
The result of this recording is a massive set of real-world screen interactions, encompassing the data and layout of each screen being sent and received. This recording also maintains detailed order and session information, making it possible, even with thousands of concurrent users, to later replay the exact same sequence of user interactions on the migrated application.
So how can we make use of these recording and replay facilities? After replaying, we compare the results using a specialized automated agent. The obvious comparison – and also the one that human testers would make – concerns the actual screen contents: are all fields present and is all data in the fields identical?

 ATF goes further however. It also checks non-visible properties: invisible fields are compared just as their visible counterparts and all specific screen and field-attributes (e.g. protected/unprotected, numeric and other ‘invisible’ properties present in IDMS OLM maps) are compared too as each inequality might indicate a migration issue. For those differences that are expected (e.g. in Natural to Java conversions the system variables DATX and TIMX will produce different results at different times of the day) the comparison agent can be trained to ignore or partially ignore any differences.

 Beyond Basic Functional Comparison Tests
What we’ve described above is the basic use case for ATF: an automated recording, followed by an automated replay and finishing with an automated analysis. This in itself is a substantial improvement from the manual, user-driven testing process described earlier. However, there is more.
ATF can be used also for the following:

• Calculate (automatically) the performance of the original and migrated applications, in terms of min., max. and average response times.
• Indicate (automatically) which transactions have been executed, and more importantly: which not. Based on this information, users can prioritize the execution of specific transactions during a subsequent recording, thereby ensuring maximal test coverage.
• Stress test (automatically) the migrated application by replaying the same set of business transactions over and over, with an ever increasing number of simulated users, and while keeping track of the evolution of the response times
• Analyze recurring patterns of screen interactions and generate a document for each such identified ‘business processes’. This effectively creates test scenario documents based on an analysis of the types of interactions that real users have with the system.
• Select specific test-scenario’s for creating a regression test-suite that can be used during maintenance.

 Summary

When compared to the traditional approach of saddling users with test-driving their migrated application, ATF offers substantial benefits:

• It eliminates the possibility of human error or oversight;
• It drastically reduces the amount of time needed for testing the whole application;
• It saves users from losing valuable time in tedious testing; and
• It drastically increases the amount of test coverage that can be reached within a certain time-frame.

As such, Anubex considers the use of ATF a very important factor in ensuring a successful migration project.