Skip to main content

Automation with SoapUI – Part II


Automation-with-SoapUI-p2 by 9series solutions

In the previous article Automation with SoapUI – Part I, we learned about various features of SoapUI for functionality testing of API services and automating test cases & creating Test Suites. However, the usage of SoapUI is not limited to Functional Testing. SoapUI has amazing abilities for non-functional testing like Performance Testing and Security Testing. One of the most important features of SoapUI for performance testing is that we can create complex functionality scenarios and then load test them. This helps test various business scenarios on different loads depending upon usage. In this blog, we will learn the various feature of SoapUI in the Performance Testing domain.
Performance Testing can be classified into various categories –
1. Load Testing – In Load Testing, the Application Under Test (AUT) is subjected to a higher load than its handling capability to test the behavior of the AUT.
2. Stress Testing – In Stress Testing, the AUT is exposed to much Higher loads than its handling capacity to test its breaking point.
3. Soak Testing – The AUT is continuously subjected to high loads for long durations, to test its reliability for higher loads.
4. Baseline Testing – Testing the AUT for the expected normal load, and check how the AUT behaves under normal load.
5. Scalability Load – AUT is subjected to rapid & sudden variations in loads to check AUT behaviour under peak loads.
A Load Test executes the parent Test Case repeatedly for a specific time with the required number of threads or virtual users. A load test can be added to a test case, by right-clicking the Test Case in the navigator panel and selecting New Load Test from the list menu. In the load test window, we can adjust various parameters like the Number of Threads, the Time Limit, Normal & Random Delay. Multiple Load Tests can be added to the Test cases and multiple load tests of multiple Test cases can be run in parallel to each other.
Testing Strategies
We can also select the strategy for the load test from the given options. These Strategies allow us to change the number of threads during execution, which allows us to change load while the load test progresses. Thus we can monitor the results, and change the load on the AUT as the Load Test executes. The data collected for the load test is shown in the Load Test statistics table and is continuously updated as the load test is being executed. The collection and of statistic data is asynchronous and independent from the actual Load Test execution, not affecting the load test directly. Let us see the load testing strategies
9series Solutions : Automation with SoapUI - Part II
1. Simple StrategyIn this strategy, a specified number of threads are running with the specified delay between each run to simulate a real-life scenario for the AUT. Simple Strategy is the default strategy in SoapUI. It has two fields namely Test Delay for entering the delay time and Random field to enter random delay time. The Simple Strategy is commonly used for Baseline testing. It helps to test the basic performance of the AUT and check if there are any bad responses or embedded resource download issues. This strategy can also be used for soak testing and load testing.
2. Variance StrategyIn this strategy, the load varies over time in a sawtooth manner. There are two fields available in this field. First is the Interval field, where we can enter the period in seconds for which the variance in load will take place. Second is the Variance, that is by how much the load needs to be varied. This strategy is good for testing AUT under rapid changes in peak loads.
3. Burst StrategyIn Burst Strategy, a high number of threads are executed for a short time to simulate a burst in web traffic. This strategy is used to test the recovery period of the AUT after it breaks for the unusually high load due to the Burst load. There are two fields in Burst mode namely Burst Delay, where we can enter the period for which the Burst load is to be delayed. Another field is Burst Duration, where we enter the period for which the high load is applied to the AUT.
4. Thread StrategyIn this strategy, you can increase load linearly while the execution of the load test is on. Its main purpose is to identify on which level, statistics begin to deviate from normal behaviour and certain errors are occurring. There are two fields, Starts Threads and End Threads, where we can enter the number of the thread to start with and the number of threads to end with respectively.
Test Statistics:
In the Statistics table, we can see all the statistics related to the load test executed. There are various statistical parameters like CNT, TPS, BPS, error and many more to determine the result of the test and form a detailed report of the performance of the test case.
2
For better visualization of test results, SoapUI provides two graphs showing results of the load test in graphical form.
~ The Statistics Graph shows all relevant statistics for a selected step or the entire test case over time, allowing you to see how values change when you increase the number of threads.
~ The Statistics History Graph shows a selected statistic value for all steps allowing you to compare them and see if the distribution of any value between test steps changes over time.
– AssertionsSimilar to functional assertions in Test Steps of a Test Case to validate the results, we can add Load Test assertions to a Load Test to check both performance and functionality during the execution of the LoadTest. The assertions are continuously applied to Load Test and can fail the Load Test just like their functional counterpart. The assertions which you can add to load test are as follows –
31. Step Average – Asserts that the average value of a Test Step or Test Case doesn’t exceed the specified limit. If the average goes above the value, it fails the load test.
2. Step TPS – Asserts the Transaction per Second (TPS) value for the corresponding Test Step or Test Case. If the TPS is less than specified, the load test is failed by this assertion.
3. Step Maximum – Asserts the maximum value for the corresponding Test Step or Test Case. If it exceeds the specified Max Time, the assertion is failed.
4. Step Status – Checks the errors occurring for the specified number of executing requests does not exceed the specified error limit, else the assertion is failed.
5. Max Errors – Checks that the number of failures for the Test Step or Test Case does not exceed the configured Max Absolute Errors and/or Max Relative Errors value.
When execution of the Load Test, assertion failures are displayed in the Load Test Log. Double-clicking a log entry displays the corresponding result, allowing you to view the failed request message.
Conclusion
SoapUI provides a unique solution to many problems posed to load test your functional automation testing. The various strategies available help in testing the AUT in many different ways and scenarios. The Statistics table provide statistics which are easy to understand and interpret. And Assertions check if the load test statistics are within specified limits. For Load testing API functional tests, SoapUI definitely seems a worthy option.

Comments

Popular posts from this blog

Employee Attrition and Retention – Part 1

Introduction This is the 1st blog in the series in which we are going to discuss the attrition in an organization. An organization is built with teams and the team’s growth is dependent on each employee, but when an employee quits the job, it can disturb a team’s dynamics. Other team members may find themselves scrambled to cover the extra work for weeks, or even for months until a new candidate gets hired. Hence, it becomes important for any organization to work towards retaining the key employees. Reasons Employees Want to Leave Their Jobs Here are some very common reasons that make an employee leave the job based on certain statistics we have collated the best information from various survey & studies to give you a clear picture of where an organization can go wrong and what causes employees to resign. ~  The job wasn’t what they’d expected ~  Lack of coaching or feedback ~  They didn’t get along with the boss ~  Lack of opportunity for career advance...

Automation with SoapUI – Part I

API test automation helps reveal integration issues across the entire chain of services or systems, thus preventing number of bugs and issues, not to mention the time and efforts required to fix the issues after bug leakage. Even a small change in an API will give a different response, and will be detected by automating API testing. The appearance of APIs (Application Programming Interface) and communication protocols like SOAP and REST took software development into an altogether different level. It became relatively easy to integrate completely different services and develop more powerful and useful softwares. APIs became an important part of software development and also one on which the entire software are sometime depended on. This led to the developers and more specifically, testers, to focus on making APIs more reliable and bug free as much as possible.  API Testing  today, is an integral part of the testing process. The APIs must be tested thoroughly by integrati...

Where I can get Business Event Mobile App Development Solutions?

9series came into being in the year 2007. 9series have completed more than 100+ mobile app projects successfully in Android, iOS (iPhone & iPad) and Windows. 9series located in New York, US and our development center in Ahmedabad, India, we also have another branch in Africa and Kenya. We are working for Domestic and International Clients and our main aim is to build a solid reputation in the Global market. We have 30+ professional mobile app developers in Android and iOS respectively. We are well known for Mobile App Development Company in India & USA . 9series are global experts in developing event management solutions and helping result oriented experts to accomplish a multitude of successful business events each year. We approach each phase of business event from planning, organizing, marketing, and registration to post-event analysis and reporting. Our business event management and conference mobile app solutions make life smooth and suitable for event organizers. ...