Siebel Scripting Challenges in Loadrunner

Load test scripts are difficult to create due to highly dynamic nature of Siebel applications. Load test scripts typically automate transactions at the protocol level for maximum scalability, however Siebel requests are very dynamic and if the recording tool captures hard-coded data values then they will not play back.
Manual parameterization is often required which is tedious and time-consuming, plus it requires extensive Siebel expertise. Once you get your tests running, the highly distributed Siebel architecture makes performance bottlenecks difficult to identify.
Correlation in Siebel:
Correlating standard HTML pages is not a difficult task, all you usually need to do is look for a value inside an HTML tag and match it to a regular expression, which tends to be an easy task, in the case of Siebel however, it does not send all of its information in an HTML format, instead it does so in a very difficult format to read by humans and to correlate using regular expressions, see the text below for an example.
@0`1`3`3``0`UC`1`Status`OK`SWEC`16`1`0`ResultSet`0`Return Alerts`<pre><br><font face="verdana" color="red" size=3>   Executive - Transfer call to Executive Specialist Team if applicable<br>or<br>Handle with Care <br> </font></pre><br><font face="verdana" color="black" size=2>   &nbsp&nbsp&nbsp1. Pension Eligible`0`12` Notifications `0`2`0``0`OP`bn` bc`S_BC4`7`0``0`br`0`ArgsArray`34*qrs WMT Contact Toggle FormApplet 1*1`size`1`cr`0`OP`g`bc`S_BC4`type `SWEIRowSelection`2`0``0`OP`en`bc`S_BC4`2`0``0`OP`bn`bc`S_BC2`7`0``0`br`0`ArgsArray`38*qrs WMT Contact Summary SR List pplet1*11*01*01*01*0`size`5`cr`4`OP`g`bc`S_BC2`type` SWEIRowSelection`2`0``0`OP`en`bc`S_BC2`2`0``0`OP`bn`bc `S_BC3`7`0``0`br`0`ArgsArray`29*FIN

While developing Siebel performance test scripts using load runner we need to do very few script enhancement.
SWEACn ---> Siebel Web Extension Applet count.
This value usually appears in web_url() and you can correlate @ first occurrence in the start and replace all other occurrences with the correlated variable name.

SWERowids ---> Siebel Web Extension Rowids
Kindly note this value is not same as the one mentioned earlier. This value would correspond with the SWEMethod implemented/used in the step. So the Ordinal No depends on the SWEMethod in the Server Response.

SWERowId = rowid to perform the action
SWERowIds = record's parent rowId

SWETS ---> Siebel Web Extension Time Stamp
This can be correlated by using web_reg_save_timestamp() just above the request and pass it.
SWEC ---> Siebel Web Extension Click count
This can be correlated by increment method followed in LR or using the web_reg_save_param ()

I know Siebel Correlation library will reduce the scripting efforts to some extent, but I would also like to focus on row_ids and SWEACn and SWC correlations.

Read "Correlating Siebel-Web Scripts" in VuGen guide which will help you to import WebSiebel77Correlation.cor file. This makes your correlation part easier

Dynamic Record Changing:
Problem: whenever any new record is added or existing record is modified the position of records will change. While recording we need to select a record (SR) from the list of records, but the selected record position will change whenever any new record is added/modified, this leads to a problem while replaying the script as the record position has changed.

 Default: while trying to correlate the record items (each record has atleast 20 fields) the boundaries of fields are changing dynamically, and there is no specific pattern for boundaries.

Example:  1-4611378121*21HealthandWelfare10*Phone
                  Call1*05*Sivakota2*HW7*Inquiry0*4*TEST5*COBRA19*0

If we want to capture ---> the boundaries of SIVAKOTA are changing dynamically without any specific pattern.

Solution: To the above problem we are explicitly capturing the entire record and storing into the parameter array.
//*1-4819497423*HRO3*ODM1*15*Sivakota7*Payroll11*Transaction0*5*ALBEE12*

web_reg_save_param("Record","LB/DIG=*#-#####","RB/DIG=*#-#####","ORD=1", LAST);

The entire record is captured into array “Record”, now we are explicitly matching the field values, by writing some code logic it will take the first value, and using StringCheck function whenever the value is not matched StringCheck function will return a null pointer. Once the value is matched, the matched value will be send as field input to server.

Pros:  The logic will handle dynamic record changes And Script maintenance is less.
Cons: All the field values should be statically kept into pointer array.

          Ex: char* LastnameArray [] = {"ALBEE","ADKINS","TEST","ADELMUND","Sivakota1"};
          Memory utilization is high.

Siebel Load and Performance Testing.

Siebel Load and Performance Testing

Unlike functional testing, performance testing works on a protocol level, in the case of Siebel that's done by simulating the HTTP requests that the Siebel UI generates. We can use any web testing tool for testing the Siebel Application.

The Siebel Test Automation framework allows you to implement secure access to test automation, in which the SWE requires a password to generate test automation information. This is useful for real-time testing in a production environment, and can be integrated with system monitoring tools.

Load testing can help you ensure that your Siebel application will perform and scale under real user workloads once it’s deployed to production. This can help you ensure that it will be able to withstand the expected number of concurrent users while maintaining acceptable performance and response times.It can also help you identify and address critical bottlenecks prior to deployment. Monitoring the components of your Siebel environment during load testing is extremely important.

Stress testing can be performed to test beyond the limits of normal operation and helps you assess the capacity and scalability of your application infrastructure automated load testing for Siebel has its own challenges. 

Advantages and Disadvantages of Performance Testing over the Cloud

Performance Testing over the Cloud

Cloud computing is becoming more and more famous and mature with the time and its usage has been increased exponentially. Performance engineers setup the copy of the production system in cloud and deploy load injectors on different geographical locations over the cloud to effectively perform the load test.

Advantages

• Cloud testing provides the flexibility of deploying the production system on discrete environment to conveniently test the application
• It’s extremely simple to fix the defects and quickly configure the changes
• It reduces the test cost due to its convenient rental models
• It provides greater test control to simulate required user load and to identify and simulate the bottlenecks

Disadvantages

• Security and privacy of data is the biggest concern in cloud computing
• Cloud computing works on-line and completely depends on the network connection speed
• Complete dependency on Cloud Service Provider for quality of service
• Although cloud hosting is a lot cheaper in long run but its initial cost is usually higher than the traditional technologies
• Although it’s a short-term issue due to the emerging technology and it’s difficult to upgrade it without losing the data

Performance Test Issues and Trouble-Shooting

Many differences are commonly found between the test and production systems after the proper validation of test system. A differently configured performance test environment will produce invalid results which can greatly mislead all the stakeholders and the application itself can fail in production. There can be a dozens of reasons why your test environment is not producing the required results and some of them are as follows:

• Load Injectors overloaded: Check the load injector machines resource utilization. Quite often load injector machines consume more processor and memory and are unable to simulate the required number of virtual users. Run a small and simple test first and check the system resources consumption on these before running the detailed test.
• Insufficient network bandwidth: Network bandwidth plays a vital role when you are conducting the performance test over the WAN. Test results can greatly differ on the basis of available bandwidth. So make sure that sufficient network bandwidth is available for starting the test. Moreover, you need two network interface cards (NICs) when web server and database server are on different layers, one NIC will be facing the clients and the other one will be used for database communication.
• Improper test data: Improper test data can also create various issues in performance testing. It’s highly possible that a variable is not parameterized and same value is being submitted to the database for every user, which can lead to low processor activity due to artificial locking.

Performance Testing Best Practices on Production System

Performance Testing Best Practices on Production System

Although the various reasons due to which it’s important to conduct the performance testing on production system have been discussed however there are still lots of issues and concerns (some of them have been discussed in the above section) on the basis of which the companies are hesitant to go for it. In this section, we will discuss some of the best practices that can be opted to minimize the impacts of performance testing on production system.

Testing During Maintenance Window

Almost all the large organizations’ applications go for scheduled maintenance and during that period of time they restrict their users from interacting with the application. You can co-ordinate with responsible teams and plan out your performance testing activity during this scheduled downtime without affecting actual users’ experience.

Test before Release

One of the best approaches could be to test the application just before making it available for actual users. You can include application performance testing part of your release management plan to make sure that performance tests are always executed before releasing the application.

Test during Off-Hours of Off-Days

Conduct the performance testing during off-hours of the off-days if you are not left with any of the above two options. Minimum number of application actual users is affected on conducting the performance testing at this schedule. It not only helps in minimizing the impact of testing on real users activities but also in identifying the bottlenecks root-causes. The best and most suitable time considered for such approach is midnight Saturday or Sunday.

Test Read-only Transactions

Many companies don’t prefer to do any testing activity on their production system due to the fear that the test data might get mixed with the actual applications users’ data. Especially in case of business critical applications, companies are not willing to take even the minor risks. That is why production database is almost never used in testing and even if it’s used, it’s used only for the read-only operations. These simple transactions don’t affect the application data but can reveal important performance bottlenecks.

Increase Load Gradually

One approach that could be exercised to minimize the impact of performance testing on real users is to increase the simulated users gradually unless the real users’ transactions are within the acceptable threshold. We have mentioned above that performance testing is not all about breaking the system but also to find out the application behavior under normal conditions. Run a test and increase the load gradually unless the users’ response time is within the acceptable range and they are able to successfully perform their transactions. Then analyze the test results, fix the bottlenecks and re-test. You can thoroughly test any application for most of its bottlenecks in multiple iterations without actually impacting the real users, however they will be experiencing slower user experience but still will be able to complete their transactions.

Careful Monitoring and Continuous Communication during Test Execution

The performance testing approach and its expected outcomes along with the involved risks should be clearly communicated to all the stakeholders. Moreover, you need to be very pro-active while testing on the production system and all the stakeholders should be carefully monitoring the test and test should be stopped immediately if and when it affects the actual users beyond their acceptable threshold.

Performance Test Environment Checklist

We all know about the importance of having test environment similar to the production system. Once we have setup the performance test environment, we can get an initial idea of the test environment state by comparing it with production environment based on the following factors:

• Number of Servers: Number of physical and virtual servers
• Load Balancing Strategy: The type of load balancing mechanism is in use
• Hardware Resources: CPUs count and type, RAM capacity, Number and type of NICs
• Software Resources: Standard application build apart from components of the AUT
• Application Components: Application components description which needs to be deployed on the server
• External Links: Links to third party application and other internal system components

Performance Testing in Cloud

Benefits of Performance Testing in the Cloud

All levels of testing could be performed in cloud infrastructure, but performance testing benefits greatly from cloud environments. 

Flexibility

Different levels of tests can be executed on discrete environments at the convenience of an enterprise. Performance testers no longer have to wait until the end of the testing phase in order to move to a production-like environment for their performance and stress tests. Instead such an environment can be brought into action at will. 

Simplicity

The cloud model provides a new level of simplicity in the form of bug fixing environments that can be launched as quickly as the configuration can be put in place. 

Comprehensive Testing

End-to-end tests for more generic processes can be performed in the cloud. All the necessary components can be published in the cloud to create the complete chain of systems. In this manner the overall business process can be tested;

Cost Reduction

Cloud environments could be enabled and disabled at will, reducing the cost of environmental management. Cost reduction is the major factor influencing companies to choose Cloud. As per IDC survey reports, economic benefits are the key drivers of cloud adoption. 

Cloud Testing leverages the cloud infrastructure, minimizing the unit cost of computing and increasing the efficiency of performance testing. The report on cloud enabled testing service providers reveals that the cost savings usually range from 40% to 70%. 

Small and medium-sized enterprises (SMEs) that cannot afford huge capital expenditures also find cloud enabled performance testing an ideal approach. As there is no need to make upfront payments in infrastructure, Public cloud allows enterprises to shift to a flexible operating expenditure model. 

In case of Private cloud, infrastructure can be deactivated once the testing process is complete. This frees enterprises from incurring expensive operational costs.

Cleaner and Greener Testing

It is apparently true that cloud computing capabilities make it significantly greener than traditional models and this is true for testing process. By just sharing cloud resources for their test infrastructure, enterprises can use IT resources on demand and eliminate waste. Consumers using cloud infrastructures can minimize energy use and deliver environmental savings in carbon dioxide of around 55%. 

Greater Control

Cloud-based environments can provide greater control on test execution, analyze application performance and find bottlenecks while the tests are running. Cloud model allows test engineers to ascend from a few thousands to millions of concurrent users to evaluate breaking points. This gives testers a perfect picture of all possible runtime errors and adapts enterprises for peak demand times. 

Internal Lab Testing vs. Cloud Testing

So what is the best choice? 

• Setup an internal copy of production as a test environment and use several computers to generate load internally

• Setup an internal copy of production as a test environment and use load injectors on the cloud to generate load distributed geographically

• Setup a copy of production on the cloud as a test environment and use load injectors on the cloud to generate load distributed geographically

We saw that performance testing from the cloud gives you a complete understanding of the final user experience and reduce drastically investment and configuration costs. However, it may not fit to all organization (security, product licenses) and can complexify the analysis of performance bottlenecks (too much variables). 

These choices depend really on the type of application to be tested and the company culture and processes. 

A first performance testing run in a simpler lab with smaller loads is still valuable as it gives an overview of early performance issues. An application which does not pass the lab test, needs to be tuned before going to larger scale testing over the Internet! 

A load testing tool which supports both lab and cloud testing with the same use of scripts ans use cases across both types of tests is definitely a winning choice as it gives you flexibility and scalability across your project.