When implementing #TestAutomation it is important to ensure your automated testing tool/s, can be flexible and supportive of a wide range of applications and languages. Integrating test automation into your development process and framework will provide a higher return on investment (ROI) for teams looking to optimize their software development processes.

Most teams tend to automate tests but seem to forget the importance of integrating their automated tests into the overall testing process. This means automated testing should complement and improve the overall software quality allowing manual testing to focus on key business and functional areas which may not be ready for automation, or simply best left for manual testing. Either way, its important that your test reporting highlights the overall status of both automated and manual testing. With the increasing push to to shift left, automated testing needs to be able to provide fast, accurate test and code coverage, application insights and reports which can be accessible for delivery teams to review and asses.

Unfortunately this is where majority of QA Process tends to get tangled at adopting the most basic automation reporting principles, needed to be in place in the software delivery process. Each time your developers commit code changes, your automation suite/s should be triggered to provide rapid validation against the most recent code changes. Failed tests need to be visible and clear, allowing both QA and Developer to coordinate and account for tasks, reviewing results and detailed test logs with actionable items for for remediation prior to moving to the next task.

Application Monitoring Automation Process

To build a focus on continuous development and production deployment, your automated testing strategy should be set up to monitor your production environment to expose issues as they occur prior to the end user.

Having a test automation framework that can capture and report on business transactions and response times coupled with application monitoring tools such as Dynatrace will provide deeper insights into the experience of the users.

For example, if the average response time for Check-Out function on your website is impacted by a recent change in the underlying application code or technology, then your monitoring process and automation reports must proactively report on this performance issue including performance trends over the build life-cycle. This is where your automation will definitely demonstrate a greater return on investment. So it is important that your automated test results can be captured in a database to produce trends with pin point analytics to address issues as they occur.

Capturing response times against business transactions and counters provides many important insights and analytics that support real issues. The following benefits will be achieved based on a well-defined test automation and application monitoring strategy and solution.

  • Measure the end user experience across technology
  • Create real insights and visibility
  • Troubleshooting
  • Identify the root cause of the problem

For more information on application monitoring see Synthetic Application Monitoring

Conclusion

Integrating automation into your development and testing process can produce many great results. Deploying software in this current competitive landscape means software test automation is becoming more and more essential. Although the adoption of some form of automation is on the rise, the full potential may not be experienced by some teams due to the lack of integration within their development and deployment process.

The right selection of automation, testing process, and team, are important ingredients for a successful automation capability. If you are looking to implement software automation testing within your organisation, or need assistance to deploy the necessary framework, you can contact Testpoint and request more information about our Test Automation Proof Of Concept Services

Wondering why Robotic Process Automation (RPA) is relevant? According to Gartner, Inc. RPA software spending is predicted to reach up to $2.4 billion by 2022. A smart and cost effective solution to automating tasks, RPA has been on the rise for a while and is only growing. What are organizations doing about it? Continue reading to get an answer.

While organizations face challenges with RPA at the start, once implemented successfully, the ROI is “ten fold”. RPA has helped many organizations to automate most of their repetitive manual tasks and processes they had in store. Gartner, Inc. estimates that by 2022, 85% of large organizations will have deployed some form of RPA.

What are the Key RPA Limitations?

RPA also comes with its own set of limitations, some of which are discussed below:

  • A major disadvantage of RPA is its inability to read non-electronic data with unstructured inputs, thereby requiring companies to use it in conjunction with other solutions
  • RPA is a new technology that most employees might not be comfortable with, so it adds confusion for the employees at the start
  • RPA requires a certain centralized infrastructure, and RPA cannot be applied if that isn’t present, requiring you to undergo a lot of infrastructural change to apply RPA

The Top 6 Challenges of RPA

Implementation of new technology such as RPA comes with its associated challenges discussed below:

  1. Employee resistance –employees usually do not welcome change, and RPA is a new change that requires the employees to be trained in it. This might result in employees resisting the implementation of this new technology altogether if not managed correctly
  2. Choosing the wrong RPA – most companies don’t have the guidance available to help them choose to correct RPA, and go on to select a solution that doesn’t suit their organizational needs without complete understanding of the technology
  3. Responsibilities are mismanaged –when deployed, RPA has to be a designated person responsible for each element of the automated process. Otherwise, the responsibilities will be mismanaged
  4. Lack of a skilled team – RPA requires that you have resources with the necessary technical skill and business process experience
  5. Expensive solution –while RPA returns an impressive ROI in the long run, it comes at an additional cost at the start which may be a challenge for organizations to meet
  6. Change Management Plan – RPA requires regular maintenance and monitoring especially prior to implementation. Changes to systems and processes need to consider your RPA environment to ensure the solution will continue to work and be effective

What solutions should Organizations be looking for to establish RPA?

To help with establishing RPA, there are multiple steps that organizations should consider when acquiring solutions according to Gartner, Inc. These considerations are listed below:

  • Formalise an automation map – list down pros and cons of RPA, see what processes are suitable for it, are there cheaper alternatives present to automate the processes
  • Focus on short term results also – revenue generation is an obvious goal of RPA, but don’t ignore possibilities for short-term results, such as increased customer satisfaction, better quality and standardisation
  • Involve IT early on –no technology can be introduced without IT, so it’s important that they understand how and why RPA is the answer to the company’s problems
  • Create a team of both IT and business RPA members –process leaders might not have the technical knowledge needed, which is why working with IT members for the process is important
  • Build employee skills –RPA requires training for most employees, so build those skills by seeking external help to run RPA
  • Clarify what RPA is for your company –there should be a clear philosophy behind this deployment for employees to follow behind

Things to consider for solutions

The above steps come with different alternatives that can provide solutions. We have listed below a few options for your consideration:

  • RPA – Optical Character Recognition (OCR)
    OCR is a technology used by RPA to convert images into text which can be converted into usable data. For example a robot would read an email to process a PDF statement file. Instead of manually keying in the data, OCR technology identifies and extracts the relevant pieces of text which would then be automatically entered into the target application by the robot
  • Artificial Intelligence
    By linking RPA with other software in the vendor’s product suite, it can solve more problems. This linking is made possible through artificial intelligence. One tool won’t solve all automation issues which is why different tools including machine learning, deep learning and NLP are all things to consider as solutions.
  • Custom Software
    There will remain the need to fix unique gaps left with problems despite deploying RPA. For this purpose, there are customer software to deal specifically with those unique gaps and to help in integrating all the pieces together into a well-oiled system
  • BPM – Business Process Management
    Before the advent of RPA, business process management changed the design of the business greatly, which is why it’s something to take into consideration. BPM is where to look when finding lists of tasks already automated, where bottlenecks exist and have been solved, and all tasks needing solving

Some of the most common used RPA tools?

Are you thinking of implementing RPA? Let us help you out by pointing out two of the most famous and commonly used RPA tools.

  • UIPath
    Launched back in 2005, UIPath is built for both business and IT and is now a leading RPA vendor that provides a complete software suite for organizations and their business automation process. The company has won many awards also, including the “Everest Group PEAK Matrix”, Q1 2018. UIPath includes a free community edition for developers, individuals and the UIPath academy
  • Automation Anywhere
    Automation Anywhere founded in 2003 is another leader in RPA. The solution automates webmasters tasks like website maintenance, website testing, data extraction & transfer, website promotions & communications

How can you be successful adopting RPA as a business solution?

To finish it off, it all boils down to how you can successfully adopt RPA as a business solution. Remember important factors such as how IT needs to be involved, the process leader, and the IT department need to form their own team for this process.

Your entire success could depend primarily on your testing, so don’t take it for granted. Keep these points in mind, and you’ll be on your way to success in no time!

Whether you are considering moving to a RPA, or currently experiencing some challenges with RPA, Testpoint can assist you.

Contact Testpoint for a free comprehensive RPA assessment to kick start your requirements.

Learn More

Testpoint – Assuring Your Technology For Tomorrow

Test automation can empower your organisation to verify early and continuously, enabling intelligent data analysis to help your organisation’ responsiveness in addressing critical business challenges for a seamless customer experience.

Effective Software Test Automation can become a sustained source of competitive advantage for organisations.  Without effective sustainable test automation, businesses will experience a drop in efficiency and rising costs with their software testing deliverables.

Without a depth of knowledge and proven experience in test automation, organisations will face several challenges, particularly the construction, management, integration and framework aspects.

This article highlights these challenges and provides insights to test automation with Vansah® and Katalon Studio (an example automation testing tool)

Overcoming Test Automation Challenges

More often than not, we see that test automation is implemented with limitations and inadequacies which disconnect testing from the entire quality assurance (QA) process.

The process of identifying what should be automated, should be a natural progression that can be managed and coordinated throughout your test automation investment.

Organisations that have experienced frustrations in this process are quick to learn that software test automation is not how it was originally sold out to be and required additional investment to remediate as opposed to having a robot to playback a set of recorded test steps.

Identifying tests to be automated can be aligned to the number or size of data required to perform the tests. Having data centralised and accessible by your team and automated tests, while being able to control the type of data required for your tests, is an important component in overcoming some of the key test automation challenges.

Four areas to consider:

There is no one-size-fits-all tool for automated testing tools. It is recommended that a proof of concept and evaluation process is managed to select what would best meet your automated testing needs.

Let us look at Katalon Studio which is a free cross-platform, automation testing tool developed by Katalon LLC. Built on top of the open-source automation framework Selenium, offers a comprehensive set of features to implement test automation solutions for Web, API, and Mobile technologies without the need for prior programming experience.

As a test automation tool and like many other tools, it comes with limitations and rely on integrations to enhance capability required to deliver an end to end automated testing outcome.

Even with existing integration opportunities, basic capabilities are limited such as, collaboration of issues identified from tests in a centralised system for reporting, traceability to requirements, detailed test logs, centralised test data and overall testing coverage across the full testing life cycle.

Regardless of the toolset, your QA process needs to be supported with the following:

Extension and Application

Testpoint™ has extended its Vansah® QA platform to allow test automation testing tools such as Katalon Studio to seamlessly integrate, creating tooling synergy for teams to drastically improve their test management processes and capabilities, with speed, efficiency, and accuracy to deliver a value-add outcome for their organisation.

The benefits are extended further when you have a need to manage multiple test automation toolsets to support your end to end testing requirements. Tests generated from multiple toolsets will be centralised and consumed into test cycles into Vansah® which provide the following advantages:

How does Vansah Work

Vansah® is a #SaaS based platform solution developed by Testpoint™ in 2016, out of a necessity and first-hand experience to help customers overcome manual and test automation challenges. The aim was to accelerate testing maturity, improving quality and velocity of IT delivery, enabling organisations to “shift left” testing early, continuously during development releases, and producing a defect-prevention culture, whilst detecting issues early in production with a “shift right” focus.

Our customers were able to see a reduction in testing costs as the manual testing backlog efforts were reduced, increased visibility to the team enabling manual testers to invest their focus across other critical business functions, and focusing on tests which were not automated.

Vansah® platform is not focused on developing a test automation tool, rather the aim is to remain tool agnostic when it comes to enabling test automation, allowing our customers to adopt the necessary tools with a single framework solution regardless of what robot tool was used.

Test Automation tools are simply used to record, develop and run tests while the necessary management and reporting of tests, coverage and traceability details are managed outside of the tools.

Shift Right with Proactive Monitoring

We have helped our customers to re-purpose their test automation capability to detect performance and user experience issues across pre-production release cycles and production monitoring.  The benefit of re-purposing your automated tests for production systems is to monitor and identify issues prior to customers and end users. The same automated tests used to verify application changes in development cycles can be re-purposed to monitor key business transactions against production systems.Automated Business Transaction Monitoring

Shift Right Benefits:

It is crucial to ensure your approach to test automation can identify the right automation testing toolset(s) necessary to support your requirements (achieved with a proof of concept exercise). This investment will have a great impact on the success of your test automation process. Delivered successfully your organisation can be confident to deliver a sustainable and rewarding test automation experience and outcomes.

Would you like to discuss your automation testing requirement? As us about our proof of concept automation testing services.

GET IN TOUCH

AUSTRALIA’S TECHNOLOGY LEADERS TO MEET IN SYDNEY

ADAPT’s overarching 2018 theme has been AGILITY – how to enable it, achieve it, and sustain it at scale.

What every single C-level event we organise has fed back is that Australian organisations are now hindered by business siloes, culture, people and talent challenges.

Even though we are also at the precipice of major tech disruption and megatrends – it is actually very little to do with technology, and everything to do with managing change, stakeholder buy in, C-suite perception, CX, employee experience, managing budgets and organisational culture.

CIOs are dealing with massive complexity and need to apply tech to solve business problems.

As such, ADAPT CIO Edge Experience is focused on Enabling Agility: Aligning Culture with Business Strategy and will be held on 13+14 September 2018 at the Hyatt Regency Hotel, 161 Sussex Street, Sydney.

This is the 14th edition of ADAPT’s CIO Edge and draws insight from previous iterations, content discussed with business leaders within 50 private events ADAPT has held in months preceding the gathering and on-going consultation with prominent technology leaders.

The event brings together leading Australian CIOs, strategy consultants and business leaders to explore the issues that are most pertinent to Australian technology leaders.

How do you manage the business siloes?

The people, the processes, the projects and the technology?

How do you create winning customer experiences, and a vibrant culture?

  • Event: CIO Edge Experience
  • Theme: Enabling Agility: Aligning Culture with Business Strategy
  • Date: 13 – 14 September 2018
  • Where: Hyatt Regency Hotel, 161 Sussex Street, Sydney.

Dialysis Australia partnered with Testpoint to use its Oncare Health  solution configured off its Vansah platform to support Medibank members transitioning to at home haemodialysis treatment.

The Medibank program commenced in April 2018 across Victoria, New South Wales and Queensland. To take part, Medibank members must have hospital insurance that pays benefits towards dialysis services and have approval from their treating nephrologist. Medibank will fund the service, which is delivered by Dialysis Australia, a qualified and experienced provider of home-based haemodialysis services supported by Testpoint’s Technology Platform.

The service contributes to a more sustainable healthcare system in Australia, freeing up services and resources in hospitals. At 7:30am (EST), June 25 2018, saw the first member transition to at home haemodialysis treatment.

The Oncare Health solution improves home patient care,  enabling healthcare professionals to deliver personalised healthcare services, assuring best quality outcomes in the following areas:

  1. Patient Management (Medical History)
  2. Nurse and Care Worker Management
  3. Care Worker/Home Care Patient Treatment Scheduling
  4. Scheduling
  5. Custom Care Plans
  6. Detailed Reporting, Time Sheet Management, Billing and Invoice Management
  7. Incident Management (Workflow + ITIL support)
  8. Patient Training (Checklist and Compliance)
  9. Lab Results
  10. Medicine Management

Testpoint consulted with Dialysis Australia in May 2018 to understand key requirements for the end to end patient treatment process. In less than 6 weeks, the first release of the solution is live in production, thanks to the high performing agile delivery teams tasked to configure and conduct the required test automation for this release. Testpoint will deliver the next iteration on 6th of August 2018. This is an exciting time for the team to be part of such an important digital improvement to the healthcare system.

Some of the latest achievements in the platform include the release of a progressive web app (PWA) removing any dependency on native mobile application requirements.  This will ensure future releases of the solution are instantly available to users preventing the need for upgrades allowing the technology to be rapidly available.

FIND OUT MORE

CIOs are constantly evaluating the latest technologies, solution providers along with the right business models for their organisations.

As leaders, CIOs must uncover new ways to approach critical business challenges by making the right decisions to help their organisation be more efficient while advancing with the right partners to produce reliable products.

Each year, Australia hosts some of the most prestigious events bringing together visionary leaders and technologies to help CIOs actively engage and find the right partners that will add the most value to their organisation.

In February 2018, Adapt hosted their annual Melbourne CIO Edge Experience event focusing on achieving Agility and Digital Transformation.

The CIO Edge is all about bringing together Australia’s top technical leaders, so they can collaborate and network and understand some of the best insights and practices that are out their globally and locally. ADAPT works closely with their delegation to understand what their investment priorities and challenges are.  ADAPT helps their delegates by going to the market to source some of the best service providers in key areas i.e: Testing.

Anthony Saba “Manager, Adapt Advisory Services”, gives a brief overview as to why ADAPT chose Testpoint as their partner to bring to the CIO Edge Experience hosted in Melbourne and now again based on the successful feedback from CIOs, Adapt has chosen Testpoint to attend the CIO Edge Experience coming up in Sydney, September 2018.

 

FIND OUT MORE

Introduction

An Enterprise Resource Planning (ERP) system functions as the beating heart of many of today’s businesses. At its most basic implementation, ERP software systems integrate various business functions into a complete system to streamline and support processes and information across the entire organisation. This works to support employees to perform their work more efficiently and accurately with access to necessary data and business reports.

This paper will explore the challenges of delivering a high-quality, fully functional ERP solution. We will focus on the advantages that comprehensive testing at all stages of the project by a well-structured test strategy can bring. Having an independent, experienced quality assurance testing partner as part of the team can reduce project risks and help ensure a successful implementation.

The Role of an ERP

ERP solutions vary in their functions and features. They usually operate on an “Enterprise” level, across the entire organisation. The organisation uses it to manage its “Resources” for optimum efficiency. Integrating those resources across the enterprise requires proper “Planning”. The result of a good implementation of an ERP should be a boost in productivity and efficiency. The key factor in an ERP is centralisation of data, resources and business processes into a common platform application.

ERP’s come in many different forms and core functions. The “big three” ERP vendors are SAP, Oracle and Microsoft. Each one offers different features, functionality and strengths. Most ERPs are split into modules – built to address the requirements of a distinct industry vertical or customer type.

Variances in Implementation

Every organisation has unique business and system processes, services and products. Entities of all types, sizes and maturity can benefit from the advantages a comprehensive ERP system brings. ERP systems play a significant role in organising and processing financial data. They also contribute to improving the short-term and long-term business development activity. Utilising ERP software can increase efficiency, decrease costs, streamline processes and improve productivity. Each organisation has different priorities and goals in mind when they select the ERP they eventually acquire. Deciding on which ERP software to implement is just the first step. The organisation must then undertake the task of implementing, configuring and testing it to meet their system and business goals.

Implementation is the process of understanding the business processes of the organisation and configuring the ERP application to deliver those requirements. This usually involves an external party coordinating the installation, configuration and activation of the various modules of the ERP software. Sometimes this party is the vendor, but more often it is a company that specialises in configuring the specific ERP software that has been acquired.

It is this phase that poses the most risk to the organisation. Failure of an ERP implementation project is more common than most would like to imagine. If the implementation team fails to configure a module properly, it can jeopardise the entire system. Without a properly prepared and implemented testing and quality assurance framework, organisations risk ending up with a system that does not achieve their business goals.

What is ERP Testing

The most important component of a successful ERP adoption project is testing. The complexity of most ERP projects requires that testing is done at all stages and all levels of the organisation.

ERP applications are integrated tightly to support an organisation’s business processes. As a critical part of the workflow of the enterprise, testing must be carried out as part of the implementation. The testing strategy should be an integral part of the project plan.

Testing an ERP Implementation is multi-faceted, and should include:

  • Unit Testing of the Application Modules (functional testing)
  • Business Process Testing (suitability to activity testing)
  • Data Conversion/Migration (data conversion & migration testing)
  • Application/Data Integration Testing (data/application interface testing)
  • Performance Optimisation Testing (speed/reliability testing)
  • Security Testing (access role & security penetration testing)
  • User Acceptance (user approval of delivery)

Determining the actual scope of the testing required depends on the situation triggering the change. These scenarios include:

  • New ERP acquisition and implementation
  • Business process changes requiring configuration
  • Interface changes to external applications
  • Upgrades of the ERP software
  • Centralisation of multiple business group

Each scenario requires that the ERP implementation is tested in its entirety to ensure there are no interruptions to business operations.

The test plan should be mapped to the full range of business processes that the ERP application will cover. Each functional area that is affected must be engaged in identifying the scope of the testing required. A risk assessment should also be included as part of the test plan and scripts, identifying the extent of sensitivity to failure.

Risk Mitigation in Testing

Effective planning and good documentation can mitigate the effort required for testing after an initial implementation. Having testing in mind at the planning stage of any project ensures that the testing phase covers the required functionality, preventing wasted time.

The test plan should be well documented and contain standardised scripts for completion. These scripts should be extensive enough that they cover the range of scenarios. A good testing phase will also utilise many types of tests, at many enterprise levels, across multiple scenarios. Wherever possible, automated testing can be leveraged to speed up the testing process.

A large factor in mitigating testing effort while maximising effectiveness is the engagement, experience and empowerment of the testing team. The team responsible for testing the ERP should be involved in the implementation plan during the discovery phase. They should have experience with the ERP application being deployed. Finally, they carry the responsibility to ensure appropriate sign-off points are in defined and managed, mapped to the business processes and application requirements. These quality assurance “gates” are the roadmap to ensure the configuration is flawless and effective. This team is the guiding hand that will guarantee the application deployment is successful.

The Testing Team is Key

The testing team should have necessary software testing experience with an understanding of the complete functional makeup of the ERP software being implemented. Where new ERP applications are being deployed, that experience may not exist, or resources may not be available within the organisation to support the project timeline.

The testing team should be engaged in all aspects of the project, starting with discovery. The key contributions at this stage are critical to the success of the project. This is the critical phase to ensure their test strategy is aligned to support the project.

The testing team should be tasked with ensuring the overall quality of the ERP deployment delivering several benefits such as:

  • Early detection of project risks
  • Creation of repeatable testing artefacts with Test Automation
  • Generation of organisational Testing Practice

Conclusion

Deploying an ERP system is a process that can be fraught with peril for an organisation. By recognising the key role that testing can play in success, an organisation can effectively mitigate the risks of project failure. Engaging a testing partner with ERP domain expertise will contribute to a successful ERP roll-out for your business.


Are you considering upgrading or replacing your company’s ERP system?

Whether you are considering moving to a SaaS, Cloud or Hybrid on-premise solution model to accommodate rapid change, Testpoint will help you to go-live quickly and successfully assuring your technology with the correct ERP testing strategy. We have the technical and industry expertise to make your implementation project a success. If you’re in the market for ERP, contact Testpoint for a free comprehensive Quality Assurance assessment to kick start your ERP testing requirements.

TestpointAssuring Your Technology For Tomorrow

GET IN TOUCH

 

Testing is often associated at times with being a complex function to execute and deliver successfully, which can result in organisations rushing or pushing testing until it is too late in the development process. However, setting up testing projects for success does not have to be complex.

It is vital that businesses understand the value of testing and the importance of enhancing the productivity aspect. Once businesses have the ability and intelligence to leverage off their testing assets and data, they can start making insightful gains in key areas of their quality assurance and software testing mindset.

Here are three (3) best practices to setting up testing projects for success.

  1. Creating a Testing Culture

Creating a testing culture within your business is a crucial step towards executing testing projects successfully. Everyone involved in product development should strive to understand the value, impact and benefits of testing and to realise the positive impact that it will have on efficiency.

So how do you create a testing culture?

  • Business and Technology leaders are in an ideal position to set the tone for how testing are going to be perceived within the organisation. Highlighting the value of testing in business and technology strategies is a key component to gaining collective buy in for testing. Testing is a quality assurance measure that is important at every step of the development process [ii] . Businesses that prioritise testing and make it a key part of their software development strategy have the capability to monitor the quality of their software throughout the development life cycle.
  • All team members involved in a development project should be privy to testing procedures. This means that testers should be involved in decision making surrounding product development. Open and fluid communication between testers and other teams associated with the software development is a key part of effective testing. Acquiring full information about a project will help testers make informed decisions in how to formulate the most appropriate test approach and strategy.
  • An important step that businesses can take to create a testing culture within their organisation is to interact with the end user[iii]. Involving the customer will help identify what gains can be made in the software development process and set the goals of performing the tests. Software testing is carried out for the benefit of both the end user and business objectives.
  1. Test Planning

Preparation helps prevent poor execution. A test plan is the playbook for all testing activities necessary to assure the quality of your project and also your communication point with stakeholders and project personnel. In order for a test plan to be effective, it needs to be continuously and proactively updated to account for and address evolving project scope and requirement changes. The test plan should be updated to reflect these amendments with clear expectations to deliver goals and outcomes.

So what should be included in your Test Plan?

  • Test Approach – The critical part of your test plan is the approach which answers the WHAT, WHY, HOW, WHO and WHEN you will be testing
  • Test Estimation – Understand the components and the features to be tested. (setting up a test data, test cases, reporting and reviews should be included)
  • Testing Scope – The kind of tests which should be delivered (i.e performance testing, cross browser/mobility, security testing)
  • Test Execution Process – The way you will manage and execute test cases (alignment to schedule) and process of capturing results, observations including raising failures and re-testing. see figure 1a below for an example test execution process automated using a testing platform (i.e Vansah QA or similar)
  • Resourcing – Allocating the right resources to support the testing process.
  • Remediation Defect Cycle – The application will require necessary time to fix specific issues which may impact your testing cycles.

  1. Enhancing Testing Efficiency and Effectiveness

Let’s understand what testing efficiency is. Testing efficiency is measured by understanding the average number of tests each resource or your team can execute per hour/day. The sooner you discover issues the better it is on your product development costs and time. This is why its important to discover how to test continuously and often against changes and across environments. Test automation is a capability to support your continuous testing to significantly improve your testing efficiency if implemented correctly.

Testing effectiveness is the rate at which your testing reveals issues before releasing your product to production and your end users.  It is not only important to run tests but equally important to know what type of tests you are running to support your testing process. see figure 2a below for an example real time dashboard to monitor your testing efficiency and effectiveness.

 

 


Testing is a crucial element of project management. Businesses that execute successful testing projects drive a testing culture, understand the value and impact of testing. Whether you are a small enterprise or a large organisation implementing a large project development Testpoint can help you achieve your business objectives with an array of testing strategies and solutions to assure your technology.

[i] Testpoint, www.testpoint.com.au

[ii] Testpoint, Quality Assurance Management, https://www.testpoint.com.au/solutions-services/quality-assurance/

[iii] Testpoint, The Importance of Customer Journey Modelling, (2017), https://testpoint.com.au/the-importance-of-customer-journey-modelling/

[iv] Kissmetrics Blog, ‘How Loading Time Affects your Bottom Line’, https://blog.kissmetrics.com/loading-time/

[v] Testpoint, Application Performance Management & Load Testing, https://www.testpoint.com.au/solutions-services/performance-load-testing/

[vi] Testpoint, Application Performance Management & Load Testing, https://www.testpoint.com.au/solutions-services/performance-load-testing/

 

The advancement of modern technology has meant that consumers have far more choice than ever before when it comes to utilising applications and technology. As a result, consumers are becoming increasingly selective of which applications they use.

The ability for your application to scale has become an absolutely vital component of any new development. An inability to scale with increasing workload can damage the brand reputation of an organisation and see potential customers turn away very quickly. Essentially, if your application cannot scale effectively with increasing workload, the success of your organisation will be impacted dramatically.

The ability for your application to scale has become an absolutely vital component of any new technology. Applications that fail under increasing workload will face a rapid backlash from end users as they look to the competition for alternative solutions. An inability to scale with increasing workload can damage the brand reputation of an organisation and see potential customers turn away very quickly. Essentially, if your application cannot scale effectively with increasing workload, your organisation’s profit margins may be affected.

This article goes into detail about the basics of scalability and how testing assists in understanding application performance. Application testing is a pivotal part of measuring performance and can provide organisations with peace of mind that applications can handle growth and scalability with aplomb. Testpoint specialise in application performance management and load testing with its Vansah technology platform to ensure that your software can always perform at an optimum level[i].

 Scalability testing

From the outside, many organisations view scalability testing as an intimidating task. However, testing an application for its ability to scale is actually a lot more straightforward than you may think. Essentially testers place an application under stress and establish where the limit of scalability is reached. Once the limit has been determined, it is important to find out why the limit has been reached, so an application can handle an increased workload.

From the testers point of view there are a number of things that can affect an application’s ability to scale:

  • Application response time
  • Resource consumption
  • Latency
  • Memory and network use
  • The frequency of requests that can be processed at any one time (throughput)
  • System availability
  • Task performance time

Testing for scalability is trying to understand how to maximise the performance of an application for end users. Testing these factors collectively and measuring their performance against a high workload provide an insight to an application’s scalability.

Why is scalability crucial?

Scalability refers to the ability of an application to respond to quick fluctuations of load, traffic, data and end users. Importantly, scalability also refers to how an application can handle an increase in these different aspects simultaneously. Often, in times of increased workload there is an increase in end users, data and traffic that places load on an application.

Scalability testing measures how your application will gauge against these fluctuations. Each application is different and testing must be tailored to suit the particular requirements of the application and the users that it serves. It is vitally important that organisations put steps in place that give your application the ability to scale efficiently during these increases. Scalability testing replicates spikes in workload and allows testers to measure the performance of an application under a variety of situations.

SMART Scalability testing

In an ideal world an application will run seamlessly for every user under all conditions. However, this utopia is not always possible to achieve. Therefore, it is important when it comes to scalability that SMART goals are identified early to help measure performance and understand what scalability will look like for your application:

  • Specific – What are the most important aspects of scalability for your application? Is it speed, ability to handle simultaneous requests?
  • Measureable – Once you understand what is vital to your scalability place measures around them. For example, define how many requests your application can handle at any one time, or define an ideal loading time for your application.
  • Achievable – It is important to set achievable targets for your application, that reflect the size of your application.
  • Realistic – The needs of your application may be as simple to handle a certain number users at once. Make sure they are realistic.
  • Time Frame – The most important aspect of scalability testing is to ensure that the application is scalable within your defined parameters before you release it to your customers. Often with applications, you only get one chance to prove the worth of your produc

Incremental testing

Once you understand the parameters of testing for scalability, it is also vitally important to test incrementally. If you have a new application it is very difficult to ascertain how many users may be using your application simultaneously.  Incremental testing will help you safeguard your application from problems exacerbating if the workload increases exponentially. Incremental testing places the application under strain with different loads. A small load test should give an indication of how an application performs normally.

Increasing the load incrementally then gives an indication of how the application performs against this normal base measure. The first sign of issues will pave the way for scalability solutions. Often different amounts of duress create different scalability issues. Testing at all levels of loading will allow you to be across all possible problems you may encounter, so that your application can perform well during peak times.

Horizontal or Vertical Scaling?

It is important to understand the type of scalability you would like your application to achieve. There are two distinct ways to scale an application:

  • Horizontal Scaling – Horizontal scaling, or scaling out, is the process of broadening the capacity of an application by adding additional components. For example, if you extend your capacity by adding an additional server you are scaling horizontally. Traditionally scaling out is an excellent way to manage high workloads, but will require more maintenance as testers need to be across more working parts.
  • Vertical Scaling – Vertical scaling, or scaling up, is the process of increasing the capacity of existing components. For example, if you replace your existing server with a more modern version that can handle a much higher workload. Scaling up is easier to manage as there is less working parts, however depending on the size of your application and the workload it needs to manage, at some stage it will be necessary to scale out as well.

How Scalability testing assists in understanding performance

An application’s loading time can make or break its success. Consumers now expect an application to load within two seconds[ii]. Scalability testing throughout the application’s production lifecycle can help testers and developers create application architecture that can handle a growing workload.

All businesses want their organisations to grow and with that is an expectation that their applications will handle this growth. As the workloads increase, the complexity of scalability also increases. Testing at high workloads creates an understanding of how an application can operate under duress and allows testers and developers to scale components of the application, either horizontally or vertically, for optimum performance. Scalability testing allows testers to maximise performance and tailor capacity to handle demand[iii].

Create a Scalability Checklist

It is important to understand how the testing process can assist in understanding performance. By having a clearly defined scalability checklist it is possible to get the most out of your performance testing:

  1. Seek guidance from business leaders – Often your organisation will have targets for the application. Seek out what these targets are and set up your tests to handle this workload as a base figure.
  2. Understand your SMART standards – Each application is different and requires its own standards of scalability testing.
  3. Have consistent testing procedure – Consistency and repetition creates accuracy for scalability testing throughout the entire application development.
  4. Know what tools you need – Understand what testing tools are required to run scalability tests for your application.
  5. Planning your test scenarios – Do you know the testing range and loading strategy?
  6. Understand the testing environment – Is your scalability testing simulating the production environment under different loads?
  7. Create and corroborate testing scripts – Make sure your visual scripts align to your testing plan.
  8. Load testing scenarios – Now you know the criteria you need for scalability testing you can load the testing scenarios for different workload increments.
  9. Complete the load tests – Find out how the application performs under all situations.
  10. Review – Explore the results and examine where improvements can be made and generate reports and recommendations.

 Outsourcing your scalability testing

Creating an application is one thing. However, making an application scalable is an entirely different proposition. Improving and increasing scalability can soak up a lot of resources and can become a costly exercise without the right level of expertise. If your organisation is light on experience it is a great idea to outsource your scalability testing. This is particularly pertinent, when you are unsure about elements of scalability testing or you are unsure about the workload your application may have to handle.

Testpoint specialise in Scalability/Load testing to help you and your organisation create applications that can perform optimally all the time. Making your application scalable under increasing workload is crucial to maintain brand reputation and ensure that end users are satisfied with the speed and efficiency of your applications.

[i] Testpoint, Software Testing Services, https://www.testpoint.com.au/solutions-services/software-testing/

[ii] Romexsoft, ‘How to increase the scalability of a web application’, (2017), https://www.romexsoft.com/blog/improve-scalability/

[iii] Testpoint, ‘The Benefits of Performance Testing’, (2017), https://testpoint.com.au/the-benefits-of-performance-testing/

GET IN TOUCH

It is absolutely essential that organisations are catering their software developments effectively for end users. Businesses must understand the performance and availability of their applications.  A successful user experience results in satisfied customers and can lead to an increase in growth and market share. Increasingly, the first impression of any software deployment has on end users is critical for determining its success and impacts on brand reputation and integrity. Effective software performance and providing an excellent user experience can provide a sustained source of competitive advantage and attract new customers that competitors find difficult to emulate. This is especially true considering that customers now interact with businesses online at all times of the day and night.

Conversely, organisations that do not measure the performance of their software effectively, risk providing an underwhelming customer experience and can lose out on revenue and potential customers.

In this article will review how Testpoint manages and supports the end user experience, performance and availability of its customer’s applications, utilising Vansah Synthetic Application Monitoring (VSAM) platform.

Businesses face many challenges in monitoring applications that can lead to frustration and financial loss. Without effective monitoring it has hard to know the end users experience and understand when issues are arising throughout the customer journey.  Here at Testpoint, businesses frequently approach us to understand how they can obtain detailed performance, user experience analytics and find the root cause of potential issues.

Businesses also find it difficult to understand how changes and upgrades to technology are influencing the overall user experience.  Furthermore, there are frequent challenges in understanding how applications are performing across browsers and devices. Ineffective testing and monitoring can also lead to security issues that have the potential to cripple applications and the business.

Testpoint’s proprietary enterprise VSAM platform helps businesses attain peak performance for their software throughout the user experience.

VSAM is a managed service delivered by Testpoint that provides solutions for businesses to help them understand and identify end user experience and performance issues earlier in the development cycle and across live production systems. Live application monitoring enhances the service by monitoring the end-user experience in real time. As a result, by utilising the VSAM platform, your organisation is ensuring the best experience and performance of your systems. Testpoint’s VSAM managed service will dramatically reduce operational costs and at the same time will increase consumer and stakeholder confidence when going to the market with new technologies.

What are the advantages of Testpoint’s managed VSAM services

Increasing the effectiveness and efficiency of your testing will increase operational performance. The VSAM platform, delivered by Testpoint, provides an automated process for monitoring the user journey and applications performance in real time. The creation of real time response rates provides up to date reports for your business application team. VSAM provides your organisation with several important insights and analytics that support real issues:

  • Measure the end user experience across technology – Testpoint’s VSAM platform and managed service is one of the few technology solutions that can measuring the end-user experience from both inside and outside of the firewall and across any technology.
  • Create real insights and visibility – Our approach is based on synthetic (agent-based) monitoring, to simulate any type of business transactions necessary to monitor and analyse response times, quality of service and availability across these transactions. The result is that VSAM can consistently deliver real insights and visibility.
  • Troubleshooting – In addition to capabilities for simply monitoring the end-user experience and alerting IT teams when a problem occurs, Testpoint provides robust capabilities for troubleshooting and resolving performance issues.
  • Identify the root cause of the problem – The combination of proactive monitoring across different applications and infrastructure joined with Testpoint’s managed services ensures we are identifying the root cause of the problem and valid issues are being escalated proactively. Early detection of problems means that solutions can be implemented earlier, whilst problems can still be managed simply, from a cost and operational perspective.

Our VSAM managed service will also help your organisation with operational efficiency. Research conducted by Gartner has identified that each hour of downtime can cost an organisation, in some circumstances, over $300,000.[i] Excessive downtime can have serious operational consequences for an organisation. Consumers now expect a certain level of speed when they utilise your applications. A lack of speed and a delay in loading times and operation will cause your customers to turn away from using your platform. Furthermore, a lack of speed can affect a businesses integrity and brand reputation.

Slow loading time is a major reason for deserting pages; A single second user delay produces a 7% loss in conversions and a 16% decrease in customer satisfaction for each second.[ii] For most organisations, customers are their biggest voice for championing their brand. Therefore, it is absolutely critical that their end-user experience is a reflection of how an organisation wants to be portrayed. A fast, efficient loading time shows to customers that your organisation is set on providing a quality customer experience.

Over the last few years in particular, there has been an increasing complexity in the data centre and application architectures. Technology is shifting towards the cloud and SaaS models swiftly. It further stresses the need to monitor vendor SLAs and organisations need to ensure that you are paying for what you were promised. More than that, your brand reputation is at stake and you need to ensure that your customers’ end-user experience is not impacted.

How do you monitor application end user experience across your organization?

Testpoint have created Synthetic Application Monitoring with Vansah to help organisations carry out efficient and effective monitoring. Testpoint carry out VSAM managed services with a focus on helping organisations understand the experience of the end user and drive the performance and availability of their applications. Identifying issues early in the development cycle provides the ability to respond quickly and efficiently to issues creating a high level of operational agility and reducing costs of deployment and execution. An effective end user experience can help your organisation enhance your brand reputation and online integrity.

 

[i] Oakley, J, ‘The Real Cost of Downtime For Business’, 100TB, (2017), https://blog.100tb.com/the-real-cost-of-downtime-for-business

[ii] Dooley, R, ‘Don’t Let a Slow Website Kill Your Bottom Line’, Forbes, https://www.forbes.com/sites/rogerdooley/2012/12/04/fast-sites/#7d3c03f353cf


Vansah VSAM™ synthetic monitoring allows you to proactively monitor your web, non-web, legacy, mobile, streaming and cloud-based applications in staging & production environments to ensure performance will meet your end user requirements.

GET IN TOUCH