What if Doctors ran alongside super-fast Athletes? Barefeet.

Pratima H

VIENNA, AUSTRIA: Speed is into a spotlight - again.

If some numbers are hinting it well, speed has shaken up the software space so much that new software is easily being churned out at a rate way faster than it can be effectively tested. If we look at 2016, software failures have been panting hard with losses of $1.1 trillion, and that touched as many as 4.4 billion people and over 50 per cent of the world’s population.

Experts toss this on aging software testing processes and tools, which are possibly not in a state to match tracks with the demand for innovative software. If conventional software testing is failing on endurance, a new alternative of Exploratory Testing, with advanced testing practices in line with modern development methodologies like Agile and DevOps, is overtaking with many breakthroughs and sprints.

Wayne Ariola, Chief Marketing Officer at Tricentis is so confident about Exploratory Testing being a sound testing approach that he feels it not only validates software changes, but also helps Agile teams rapidly and reliably deliver comprehensive feedback. Higher risk coverage at significantly faster speeds is what stands out for Wayne.

No wonder, Tricentis is bringing out what it calls the first Exploratory Testing tools built natively for Atlassian JIRA, a leading agile development planning tool with $457.1 million in annual revenue. The idea is to simplify the planning, documentation, and reporting of exploratory testing directly within the JIRA environment—and allow full traceability between exploratory test results and the associated JIRA issues.

This seems to align well with the trend of many development teams shifting their software development processes to become more agile and iterative; and hence the need for a fundamental change in the way we test software - the company believes.

But just how sturdy, stamina-packed, documentation-savvy, coverage-friendly, error-sharp, efficient and easy is this new escort? Can Exploratory Testing withstand the brutal forces at work on the real ground while also fixing sprained knees and unexpected clots in time? More importantly, why or why not, is it an adversary to the old-school way of testing? Let’s ask Wayne.

What are the major failure areas where erstwhile testing and QA is slipping up?

Expectations for software delivery have shifted significantly. The business expects more frequent releases of features that provide competitive differentiation. This shift in delivery cycles impacts testing significantly. Yesterday’s software testing tools and processes don’t fit today’s business needs. This is the main challenge that testers face today.

Exploratory Testing (ET) complements the new pace of software delivery by giving organisations the ability to inject testing cycles into accelerated release cycles. It enables any role within the organisation to contribute to the verification or validation of changes in the application. As soon as new functionality is completed, any role (product owner, technical writer, business analyst, tech support, UX designer, etc.) can instantly start testing by simply accepting a testing invitation.

Wayne Ariola, Tricentis

How about tackling the error-side?

ET exposes the following types of errors that evade traditional specification-based testing:

• Usability issues (e.g. confusing interfaces, inconsistent usage patterns).
• Missing requirements (i.e. functionality that is critical for the end user experience, but was not specified).
• Problems with functionality that was implemented beyond the boundaries of specification (and therefore not covered by specification-based tests).

But how is Exploratory testing quintessentially different from conventional scripted testing? Is it about the tools, the mindset, the process, the test goals or the context? What if people confuse it with ad-hoc testing?

ET represents a new way to achieve software quality. It is a new approach to the process, it’s a new tool that enables the coordination of quality processes, and it allows broader participation from team members beyond QA.

Scripted testing focuses on bottom-up verification of whether the acceptance criteria outlined in the requirements/user story are met. It does not necessarily take into account the user experience. ET is a great combination of both. Using exploratory testing, the traditional bottom-up verification of requirements is supplemented with a top-down assessment in an effort to protect the user experience across frequent releases.

The charter can express the goal of exercising a specific component of the application. However, since it’s exploratory and lightly-structured, testers have the ability to exercise the scope of the charter through the eyes of the end user.

Studies show that 80 per cent of testing today is essentially ad-hoc. ET is distinctly different, since it outlines a distinct goal and provides a focused approach versus ad hoc testing.

Is it adequate for customer-facing products or products with a regulatory ingredient? How good is it on test coverage, ramp-up errors and overall efficiency?

Tricentis ET makes ET appropriate for organisations with compliance-driven reporting requirements. We record and capture each tester’s actions, so all test paths are documented comprehensive and accurately—without requiring any additional effort by the tester. Documentation is one of the greatest challenges with compliance. We handle documentation automatically. Additionally, ET results can be integrated into a reporting platform that centralised results from automated, manual, and exploratory testing.

Test coverage metrics can be calculated as tests are conducted, and correlated to defects discovered. Also, Tricentis Exploratory Testing helps users use exploratory testing to increase coverage around their existing manual and/or automated test cases.

Can this be amenable for enterprises that are leaning towards app-economy, rapid app cycles, Devops, NoOps, Dockers etc?

Absolutely, the flexibility associated with ET essential in these cases. Exploratory testing is vital for accelerating test cycles and keeping pace with release expectations

Is the new software-breed better for disposable/short-life-cycle apps instead of the ones that ISVs go by or those with long shelf-lives?

ET is well- aligned to the new breed of applications that are evolving rapidly. Modern application, whether disposable 'short-life apps' or apps leveraging micro-services, require more rapid and iterative testing practices that are closely aligned with the end user. ET takes the perspective of the end user as the baseline for the end user experience. This cannot be achieved with specification-based testing.

So this new genre of testing can actually align well with documentation, bug-reporting, reviews, scenario-testing etc strongly enough while cutting down on arbitrary component?

Yes, with Tricentis ET documentation is automatically generated as testers explore the application. Multiple 'record' functions allow users to capture everything they explore, resulting in editable screenshots and videos for comprehensive documentation. This makes it simple for developers to reproduce reported issues—and it helps QA accurately verify their fix.

Can both kinds of testing be complements instead of alternatives? Does the explorative option reduce the distance between developers and testers as well as the interstitial fragmentation therein?

We believe that ET is a replacement for manual testing because it accommodates the user experience, it initiates the test cycle iteratively as part of the delivery process, and it includes multiple roles within (and beyond) cross-functional teams.