Get up to speed fast on the techniques behind successful enterprise application development, QA testing and software delivery from leading practitioners.
How software testers can demonstrate value to the business
How DevOps teams are using—and abusing—DORA metrics
How to improve your observability systems
What people don’t get about value stream management
Usability: Where software testing tools fall short
Trends and best practices for provisioning, deploying, monitoring and managing enterprise IT systems. Understand challenges and best practices for ITOM, hybrid IT, ITSM and more.
The new IT services model: Why you need to get product-centric
The 8 flavors of serverless: How to choose wisely
5 steps to becoming a data-sharing master
How AIOps is a game-changer for predictive analytics and CloudOps
The state of IT operations management: 6 trends to watch
All things security for software engineering, DevOps, and IT Ops teams. Stay out front on application security, information security and data security.
5 infrastructure security tasks your developers can automate
What you need to know about KVKK data-privacy requirements
Transform your security approach: 7 ways to shift to cyber resilience
3 best practices for locking down your hybrid cloud security approach
Let’s fight cybercrime like we did piracy in the 18th century
TechBeacon Guides are collections of stories on topics relevant to technology practitioners.
TechBeacon Guide: The State of SecOps 2021
TechBeacon Guide: Application Security Testing
TechBeacon Guide: Data Masking for Privacy and Security
TechBeacon Guide: Cloud Security & Data Privacy
TechBeacon Guide: Unstructured Data Security
Discover and register for the best 2021 tech conferences and webinars for app dev & testing, DevOps, enterprise IT and security.
DevOps World 2021
SKILup Days: 2021 – Observability
Webinar: Threat Hunting—Stories from the Trenches
Webinar: Cybersecurity Executive Order Challenges and Strategies
Webinar: Data Privacy and CIAM—Complete Your Identity Stack
Tools to support and automate parts of software testing activities are important, even essential in some areas, and yet they also can be problematic. I’ve seen teams struggle with today’s testing tools and have wondered why: Is this a problem with the testers’ skill sets, or with the usability of the tools themselves?
I’ve worked in the software industry for decades, and as part of a research project as a PhD student at the University of Malta, I asked testers about their experiences with testing tools and the concerns and challenges they face when implementing and using them. I collected data through surveys, workshops, and interviews with over 100 testers from across the globe, amassing an enormous number of stories about their experiences, good and bad, with testing tools and automation.
As it turns out, testers say usability issues are affecting success with testing tools. Here’s what else I learned—a sneak peek before my in-depth presentation at the upcoming EuroSTAR conference.
As a software tester, the information that you supply to decision makers and your colleagues must be useful, timely, and accurate. Delivering software on time that then fails is not a good outcome, but neither is delivering software late. The use of tools to support testing makes sense, but while tools that aid in testing have been around for decades, so have problems with their acquisition and use.
In the 1980s, I worked on a project where we used tools to execute tests. The tools allowed us to deliver on time and with enhanced certainty about software quality. The software, tests, and tools grew together. But over the decades, I noticed that successful tool use is unusual. While more tools are available today, testers complain that their tools have not been easy to implement and use over time.
As the IT industry has grown and software has become ubiquitous via the Internet and mobile technologies, the work of testers has become more pressured. We’re doing more, with a requirement to deliver more quickly. Tools to support all aspects of testing have become even more important, in terms of both the scope and the speed of the testing. However, implementing tools and using them successfully remains problematic.
Issues include lack of management support, misconceptions and over-optimism about what tools can achieve, and technical problems with tools. These problems are well documented, but still ubiquitous. Additionally, there’s a push to change testers’ skill sets to include coding, in order to support scripting in test tools.
The first result is rather depressing. In all the time our industry has been attempting to adopt and improve tool support—for all the presentations, tutorials, webinars, books, and discussions we’ve shared—we are still making the same old mistakes.
Some 82 of the 111 participants surveyed raised at least one issue with their tools. The table below shows the percentage of comments shared about three main topic groupings. Management decision making continues to be important, with challenges, blockers, and enablers identified in recruitment errors, lack of training support, unreasonable expectations, and poorly defined targets.
Technical blockers abound, including installation problems, test and tool maintenance problems, IT security preventing progress, and tools that are mutually incompatible, blocking a stream of test activity.
Usability is indeed an issue: Testers said they want greater usability to be built into the design of their tools, and 37% of all issues raised were usability-related.
A second finding was that testers had very emotional responses to many of the survey questions, and this was directly related to their experiences with tools and automation. Testers said they feel “stuck” because “tools are promoted as ‘magical solutions‘” and also they feel “scared” and “frustrated” but also “quietly proud” of what they do.
Two participants reported that they were required to use a particular test tool, but because of organizational security policies and the technical configuration of the test environment, they were unable to gain access to the tool, leading to unnecessary frustration, anger, and demotivation. Two people used the word magical to describe management expectations of tool implementation—reduced team sizes, costs, and time scales—while other testers spoke of fears and feelings of isolation and worthlessness.
Usability is a focus for tool designers and vendors, but even so, testers reported problems. Usability issues are not a matter of complete neglect, but in some cases the result of a superficial approach to usability. Here are five key issues:
Taken together, this means that when you acquire tools you can fall into an “illusion of usability.” This is where you think a tool will be useful and easy to use and will solve your testing problems, only to be disappointed when it does not help you complete your work because of one or more of these five issues.
Usability is a vital aspect of a testing tool’s success, but it’s not trivial to achieve during tool design. Testers come from a wide range of backgrounds and experience, so it’s not surprising that the tools are not a good fit for everyone.
But how can this be resolved? To help your own tool implementation be more successful, use already-published materials such as the Test Automation Patterns website to help you overcome management and technical blockers to success.
Want to know more? Pick up a copy of D. Graham and M. Fewster’s “Experiences of Test Automation: Case Studies of Software Test Automation” or K. Wiklund’, “Impediments for automated software test execution: A systematic literature review.”
And don’t miss my EuroSTAR conference session, “Who Are We? What Are We Doing? How Are We Doing It?” where I’ll talk more about my results and my research, what it means for testers, and how you can contact me to participate in the next phase about what can be done to improve tool usability design. The conference runs from September 28 to September 30, 2021. TechBeacon readers can save 10% on individual tickets using code “TechBeacon10” until Sept 6.
Keep up with QA’s evolution with the World Quality Report 2020-21.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon’s Buyer’s Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon’s Guide.
Get the best of TechBeacon, from App Dev & Testing to Security, delivered weekly.
Brought to you by
I’d like to receive emails from TechBeacon and Micro Focus to stay up-to-date on products, services, education, research, news, events, and promotions.
Check your email for the latest from TechBeacon.