Test Automation Framework for Accelerating Product Testing and Verification

about project

The client needed a solution to automate the testing and verification process for new products. The project involved developing a Test Automation Framework (TAF) to create and execute test scenarios for desktop, web, and mobile applications.

‍

The framework required capabilities for generating detailed test reports, saving logs, screenshots, video recordings, and integration into the CI/CD pipeline. It was also designed to meet IEC 62304 standards and synchronize with the Requirements Management System for full traceability.

‍

Client aimed to standardize the testing process across all products, improve product quality, and reduce testing costs by moving away from third-party tools. By automating testing, TAF helped streamline verification, enhance efficiency, and lower manual testing efforts.

Tasks

Automate Software and Hardware Testing: Develop an automated solution for testing software and hardware during the verification and certification process.
Develop a Comprehensive Test Automation Framework:
- Generate detailed test execution reports.
- Save logs, screenshots, and video recordings of the test executions.
- Ensure the reports align with IEC 62304 requirements.
- Enable test results to be integrated with Requirements Management System.
- Implement image recognition to compare test results with initial templates.
Standardize the Testing Process: Build a framework that can be used across all Tecan products, simplifying maintenance, reducing testing costs, and accelerating the development of test cases.

‍

Results

Improved Testing Efficiency: The framework streamlined the testing process, resulting in faster and more reliable software testing.
Accelerated Product Release: By reducing testing time, the framework enabled quicker market launches for new products.
Reduced Verification Costs: The automation of testing and documentation significantly lowered the costs associated with the verification process, particularly in compliance with IEC 62304.
Standardized Test Automation: The platform provided Tecan with a unified test automation process, enabling faster and more efficient testing across all teams.

‍

Tech Stack and Standards

Technologies:
- Selenium, Web Stack, Electron, WebApi
- C#, MEF, Appium, WinAppDriver
- UWP, WPF, WinForms
- BasicAuth, Jwt, Ntlm, OAuth2
Domain Knowledge: Robotics and life science, Highload, Layered Modular Architecture
Standards: IEC 62304

‍

process

Project Implementation

Initial Request and Requirement Gathering

The project started with gathering the requirements for automating both desktop and web applications for Tecan, ensuring compatibility with their internal systems and compliance with industry standards.

Evaluation and Planning

Client aimed to create a unified testing framework to replace 3rd party testing solution. The decision was made to build a custom framework that would meet the needs of their complex applications and internal standards. The planning phase involved identifying the gaps in existing solutions and designing the foundation of TAF.

Requirement Specification Development

The team developed specifications for the framework, including features like test report generation, logging, and integration with Requirement Management System.

Development

- The project started with building an MVP for automating desktop applications. This was later extended to web applications, API testing, and mobile app testing.

- Initially positioned as a desktop solution, the framework evolved with features added based on testing needs. The first application tested on TAF was a simple client’s desktop application, which helped validate the framework's core functionalities.

- The framework was built on C#, Selenium, WinAppDriver, MEF, Appium, UWP, and WPF. Over time, as client expanded, additional teams adopted the framework, which was continuously expanded with new features.

- Integration with Requirement Management System allowed test runs to be linked with requirement IDs for traceability, and test results were automatically synced with the requirements system.

- Due to Microsoft's lack of support for WinAppDriver, the framework was later adapted to FlyUI.

‍

Test Coverage and Automation

The system was initially validated with manual tests and then gradually automated as the framework's capabilities expanded. By automating the test processes, the platform ensured broad coverage and reduced manual testing efforts.

Documentation Development

Initially, documentation was manually maintained, but the framework eventually evolved to support automated documentation generation, reducing the manual workload for documenting test cases and results.

Release Candidate Preparation

The development team prepared a candidate release for the framework and began testing it across multiple applications.

‍

Verification Process

Rigorous verification was carried out to ensure the framework met all of client's needs, including the generation of automated reports, logging, and successful integration with the Requirement Management System.

Validation and Final Release

Final validation was conducted, ensuring the framework was capable of handling complex test scenarios, from desktop applications to mobile apps, with full automation capabilities.

Final Release and Support

After a successful release, the framework continued to be supported and improved based on user feedback and emerging needs.

review