One of our first customers using the testing as a service was a start-up company that created an accounting software package. For their new version they wanted to have a professional take on testing, and they wanted to utilize our testing as service solution.
Their choice of plan was the release testing solution and an extra functional testing package covering a new feature.
Functional Testing
The new feature they created was a search functionality. At the start of our testing we had meetings with the product owner/analyst on what was expected from the feature.
Based on the input and the provided documentation we had a good view on what was expected of this feature. The feature itself could be split up in 2 big pieces, the frontend GUI for showing the result and the backend API that was used for the searching mechanics.
For the backend API we used postman to do our rest calls and inspect our data. We created testcases based on the provided search possibilities, to make our positive cases.
But also have several negative testcases like API calls without credentials, incorrect search queries and incorrect data given.
For the GUI part our cases were based on the searching methods that we could re-use from the API testcases. Also, we added negative testcases that would enter unsearchable queries.
Next to these cases we also did load testing to check the performance of the application under stress.
Release Testing
For the release testing we did a further analysis of the documentation and had feedback from the product owner and the company’s training videos on the usage of the application.
With all this information we could create several user simulations to test the application as a user would use it.
Next to this user simulation, we touched every page of the application, searching for objects and buttons. With these objects we would determine which actions should be executed when pressed. With this we could find deadlinks and incorrect attached buttons.
While using the application we could also check the UI on possible issues or updates that could be taken to improve the user experience.
Reporting
After we did the testing of the application, we created a detailed report with our findings and bugs.
- Functional testing
- Issues with the load tests, when testing the application with the provided limits, we found that the application slowed down. As we noticed this behaviour we added more and found that the application was reacting too slow to be useful. We reported this bug immediately as this was a critical issue. This also resulted in a delay of the launch of the new feature.
- The other issue we found was that we could break the application when using special signs in the application. This we reported as a blocking issue.
- We found some minor bugs also and we gave as advice this feature should be fixed first before releasing it.
- Release testing
- For the use simulation we did not find any bugs in the main features of the application.
- We did find 2 broken links and a button that did not respond.
- We did have some remarks on the user experience as some of the error messages weren’t clear, so we gave the remark to update these and provided some options they could use.
