Myths of Script less automation

 

Before we discuss about Scriptless automation, we want the Automation challenges to be understood in broader view. When testing industry has caught the opensource wave, Selenium supported by Google has gained prominence. Automation with Selenium has provided engineers powerful capabilities with both forward and backwards integration with Unit testing and BDD plugins. How ever the scripts took time to Develop and changes in application needed scripts to be updated.

 To decrease amount of time spent in Automation development and maintenance work, the commercial tool vendors have re-oriented their offerings towards ‘Record and Play back’ activities that are easy for Business Analysts to create and move towards “Script less automation”. Tools like Tosca, Accel Q, UI path ( Robotic Process Automation), UFT etc, are successful at various levels. I want to demyth some facts about scrip less automation.

Myths about Script less testing	Reality	POV
There is No script, everyone can create automation	User friendly framework for Naive users. However, Automation needs to grow organically from within your environment integrating Business and Operational logic step by step until it gets to a point where no further scripting is needed	For Hybrid Objects and Lengthy E2E Automation scenarios, you will still need to script few areas. Complex / Hybrid integrations need careful Framework   architecture.
Everything can be recorded and Played back	The user will have to depend on recorded scripts that were initially created with test data and not with actual live scenario data.	For Highly Dynamic data, where workflows changes with data inputs, you need to intervene and customize
There will be No Test Maintenance	As more functionality gets added and as more functionality is fine-tuned. The tool will not know what is updated and will identify new changes as bugs and tests will fail.	With no scripts too, you need to know what changed at feature level and at workflow level to make changes to automation.
Changes are not needed for scripts. For region specific workflows, tool will automatically update	As complexity is shifted to configurations, you need thorough understanding of configurations and it will take time to change configurations. Wrong and poor application of configurations can lead to test failure	In traditional scripting, you need to spend considerable time in updating code. However, this is reduced as time is spent on configurations.
Automation architects are not needed	Scripless  is in no way implies the absence of scripting. It is an optimized process of creating a testing framework that allows your testers to develop new test cases with reusable scripts.	As business or operational scope expands, need for new components will rise and automation architects are needed to integrate them.

 

While commercial script less tools can reduce some of the challenges even in scrip less automation testing you still need to develop automation and apply best practices like.

1. -          Modularity
2. -          Have right Structure and create core components
3. -          Identify and segregate Business functions, Data and configurations so maintenance is easy
4. -          Engauge Business users early and frequently in Development.

Complete test approach and tools in QA driven Devops model

It is indeed quite interesting to implement tetsing stages into CI? CD pipeline for products deployments like Gmail. This obviously needs a team effort between developer- Devops team and Tet team. If you ask most DevOps experts what goes into a Continuous Integration or Continuous Delivery chain, they’ll mention components like CI servers and code repositories. They’re less likely to discuss automated testing tools, despite the fact that automated testing is just as crucial in order to achieve complete CI/CD.
How ever the key to building quality into our software is making sure we can get fast feedback on the impact of changes. And automation testing is most mature way to handle testing for CICD pipeline.
First and foremost you have to design/ segregate tests appropriately
1. Automated: The idea is to automate as early as possible and as much as possible into.
-Unit test, ( API test)
-Component tests
-System tests
- Functional acceptance test
2. Manual Tests:
- Shocase test- non responsive Ui tests
-Usability tests
-Exploratiory/ negative testing
3.Non Funcation tests
- Performance ( with both Automated & manual)
- Security ( with both Automated & manual)
Once we have right tests planned... we create a deployment pipeline (the key pattern in continuous delivery). In the deployment pipeline pattern, every change runs a build that a) creates packages that can be deployed to any environment and b) runs unit tests (and possibly other tasks such as static analysis), giving feedback to developers in the space of a few minutes. Packages that pass this set of tests have more comprehensive automated acceptance tests run against them. Once we have packages that pass all the automated tests, they are available for self-service deployment to other environments for activities such as exploratory testing, usability testing, and ultimately release. Complex products and services may have sophisticated deployment pipelines; a simple, linear pipeline is shown below:
( color coding done to show issues/ pas at each stage)


Stage 1. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  -->Feedback ( Loop) to DELIVERY TEAM
Stage 2. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  --> Trigger-->AUTO ACCEPTANCE Test-->issues Feedback ( Loop) to DELIVERY TEAM
Stage 3. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  --> Trigger-->AUTO ACCEPTANCE Test-->Approval -->MANUAL VALIDATIONS-->issues Feedback ( Loop) to DELIVERY TEAM
Stage 4. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  --> Trigger-->AUTO ACCEPTANCE Test-->Approval -->MANUAL VALIDATIONS-->Approval-->RELEASE--> Feedback to project team on what value we created.


This way  we can go on having additional test stages with as many layers of testing from early code commit starting with Unit tests, .. and so on. we can also have stages for sonar cube, early component wise , Junit tests etc. as well as having component leavel performance test scripts triggered also.
The following are tools I would suggest for the entire cycle. Depending on actual implementation, skill set of resources and technology actually used internal to gmail we can prefer one over the other


1. Planning: Jira, Rally, MPP etc.
2. CI/CD platforms: Jenkins, Travis, Circle CI, GIT Lab etc.
3. Testing:

Unit tests: TestNG, N Unit , J Unit
Static code Analysis: PMD, Sonar Cube
API tests: Jmeter, Postman, SOAP UI, Rest Assured, Red Sharp, Test Complete, Tosca, UFT,
Frameworks: ATDD, BDD, Hybrid, Keyword drivem, Modular
Web browser/ UI based: QTP/UFT, Selenium webdriver, Protratcor, CodedUI, Egg Plant, Quish, test Complete
Performance test: Jmeter, Load runner, Blaze meter
Mobile test: Appium, Eggplant, Ranorex, Solenoid, test complet, UFT