Complete test approach and tools in QA driven Devops model

It is indeed quite interesting to implement tetsing stages into CI? CD pipeline for products deployments like Gmail. This obviously needs a team effort between developer- Devops team and Tet team. If you ask most DevOps experts what goes into a Continuous Integration or Continuous Delivery chain, they’ll mention components like CI servers and code repositories. They’re less likely to discuss automated testing tools, despite the fact that automated testing is just as crucial in order to achieve complete CI/CD.
How ever the key to building quality into our software is making sure we can get fast feedback on the impact of changes. And automation testing is most mature way to handle testing for CICD pipeline.
First and foremost you have to design/ segregate tests appropriately
1. Automated: The idea is to automate as early as possible and as much as possible into.
-Unit test, ( API test)
-Component tests
-System tests
- Functional acceptance test
2. Manual Tests:
- Shocase test- non responsive Ui tests
-Usability tests
-Exploratiory/ negative testing
3.Non Funcation tests
- Performance ( with both Automated & manual)
- Security ( with both Automated & manual)
Once we have right tests planned... we create a deployment pipeline (the key pattern in continuous delivery). In the deployment pipeline pattern, every change runs a build that a) creates packages that can be deployed to any environment and b) runs unit tests (and possibly other tasks such as static analysis), giving feedback to developers in the space of a few minutes. Packages that pass this set of tests have more comprehensive automated acceptance tests run against them. Once we have packages that pass all the automated tests, they are available for self-service deployment to other environments for activities such as exploratory testing, usability testing, and ultimately release. Complex products and services may have sophisticated deployment pipelines; a simple, linear pipeline is shown below:
( color coding done to show issues/ pas at each stage)


Stage 1. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  -->Feedback ( Loop) to DELIVERY TEAM
Stage 2. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  --> Trigger-->AUTO ACCEPTANCE Test-->issues Feedback ( Loop) to DELIVERY TEAM
Stage 3. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  --> Trigger-->AUTO ACCEPTANCE Test-->Approval -->MANUAL VALIDATIONS-->issues Feedback ( Loop) to DELIVERY TEAM
Stage 4. DELIVERY TEAM-->Checksin code-->VERSION CONTROL-->Trigger-->COMMIT STAGE  --> Trigger-->AUTO ACCEPTANCE Test-->Approval -->MANUAL VALIDATIONS-->Approval-->RELEASE--> Feedback to project team on what value we created.


This way  we can go on having additional test stages with as many layers of testing from early code commit starting with Unit tests, .. and so on. we can also have stages for sonar cube, early component wise , Junit tests etc. as well as having component leavel performance test scripts triggered also.
The following are tools I would suggest for the entire cycle. Depending on actual implementation, skill set of resources and technology actually used internal to gmail we can prefer one over the other


1. Planning: Jira, Rally, MPP etc.
2. CI/CD platforms: Jenkins, Travis, Circle CI, GIT Lab etc.
3. Testing:

Unit tests: TestNG, N Unit , J Unit
Static code Analysis: PMD, Sonar Cube
API tests: Jmeter, Postman, SOAP UI, Rest Assured, Red Sharp, Test Complete, Tosca, UFT,
Frameworks: ATDD, BDD, Hybrid, Keyword drivem, Modular
Web browser/ UI based: QTP/UFT, Selenium webdriver, Protratcor, CodedUI, Egg Plant, Quish, test Complete
Performance test: Jmeter, Load runner, Blaze meter
Mobile test: Appium, Eggplant, Ranorex, Solenoid, test complet, UFT

How to move from traditional engagement ( T&M) model to more mature models

It takes a delicate balance to set up a sourcing partnership that is mutually beneficial to buyers and providers of IT services. Lasting win-win partnerships flourish only when both sides are satisfied. Many organizations are seeking sourcing arrangements that align buyer and provider incentives and foster collaborative co-creation. Riding this wave, managed services agreements using output- and outcome-based commercial pricing constructs are gaining traction as the sourcing paradigm of the future.

In this article I share a  two tone approach that I would like to initiate to migrate from T&M model to managed services model.

1. Pre Conditioning the customer

2.  Internal assessment and identification of opportunities

Please not that this is not one time activity but a journey, which I will explain each of them detail.

1. Pre conditioning the customer: We cannot mode a traditional customer who approached sourcing as a way to access the right skills at the right price at the right time. 

a. Customer connect: You need to really connect with customer decision makers and try to get feed back on current engagement ensure that you act on it. 

b. Mind share: Share views and ideas where you as service provider can optimize the resources, and try to understand the Goals and indicative work trends for upcoming qtr/ year. Share the idea of managed services and how it can be beneficial. Give him example of successful managed services is implemented and try to get a buy in to give a proposal to migrate from current T&M model to managed services model.

c. Influence lobbying: If needed connect with additional stake holders to coin the idea and see if there are any apprehensions from their side. Like loosing control on resources etc. rephrase the apprehensions and tell them that you will be coming up with proposal to discuss details and all their apprehensions will be taken care off.

Now that you have a good understanding of clients anticipated work load / goals and there is enough intention to move towards Managed services model and a formal request from client, you need to come up with formal proposal on the same. For doing so I proactively do a through analysis of delivery account.


2. Internal Assessment of delivery : I will do a formal review of delivery and from delivery stand point proactively look at metrics, try to spot trends and areas for improvement, wastage/ rework. 

a.Alignment: Ensure that client objectives are well captured and all tasks and deliverable s are aligned to tasks. This will help you identify areas which are duplicate/ redundant or of low priority. See if low priority tasks can be clubbed or avoided.

b. Work with Architects and Business Process SME's: Work with Architects and Process Sme's on the initiative and help need from each one of them to do through internal evaluation.

c.Critical path fine tuning: Identify critical path and the dependencies. As part for project delivery ensure the critical paths and dependencies are fine tuned 

c. Resource re fitment /Cross skill : Ensure that resources are cross skilled and each resource works thing end to end cycle and look of outcomes while they work on tasks. If needed ensure you on board new resources/ experts

d.Environment and external vendor management: Ensure the areas identifies to optimize with regards to environments and external vendors how they are currently engaged and kind of SLA's in place how changes are manged

e.Review work metrics: Do a through check of work packets are measured and how they are tagged to goals, what is smallest measure of how and how is it fairing with industry standards. Check the traditional information how the metrics are captured and any clues of wastage / rework.

f.Technology Upgrade/ Automation: With help of SME's see ares where you can on board automation or new technology to replace redundant technology

g.Identify risks and risk mitigation plan: Work with SME's and Architects to come across the anti paced risks and their grading and anticipated mitigation plan form your side.

h. Plan: Chalk out a new plan on areas you would like to re engineer the process, technology and engagement. In terms of new work flows, Day in and Day out of resource work book, Silo charts with various actors, New technology . Time needed for the implementation/ migrations, costs involved. Costs involved could be based on work packets/ tickets based based on complexity and nature of work or it can be out come based or it could just me managed model with FB.

Now that you did delivery revisit come up with proposal to client this could be PPT or work document It should contain

1. Back ground information on Program, Client goals, need for managed services

2. What is currently being delivered and 

3. Challenges and Observations from project, metrics

3. Opportunities to improvement in current mode of work.

4. Proposed model in terms of work flow, Technology upgrade as applicable. ( areas that can continue in T&M model and areas which can be taken up for managed services)

5. Silo charts and RACI ( Responsible, Accountable , Consulted and Informed) for various activities.

6. Time lines and Dependencies and how it will e implemented.

7. Anticipated challenges and how risks will be mitigated. 

8. Success stories where similar programs were implemented and lessons learned from them.

9.New resource model and Costs-Billing models. ( along with recommended billing model if going for outcome based model)

10. Q&A and next steps. etc

In earlier part of my carrier, I worked on T&M model but for last 8 years the trend is more towards  Fixed bid, Managed services, Outcome based model.

the following are sample implemented recently 

1. A large Bank based out of Europe and Africa ( Name white labeled intentionally). We  were involved in T&M model for last three years. When this account was given to me as part of my port folio, I observed that to a large part their deliverable were clear and well planned. I found that resources were underutilized and there is no innovation. I worked with manager and Architects and SME's and reworked a proposal for managed services where we manage the team and paid of  outcomes and quality of work done.

Quarter on quarter the improvement is measure of its a per promised or not and SLA's are met 

2. A large Bank based out of Nordics ( Name white labeled for intentionally):

We were involved in Application maintenance and testing, Environment management. Data management and recovery along with Infrastructure management. This account too we did a major revamp and brought engagement it into hybrid engagement that has various new buckets like

a. Based on work packets/ tickets resolved. 

b. Some part of engagement to Outcome based 

c. Some part as fixed bid, this is more on server maintenance etc.

e. Some part still remained as T&M

3. I also worked on other models  like

a. Device-Based- The unit of pricing is “device.” Pricing is applied per device based on device type and service class.

service Catalog Based- The buyer is offered a catalog of services with SLA commitments. Pricing can vary by service.

c. Gain-Share A commercial arrangement in which the client and service provider share the upfront investment requirements and the risks and rewards of their relationship.

more models can be discussed in person

Master Transition in just three steps and Magic Bullets to win it

As per Linkedin survey across IT, Legal, FM, HR, Finance and Procurement, 46% of the respondents said that they are very likely to Outsource Software Development if their company can realize cost benefits out of it. As more and more projects are transitioned offshore or they scale up at offshore, It’s imminent that these projects are transitioned well and newly on boarded team start productivity as quickly as possible. As shown in below diagram, clients may want to transition/ outsource for variety of reasons, you need to understand the client sensitives and regulations that he may come across as part of solutioning

 

For customers the paramount of transition success is it should carry no risk or minimal risk and should cost less. While for an outsourcing provider the success is on

1. Ensuring that transition is smooth
2. relationships are maintained
3. value is achieved
4. Benefits are realized along with establishing new connections and
5. Improving relationships between client stakeholders and their employees.

Many friends asked me to share a comprehensive plan so I am sharing this

Now that gave you a formal deep drive into how to go about transition, let me discuss three Magic Bullets you need to be aware off and to be included as part of transition.

1. Automation- Are you automating as much as possible and as fast as possible?
2. Innovation: What innovation you are bringing on the board.
3. Process upgrade: Update the workflows/ processes through implementation of tools

 

1. Benefits: Are benefits measured at each level? Do you have online transparency through proper dash board?
2. Quality: How is quality ensured are you failing fast and do you have enough check lists in place
3. Predictability: Do you have formalize preventive action and corrective action mechanism in place? Have you embraced AI

1. Regulatory: how are you managing and retaining client data? What mechanisms do you have to ensure that your operations meet compliance requirements