Why automation fails and simple recommendations : Test Automation vs Managed Test Automation

“Automation thy name is magic” this is how most project managers perceives about test automation when they are burdened with resource crunch and release deadline. Well that precisely can be one major reason for test automation failure beside several reasons; some of them are listed below:

  • Poor test strategy

    Typically most test automation fails and management loose interest as they don’t get a clear picture of how the test automation is helping the organization. Test Automation strategy becomes too isolated and is not part of the implementation and release plan. It would be a good idea to develop a core test automation strategy that is always part of wider test management practice and also provide key inputs to release management. For e.g. release manager should ask how much test automation can help in identifying the bottleneck in regression test cycles. Are automated scripts reflecting the high risk and core functionalities of the company? How can we depend on test automation during the release? Are these tests matured and maintainable?

  • Poor Test Automation Architecture

    Test Automation should be treated as same way as a software development methodology. Organizing and bringing discipline to the test automation is very important. Simple folder structures can help you organize things faster.

    • Folder : Order To Cash
    • Sub Folder: Auto scripts
    • Sub Folder: Data Table
    • Sub Folder: Documentation
  • It’s not about record and replay

    Most project manager’s purchase automation tools with the face value of record and replay and promises made by the sales pitch. Then they start facing issues from poor scripting to improper test automation framework. The tools don’t do automation its “Staged Managed”. The recording and replaying has been just a characteristic of test automation. You can buy several tools in the market, but if you don’t understand the discipline of test automation then the whole practice fails. It is important that stakeholders should clearly understand limitations of test automation.

  • Lack of understanding

    Test Automation should be considered for project delivery from the beginning. Project Managers should take test automation with the same seriousness as implementation and development work stream. In a maintenance project a big budget is allocated to the development portion, but a fraction of the budget for testing & very little for test automation. This budget gaps itself shows the lack of understanding of test automation. If there is a high risk of project delivery, then test automation will only benefit. So the key point for PMs is to consider

    • How will test automation be useful in a complex landscape?
    • Can we make separate test automation regression test environment?
    • How will test automation help in release cycle?
    • Should we execute test automation frequently?
    • Does the automation tools has sufficient accelerators that is compatible to system landscape?
  • Specialized skills

    Test Automation is a specialized skill and a different profession than manual testers. Its equivalent to software developers. PM’s should consider hiring people who are test automation engineers, especially who are capable of writing custom code like subroutines. SAP TAO is a great framework to work, but if you get stuck then the code can be patchy. SAP TAO allows you to enhance the custom code in a wrapper, but it requires specialized programming skills. Also, if your scenario is passing through several products that is non-SAP then your testing becomes complex and SAP TAO will not be able to work.

  • High Expectations

    Your ROI will come with maturity in test automation. Nothings happen in short time. If you are thinking of test automation just for a couple of releases then it’s better to do manual testing. You should develop your test automation suite iteratively and incrementally. Test Automation maturity comes from the frequent execution of test scripts in a virtual lab environment as many times as possible. Project Managers should keep the high expectations, frequent execution. You will improve your regression suite over the period of time.

  • Long Term Investment

    Remember there are many challenges in SAP. Every new enhancement, business line needs to be tested. Building Test Automation Team and Managed Test Automation is a key to success in the hard pressed resources.