该文翻译自A look into the pro’s and con’s of Automated Testing and Manual Testing
几个星期以前我遇到我们团队中的自动测试专家，从他们那里得到了一些什么时候适宜进行自动测试什么时候适合进行手动测试的信息。其实掰掰手指头都能想到，必须使用常识来决定。如果你只需要运行测试一两次，或者做自动测试非常昂贵，那最好采用手动测试。不过，在使用常识之前，你必须拿出确定的一些关于何时开始自动化测试的指导方针。
自动测试的好处：
如果你需要反复运行一组测试，那么自动测试将会对你非常有用。
自动测试使你能够应对频繁改变的代码从而跟上周期性回归测试的脚步。
自动测试可以使你能够自动运行主流业务场景从而跟上周期性回归测试的脚步。（原文：It gives you the ability to run automation in main stream scenarios to catch regressions in a timely manner）
自动测试可以帮助你测试大量测试矩阵（在不同操作系统上的不同语言）。自动测试可以使你的测试同时运行在不同的机器上，而手动测试必须不断地继续执行。
自动测试的限制：
花费大。编写测试用例，编写和配置自动化测试框架将会在测试开始时花费比手动测试更多的费用。
无法自动测试一些可视的场景。例如，如果你无法通过代码告诉自动测试工具字体颜色，那么只好使用手动测试。
手动测试的好处：
如果一个测试用例在编码阶段只运行两次，那最好使用手动测试，它将比自动测试花费少得多的费用。
手动测试允许测试员进行更多的随机测试。以我的经验来看，更多的bug将会由随机测试发现，而不是自动测试。并且，一个测试员花费越多的时间进行随机测试，发现真正的用户bug的几率就越大。
手动测试的限制：
手动进行测试将花费大量的时间。
每次有了新的build，测试员必须重新运行测试-经过一段时间以后将会非常繁琐和疲惫。
其他的因素：
你将哪些部分进行自动测试也由你使用的工具决定。如果该工具有很多限制，那么这些部分还是手动测试吧。
是否投资的回报值得运行自动测试？是否你自动化测试的产出值得建立和支持测试用例，自动框架和运行测试用例的系统？
自动测试的标准
有两个问题可以用来判断是否应该为你的测试用例进行自动化。
Q1:是否测试场景可以自动化？
A1:是的，并且花费很少。
A2:是的，但是花费很多。
A3:不，不可能进行自动化。
Q2:该测试场景有多么重要？
A1:我必须在任何可能的时候都对其进行测试。
A2:我需要有规律地对该场景进行测试。
A3:我只需要测试该场景一次。
如果这两个问题你的答案都是#1，那么你肯定需要自动化该测试。
如果这两个问题你的答案是一个#1和一个#2，那么你最好自动化该测试。
如果这两个问题你的答案都是#2，那么你应该好好考虑一下是否你值得为自动化测试投资。
如果你无法自动测试，会有什么结果
让我们假设如果你有一个测试必须在任何可能的时间运行，但是却无法自动化它，你的选择是：
再评估 – 是否我真的需要如此频繁地运行它？
如果手动测试它会有多大的花费？
寻找新的测试工具。
考虑使用test hooks.
——————————————————————————————————————————-
原文如下：
A look into the pro’s and con’s of Automated Testing and Manual Testing
I met with my team’s automation experts a few weeks back to get their input on when to automate and when to manually test. The general rule of thumb has always been to use common sense. If you’re only going to run the test one or two times or the test is really expensive to automation, it is most likely a manual test. But then again, what good is saying “use common sense” when you need to come up with deterministic set of guidelines on how and when to automate?
Pros of Automation
If you have to run a set of tests repeatedly, automation is a huge win for you
It gives you the ability to run automation against code that frequently changes to catch regressions in a timely manner
It gives you the ability to run automation in mainstream scenarios to catch regressions in a timely manner (see What is a Nighlty)
Aids in testing a large test matrix (different languages on different OS platforms). Automated tests can be run at the same time on different machines, whereas the manual tests would have to be run sequentially.
Cons of Automation
It costs more to automate. Writing the test cases and writing or configuring the automate framework you’re using costs more initially than running the test manually.
Can’t automate visual references, for example, if you can’t tell the font color via code or the automation tool, it is a manual test.
Pros of Manual
If the test case only runs twice a coding milestone, it most likely should be a manual test. Less cost than automating it.
It allows the tester to perform more ad-hoc (random testing). In my experiences, more bugs are found via ad-hoc than via automation. And, the more time a tester spends playing with the feature, the greater the odds of finding real user bugs.
Cons of Manual
Running tests manually can be very time consuming
Each time there is a new build, the tester must rerun all required tests – which after a while would become very mundane and tiresome.
Other deciding factors
What you automate depends on the tools you use. If the tools have any limitations, those tests are manual.
Is the return on investment worth automating? Is what you get out of automation worth the cost of setting up and supporting the test cases, the automation framework, and the system that runs the test cases?
Criteria for automating
There are two sets of questions to determine whether automation is right for your test case:
Is this test scenario automatable?
Yes, and it will cost a little
Yes, but it will cost a lot
No, it is no possible to automate
How important is this test scenario?
I must absolutely test this scenario whenever possible
I need to test this scenario regularly
I only need to test this scenario once in a while
If you answered #1 to both questions – definitely automate that test
If you answered #1 or #2 to both questions – you should automate that test
If you answered #2 to both questions – you need to consider if it is really worth the investment to automate
What happens if you can’t automate?
Let’s say that you have a test that you absolutely need to run whenever possible, but it isn’t possible to automate. Your options are
Reevaluate – do I really need to run this test this often?
What’s the cost of doing this test manually?
Look for new testing tools
Consider test hooks