 |
 |
 |
 |
TestNG is a Java testing framework that is particularly well-suited for use on large projects. It provides especially strong support for writing functional and integration tests as well as unit tests. TestNG key features include:
TestNG uses Java annotations extensively. This means no reliance on class inheritance or method naming conventions as in JUnit 3. While JUnit 4 also uses annotations and shares some features with TestNG, there are significant differences in philosophy and implementation. JUnit strives to keep tests isolated from one another, while TestNG is more flexible, allowing various types of dependencies between tests. The key difference is the context in which the tests run.
| TestNG | JUnit |
|---|---|
| All test methods in the same class execute in the same instance of the class | Each test method executes in its own class instance |
| Easy to share resources among tests | Sharing resources among tests limited and not encouraged |
TestNG can be invoked in a number of ways - from the command line, from
within an IDE such as Eclipse, IntelliJ IDEA, or NetBeans, from the
testng Ant task, or from Maven. In all cases, a suite
XML file (also referred to as a testng.xml file) is
used to define the test suites. Although it is not required, a suite XML
file provides the most power and flexibility in configuring and selecting
which tests to run. Here's a simple command line example, which runs the
suite defined by testng.xml (assuming the testng jar file
and any classes used during testing are on the classpath):
java org.testng.TestNG testng.xml
There are several useful optional arguments, including
A testng.xml file defines a single test suite. A suite contains one or more <test> elements, each of which specifies a set of test classes. The tests which make up the <test> element can be specified in one or more of the following ways:
Here is an example suite which defines a test named
CoreTests that includes classes ClassATest and
ClassBTest, plus all test classes in the package
com.ociweb.jnb.testng.core and any subpackages (because the
package name ends with ".*"). In addition, the group long-tests is
excluded from the test.
<suite name="Suite" parallel="false">
<test name="CoreTests">
<classes>
<class name="com.ociweb.jnb.testng.ClassATest"/>
<class name="com.ociweb.jnb.testng.ClassBTest"/>
</classes>
<packages>
<package name="com.ociweb.jnb.testng.core.*"/>
</packages>
<groups>
<run>
<exclude name="long-tests"/>
</run>
</groups>
</test>
</suite>
For more details on the suite XML file, an HTML version of the DTD schema is available at http://testng.org/dtd/.
Any class can be turned into a TestNG test class by adding at least one
TestNG annotation. The @Test annotation is used to designate
the test methods in a class. It can be applied to individual methods, as
in the following example:
public class ExampleMethodAnnotations {
@Test
public void checkSomething() {}
@Test
public void checkSomethingElse() {}
public void notATestMethod() {}
}
The @Test annotation can also be applied to a class, in
which case all public methods of that class will be test methods, unless
they are annotated by one of the @BeforeXXX or
@AfterXXX configuration annotations described below.
@Test
public class ExampleClassAnnotation {
public void checkSomething() {}
public void checkSomethingElse() {}
}
Configuration annotations can be applied to methods to perform some action either before or after specific events:
@BeforeSuite/@AfterSuite: Executes before/after any tests
in the test suite
@BeforeTest/@AfterTest: Executes before/after any tests in
the <test> element which contains this class
@BeforeClass/@AfterClass: Executes before/after any tests
in this class
@BeforeMethod/@AfterMethod: Executes before/after every
test method in this class
@BeforeGroups/@AfterGroups: Executes before/after any
tests in the specified groups.
For example, the methods in the class TestBeforeAfter are
executed in the following order:
beforeSuite
beforeTest
beforeClass
beforeMethod
test1
afterMethod
beforeMethod
test2
afterMethod
afterClass
afterTest
afterSuite
public class TestBeforeAfter {
@BeforeSuite
public void beforeSuite() {}
@AfterSuite
public void afterSuite() {}
@BeforeTest
public void beforeTest() {}
@AfterTest
public void afterTest() {}
@BeforeClass
public void beforeClass() {}
@AfterClass
public void afterClass() {}
@BeforeMethod
public void beforeMethod() {}
@AfterMethod
public void afterMethod() {}
@Test
public void test1() {}
@Test
public void test2() {}
}
It's natural to want to place tests into groups. For example, you may have a core group of tests which should be run before any files are checked in to the source code repository. You could also have groups for different types of tests, such as integration tests and performance tests. Tests which cover different features could also be in different groups. A "broken tests" group can be useful for tests which are known to be broken, and won't be fixed for a while.
In TestNG, groups are used to control execution of tests, both by including or excluding tests from a suite, and by controlling the order of tests and whether tests are skipped. A test method can be a member of any number of groups. Test methods can be assigned to groups at the class level (all test methods in the class are members of the class-level groups), on a per-method basis, or a combination of both.
An example of assigning groups at the method level:
public class MethodLevelGrouping {
@Test(groups={"a"})
public void a() {}
@Test(groups={"b"})
public void b() {}
@Test(groups={"a", "b"})
public void ab() {}
}
To assign groups at the class level, add a @Test annotation
to the class. This turns all public methods in the class into test
methods (unless they have a @BeforeXXX or
@AfterXXX annotation). These methods also acquire any
annotation elements (such as groups) defined by the
class-level @Test annotation. In the following example,
testB1 and testB2 are test methods belonging to
group b, and method testBC belongs to groups b and c.
@Test(groups={"b"})
public class ClassLevelGrouping {
public void testB1() {}
public void testB2() {}
@Test(groups={"c"})
public void testBC() {}
}
Tests can depend on other methods or groups. This effects test execution order and can cause tests to be skipped depending on the results of other tests. The execution order rules are as follows:
There are two types of dependencies, hard (the default) and soft. This determines whether tests are skipped or not.
@Test(alwaysRun=false))
means that the dependent test will be skipped if any tests it
depends on fails.
@Test(alwaysRun=true)) means
that a test will run regardless of the outcome of the tests it depends
on. However, it will always execute after all the tests it depends on.
In this example, test depTest depends on methods
pre1 and pre2. depTest always runs
after pre1 and pre2, and will be skipped if
either pre1 or pre2 fail. Note that
pre1 and pre2 can be executed in any order.
public class DependsOnMethods {
@Test(dependsOnMethods={"pre1", "pre2"})
public void depTest() {}
@Test
public void pre1() {}
@Test
public void pre2() {}
}
Here's an example where a number of tests depend on an external server
and some environment configuration. We want to avoid executing
the server-dependent tests if the server is down. One way to do this
is do create test methods which
verify that the server is working and the environment is configured correctly.
For convenience, these methods are placed in an init
group, and the server-dependent tests depend on this group.
Then, if the server is down, the
server-dependent tests will be skipped. This has the benefit of
potentially saving a lot of time (no waiting for the server to time-out
for each test), plus the error report is more accurate, allowing one to
focus more quickly on the actual failures. In this example, the
init group methods are placed in their own class. Alternatively,
all the methods could be placed into one class, though this would
require annotations on each method.
@Test(groups={"init"})
public class InitTests {
public void checkEnvironment() {}
public void checkServer() {}
}
@Test(dependsOnGroups={"init"})
public class ServerDependentTests {
public void sdtest1() {}
public void sdtest2() {}
}
Test and configuration methods can have parameters. Parameter values can
be assigned in two ways - by using the @Parameters
annotation, or by using DataProviders.
Here's an example of using the @Parameters annotation to
assign a server name and port number.
@Parameters({"server-name", "port"})
@BeforeTest
public void setupServer(String serverName, int port) {
this.serverName = serverName;
this.port = port;
}
The parameter values are assigned in the suite XML file (note that the
name attributes must match the names listed in the
@Parameters annotation):
<test name="ParametersTest">
<parameter name="server-name" value="test-server "/>
<parameter name="port" value="1234"/>
...
</test>
Parameter elements can be placed under the <suite> or
<test> elements. Suite parameters apply to all tests
unless they are overridden by a parameter of the same name under a
<test> element.
While the @Parameters approach is easy to use, it has some
limitations. Parameter types are limited to simple types such as
String, int, and double. Also, the
parameter values are only assigned once, so it is not possible to invoke
the same method with multiple sets of parameter values.
DataProviders overcome these limitations.
A DataProvider is a method annotated with @DataProvider
which returns either an Object[][] or
Iterator<Object[]>. In either case, the inner
Object[] contains the parameters for a single invocation of
a test method. The number and types of elements in each row must match
the number and types of parameters in the test method. Here's a simple
example:
@DataProvider(name = "indexOfProvider")
public Object[][] indexOfTestGenerator() {
return new Object[][] {
{ -1, "something", "x" },
{ 4, "something", "thing" }, };
}
@Test(dataProvider="indexOfProvider")
public void testStringIndexOf(int expect, String s1, String s2) {
assertEquals(expect, s1.indexOf(s2));
}
In this example, the test method will be invoked twice, first with
parameters {-1, "something", "x"}, then with parameters
{4, "something", "thing"}.
DataProviders can be particularly useful when implementing data-driven tests - tests where it is necessary to perform the same checks with a variety of input data sets. Typical ways of handling these types of tests include:
TestNG supports running tests in multiple threads. Use this feature to
speed up execution of thread-safe tests. By default, all tests run in a
single thread. To use multiple threads, use the parallel and
thread-count attributes of the <suite>
and <test> tags. The parallel attribute
can be set to the following values:
"tests": All test methods in a given
<test> tag are executed in the same thread, but
different <test> tags may execute in different
threads. This value is only valid for <suite> tags.
"classes": All test methods in the same class execute in a
single thread, but methods in different classes may execute in
different threads.
"methods": All test methods may potentially execute in
different threads.
"false": Don't use multiple threads.
The thread-count attribute is used to set the number of
threads in the thread pool. If parallel or
thread-count attributes are specified on a
<test> tag, their values take precedence over the
<suite> values.
Here's one way to define a suite where parallel is "methods"
by default, but is overridden by a test which contains methods which
should not be run in parallel:
<suite name="Suite" parallel="methods" thread-count="10"> ... <test name= "not-parallel" parallel="false">
By default, TestNG produces an HTML report and a TestNG specific XML
file. In addition, XML files are produced which are compatible with the
Ant JUnitReport task. This is handy for integrating into existing build
frameworks, but lacks any TestNG-specific data. It is also possible to
produce custom reports by implementing an IReporter reporter
class. This class is called when all the test suites have completed, and
it is passed the test results for all tests in the suites.
TestNG provides good support for managing large test suites, and provides solutions to common problems which arise particularly when writing functional and integration tests. This makes it a natural fit for many projects, where it can make it easier to write tests, more efficient to run the tests, and easier to interpret the results.
![[Valid RSS]](images/valid-rss-rogers.png "Validate my RSS feed")