A Quick JUnit vs TestNG Comparison – JUnit与TestNG的快速比较

1. Overview

1.概述

JUnit and TestNG are undoubtedly the two most popular unit-testing frameworks in the Java ecosystem. While JUnit inspires TestNG itself, it provides its distinctive features, and unlike JUnit, it works for functional and higher levels of testing.

JUnit和TestNG无疑是Java生态系统中最流行的两个单元测试框架。虽然JUnit启发了TestNG本身，但它提供了其独特的功能，与JUnit不同，它适用于功能测试和更高层次的测试。

In this post, we’ll discuss and compare these frameworks by covering their features and common use cases.

在这篇文章中，我们将讨论和比较这些框架，包括它们的功能和常见的用例。

2. Test Setup

2.测试设置

While writing test cases, often we need to execute some configuration or initialization instructions before test executions, and also some cleanup after completion of tests. Let’s evaluate these in both frameworks.

在编写测试用例时，我们经常需要在测试执行前执行一些配置或初始化指令，也需要在测试完成后进行一些清理。让我们在这两个框架中评估一下这些。

JUnit offers initialization and cleanup at two levels, before and after each method and class. We have @BeforeEach, @AfterEach annotations at method level and @BeforeAll and @AfterAll at class level:

JUnit在两个层面上提供了初始化和清理，在每个方法和类之前和之后。我们在方法层面上有@BeforeEach, @AfterEach注解，在类层面上有@BeforeAll和 @AfterAll。

public class SummationServiceTest {

    private static List<Integer> numbers;

    @BeforeAll
    public static void initialize() {
        numbers = new ArrayList<>();
    }

    @AfterAll
    public static void tearDown() {
        numbers = null;
    }

    @BeforeEach
    public void runBeforeEachTest() {
        numbers.add(1);
        numbers.add(2);
        numbers.add(3);
    }

    @AfterEach
    public void runAfterEachTest() {
        numbers.clear();
    }

    @Test
    public void givenNumbers_sumEquals_thenCorrect() {
        int sum = numbers.stream().reduce(0, Integer::sum);
        assertEquals(6, sum);
    }
}

Note that this example uses JUnit 5. In the previous JUnit 4 version, we would need to use the @Before and @After annotations which are equivalent to @BeforeEach and @AfterEach. Likewise, @BeforeAll and @AfterAll are replacements for JUnit 4’s @BeforeClass and @AfterClass.

请注意，这个例子使用了JUnit 5。在之前的JUnit 4版本中，我们需要使用@Before 和@After 注解，这相当于@BeforeEach和@AfterEach。同样，@BeforeAll和@AfterAll是JUnit 4的@BeforeClass和@AfterClass.的替代。

Similar to JUnit, TestNG also provides initialization and cleanup at the method and class level. While @BeforeClass and @AfterClass remain the same at the class level, the method level annotations are @BeforeMethod and @AfterMethod:

与JUnit类似，TestNG也在方法和类级别提供初始化和清理。虽然@BeforeClass 和@AfterClass 在类级别保持不变，但方法级别的注释是@BeforeMethod 和@AfterMethod:。

@BeforeClass
public void initialize() {
    numbers = new ArrayList<>();
}

@AfterClass
public void tearDown() {
    numbers = null;
}

@BeforeMethod
public void runBeforeEachTest() {
    numbers.add(1);
    numbers.add(2);
    numbers.add(3);
}

@AfterMethod
public void runAfterEachTest() {
    numbers.clear();
}

TestNG also offers, @BeforeSuite, @AfterSuite, @BeforeGroup and @AfterGroup annotations, for configurations at suite and group levels:

TestNG还提供，@BeforeSuite、@AfterSuite、@BeforeGroup和@AfterGroup注释，用于套件和组级别的配置：。

@BeforeGroups("positive_tests")
public void runBeforeEachGroup() {
    numbers.add(1);
    numbers.add(2);
    numbers.add(3);
}

@AfterGroups("negative_tests")
public void runAfterEachGroup() {
    numbers.clear(); 
}

Also, we can use the @BeforeTest and @AfterTest if we need any configuration before or after test cases included in the <test> tag in TestNG XML configuration file:

另外，如果我们需要在TestNG XML配置文件中的<test>tag所包含的测试案例之前或之后进行任何配置，我们可以使用@BeforeTest和@AfterTest。

<test name="test setup">
    <classes>
        <class name="SummationServiceTest">
            <methods>
                <include name="givenNumbers_sumEquals_thenCorrect" />
            </methods>
        </class>
    </classes>
</test>

Note that the declaration of @BeforeClass and @AfterClass method has to be static in JUnit. By comparison, TestNG method declaration doesn’t have these constraints.

注意，在JUnit中，@BeforeClass和@AfterClass方法的声明必须是静态的。相比之下，TestNG的方法声明就没有这些限制。

3. Ignoring Tests

3.无视测试

Both frameworks support ignoring test cases, though they do it quite differently. JUnit offers the @Ignore annotation:

这两个框架都支持忽略测试用例，尽管它们的做法完全不同。JUnit提供了@Ignore注解。

@Ignore
@Test
public void givenNumbers_sumEquals_thenCorrect() {
    int sum = numbers.stream().reduce(0, Integer::sum);
    Assert.assertEquals(6, sum);
}

while TestNG uses @Test with a parameter “enabled” with a boolean value true or false:

而TestNG使用@Test，参数 “enabled “为布尔值true或false。

@Test(enabled=false)
public void givenNumbers_sumEquals_thenCorrect() {
    int sum = numbers.stream.reduce(0, Integer::sum);
    Assert.assertEquals(6, sum);
}

4. Running Tests Together

4.一起运行测试

Running tests together as a collection is possible in both JUnit and TestNG, but they do it in different ways.

在JUnit和TestNG中，作为一个集合一起运行测试是可能的，但它们以不同的方式进行。

We can use the @Suite, @SelectPackages, and @SelectClasses annotations to group test cases and run them as a suite in JUnit 5. A suite is a collection of test cases that we can group together and run as a single test.

我们可以使用@Suite, @SelectPackages和@SelectClasses注解来分组测试用例，并在JUnit 5中作为一个套件运行。套件是一个测试用例的集合，我们可以将其分组并作为一个测试运行。

If we want to group test cases of different packages to run together within a Suite we need the @SelectPackages annotation:

如果我们想将不同包的测试用例分组，在一个Suite中一起运行，我们需要@SelectPackages注解。

@Suite
@SelectPackages({ "org.baeldung.java.suite.childpackage1", "org.baeldung.java.suite.childpackage2" })
public class SelectPackagesSuiteUnitTest {

}

If we want specific test classes to run together, JUnit 5 provides the flexibility through @SelectClasses:

如果我们希望特定的测试类一起运行，JUnit 5通过@SelectClasses提供了灵活性。

@Suite
@SelectClasses({Class1UnitTest.class, Class2UnitTest.class})
public class SelectClassesSuiteUnitTest {

}

Previously using JUnit 4, we achieved grouping and running multiple tests together using @RunWith and @Suite annotations:

以前使用JUnit 4时，我们使用@RunWith和@Suite注释实现了分组和一起运行多个测试。

@RunWith(Suite.class)
@Suite.SuiteClasses({ RegistrationTest.class, SignInTest.class })
public class SuiteTest {

}

In TestNG we can group tests by using an XML file:

在TestNG中，我们可以通过使用XML文件对测试进行分组：。

<suite name="suite">
    <test name="test suite">
        <classes>
            <class name="com.baeldung.RegistrationTest" />
            <class name="com.baeldung.SignInTest" />
        </classes>
    </test>
</suite>

This indicates RegistrationTest and SignInTest will run together.

这表明RegistrationTest和SignInTest将一起运行。

Apart from grouping classes, TestNG can group methods as well using the @Test(groups=”groupName”) annotation:

除了对类进行分组，TestNG也可以使用@Test(groups=”groupName”)注解对方法进行分组。

@Test(groups = "regression")
public void givenNegativeNumber_sumLessthanZero_thenCorrect() {
    int sum = numbers.stream().reduce(0, Integer::sum);
    Assert.assertTrue(sum < 0);
}

Let’s use an XML to execute the groups:

让我们用一个XML来执行组。

<test name="test groups">
    <groups>
        <run>
            <include name="regression" />
        </run>
    </groups>
    <classes>
        <class 
          name="com.baeldung.SummationServiceTest" />
    </classes>
</test>

This will execute the test method tagged with group regression.

这将执行标记为regression组的测试方法。

5. Testing Exceptions

5.测试异常情况

The feature for testing for exceptions using annotations is available in both JUnit and TestNG.

使用注解测试异常的功能在JUnit和TestNG中都是可用的。

Let’s first create a class with a method that throws an exception:

让我们首先创建一个带有抛出异常的方法的类。

public class Calculator {
    public double divide(double a, double b) {
        if (b == 0) {
            throw new DivideByZeroException("Divider cannot be equal to zero!");
        }
        return a/b;
    }
}

In JUnit 5 we can use the assertThrows API to test exceptions:

在JUnit 5中，我们可以使用assertThrowsAPI来测试异常。

@Test
public void whenDividerIsZero_thenDivideByZeroExceptionIsThrown() {
    Calculator calculator = new Calculator();
    assertThrows(DivideByZeroException.class, () -> calculator.divide(10, 0));
}

In JUnit 4, we can achieve this by using @Test(expected = DivideByZeroException.class) over the test API.

在JUnit 4中，我们可以通过使用@Test(expected = DivideByZeroException.class) 过测试API来实现。

And with TestNG we can also implement the same:

而通过TestNG，我们也可以实现同样的功能。

@Test(expectedExceptions = ArithmeticException.class) 
public void givenNumber_whenThrowsException_thenCorrect() { 
    int i = 1 / 0;
}

This feature implies what exception is thrown from a piece of code, that’s part of a test.

这个功能意味着从一段代码中抛出什么异常，这是测试的一部分。

6. Parameterized Tests

6.参数化测试

Parameterized unit tests are helpful for testing the same code under several conditions. With the help of parameterized unit tests, we can set up a test method that obtains data from some data source. The main idea is to make the unit test method reusable and to test with a different set of inputs.

参数化单元测试有助于在多种条件下测试同一代码。在参数化单元测试的帮助下，我们可以设置一个测试方法，从一些数据源获取数据。其主要思想是使单元测试方法可重复使用，并以不同的输入集进行测试。

In JUnit 5, we have the advantage of test methods consuming data arguments directly from the configured source. By default, JUnit 5 provides a few source annotations like:

在JUnit 5中，我们拥有测试方法直接从配置的源头消耗数据参数的优势。默认情况下，JUnit 5提供了一些source注释，如。

• @ValueSource: we can use this with an array of values of type Short, Byte, Int, Long, Float, Double, Char, and String:

@ParameterizedTest
@ValueSource(strings = { "Hello", "World" })
void givenString_TestNullOrNot(String word) {
    assertNotNull(word);
}

• @EnumSource – passes Enum constants as parameters to the test method:

@ParameterizedTest
@EnumSource(value = PizzaDeliveryStrategy.class, names = {"EXPRESS", "NORMAL"})
void givenEnum_TestContainsOrNot(PizzaDeliveryStrategy timeUnit) {
    assertTrue(EnumSet.of(PizzaDeliveryStrategy.EXPRESS, PizzaDeliveryStrategy.NORMAL).contains(timeUnit));
}

• @MethodSource – passes external methods generating streams:

static Stream<String> wordDataProvider() {
    return Stream.of("foo", "bar");
}

@ParameterizedTest
@MethodSource("wordDataProvider")
void givenMethodSource_TestInputStream(String argument) {
    assertNotNull(argument);
}

• @CsvSource – uses CSV values as a source for the parameters:

@ParameterizedTest
@CsvSource({ "1, Car", "2, House", "3, Train" })
void givenCSVSource_TestContent(int id, String word) {
	assertNotNull(id);
	assertNotNull(word);
}

Similarly, we have other sources like @CsvFileSource if we need to read a CSV file from classpath and @ArgumentSource to specify a custom, reusable ArgumentsProvider.

同样，如果我们需要从classpath读取一个CSV文件，我们还有其他来源，如@CsvFileSource和@ArgumentSource来指定一个自定义的、可重复使用的ArgumentsProvider。

In JUnit 4, the test class has to be annotated with @RunWith to make it a parameterized class and @Parameter to use the denote the parameter values for unit test.

在JUnit 4中，测试类必须用@RunWith来注解，使其成为一个参数化的类，并使用@Parameter来表示单元测试的参数值。

In TestNG, we can parametrize tests using @Parameter or @DataProvider annotations. While using the XML file annotate the test method with @Parameter:

在TestNG中，我们可以使用@Parameter 或@DataProvider 注释来对测试进行参数化。在使用XML文件时，用@Parameter：注释测试方法。

@Test
@Parameters({"value", "isEven"})
public void 
  givenNumberFromXML_ifEvenCheckOK_thenCorrect(int value, boolean isEven) {
    Assert.assertEquals(isEven, value % 2 == 0);
}

and provide the data in the XML file:

并提供XML文件中的数据。

<suite name="My test suite">
    <test name="numbersXML">
        <parameter name="value" value="1"/>
        <parameter name="isEven" value="false"/>
        <classes>
            <class name="baeldung.com.ParametrizedTests"/>
        </classes>
    </test>
</suite>

While using information in the XML file is simple and useful, in some cases, you might need to provide more complex data.

虽然使用XML文件中的信息是简单而有用的，但在某些情况下，你可能需要提供更复杂的数据。

For this, we can use the @DataProvider annotation which allows us to map complex parameter types for testing methods.

为此，我们可以使用@DataProvider注解，它允许我们为测试方法映射复杂参数类型。

Here’s an example of using @DataProvider for primitive data types:

下面是一个为原始数据类型使用@DataProvider的例子。

@DataProvider(name = "numbers")
public static Object[][] evenNumbers() {
    return new Object[][]{{1, false}, {2, true}, {4, true}};
}

@Test(dataProvider = "numbers")
public void givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect
  (Integer number, boolean expected) {
    Assert.assertEquals(expected, number % 2 == 0);
}

And @DataProvider for objects:

而@DataProvider为对象。

@Test(dataProvider = "numbersObject")
public void givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect
  (EvenNumber number) {
    Assert.assertEquals(number.isEven(), number.getValue() % 2 == 0);
}

@DataProvider(name = "numbersObject")
public Object[][] parameterProvider() {
    return new Object[][]{{new EvenNumber(1, false)},
      {new EvenNumber(2, true)}, {new EvenNumber(4, true)}};
}

In the same way, any particular objects that are to be tested can be created and returned using data provider. It’s useful when integrating with frameworks like Spring.

以同样的方式，任何要测试的特定对象都可以使用数据提供者创建和返回。在与Spring等框架集成时，这很有用。

Notice that, in TestNG, since @DataProvider method need not be static, we can use multiple data provider methods in the same test class.

注意，在TestNG中，由于@DataProvider 方法不需要是静态的，我们可以在同一个测试类中使用多个数据提供者方法。

7. Test Timeout

7.测试超时

Timed out tests means, a test case should fail if the execution is not completed within a certain specified period. Both JUnit and TestNG support timed out tests. In JUnit 5 we can write a timeout test as:

计时测试是指，如果一个测试用例在某个指定的时间段内没有完成执行，就应该失败。JUnit和TestNG都支持超时测试。在JUnit 5中，我们可以写一个超时测试：。

@Test
public void givenExecution_takeMoreTime_thenFail() throws InterruptedException {
    Assertions.assertTimeout(Duration.ofMillis(1000), () -> Thread.sleep(10000));
}

In JUnit 4 and TestNG we can the same test using @Test(timeout=1000)

在JUnit 4和TestNG中，我们可以使用@Test(timeout=1000)进行相同的测试。

@Test(timeOut = 1000)
public void givenExecution_takeMoreTime_thenFail() {
    while (true);
}

8. Dependent Tests

8.从属测试

TestNG supports dependency testing. This means in a set of test methods, if the initial test fails, then all subsequent dependent tests will be skipped, not marked as failed as in the case for JUnit.

TestNG支持依赖性测试。这意味着在一组测试方法中，如果初始测试失败，那么所有后续的依赖测试将被跳过，而不是像JUnit那样被标记为失败。

Let’s have a look at a scenario, where we need to validate email, and if it’s successful, will proceed to log in:

让我们看看一个场景，我们需要验证电子邮件，如果成功，将继续登录。

@Test
public void givenEmail_ifValid_thenTrue() {
    boolean valid = email.contains("@");
    Assert.assertEquals(valid, true);
}

@Test(dependsOnMethods = {"givenEmail_ifValid_thenTrue"})
public void givenValidEmail_whenLoggedIn_thenTrue() {
    LOGGER.info("Email {} valid >> logging in", email);
}

9. Order of Test Execution

9.测试执行的顺序

There is no defined implicit order in which test methods will get executed in JUnit 4 or TestNG. The methods are just invoked as returned by the Java Reflection API. Since JUnit 4 it uses a more deterministic but not predictable order.

在JUnit 4或TestNG中，没有定义测试方法执行的隐含顺序。这些方法只是按照Java Reflection API返回的顺序被调用。自JUnit 4以来，它使用了一个更确定的但不可预测的顺序。

To have more control, we will annotate the test class with @FixMethodOrder annotation and mention a method sorter:

为了有更多的控制，我们将用@FixMethodOrder注解测试类，并提到一个方法分类器。

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SortedTests {

    @Test
    public void a_givenString_whenChangedtoInt_thenTrue() {
        assertTrue(
          Integer.valueOf("10") instanceof Integer);
    }

    @Test
    public void b_givenInt_whenChangedtoString_thenTrue() {
        assertTrue(
          String.valueOf(10) instanceof String);
    }

}

The MethodSorters.NAME_ASCENDING parameter sorts the methods by method name is lexicographic order. Apart from this sorter, we have MethodSorter.DEFAULT and MethodSorter.JVM as well.

MethodSorters.NAME_ASCENDING参数按照方法名称进行排序，是按词典顺序排列的。除了这个排序器之外，我们还有MethodSorter.DEFAULT和MethodSorter.JVM。

While TestNG also provides a couple of ways to have control in the order of test method execution. We provide the priority parameter in the @Test annotation:

虽然TestNG也提供了一些方法来控制测试方法的执行顺序。我们在@Test 注释中提供了priority 参数。

@Test(priority = 1)
public void givenString_whenChangedToInt_thenCorrect() {
    Assert.assertTrue(
      Integer.valueOf("10") instanceof Integer);
}

@Test(priority = 2)
public void givenInt_whenChangedToString_thenCorrect() {
    Assert.assertTrue(
      String.valueOf(23) instanceof String);
}

Notice, that priority invokes test methods based on priority but does not guarantee that tests in one level are completed before invoking the next priority level.

注意，优先级根据优先级调用测试方法，但不保证一个级别的测试在调用下一个优先级之前完成。

Sometimes while writing functional test cases in TestNG, we might have an interdependent test where the order of execution must be the same for every test run. To achieve that we should use the dependsOnMethods parameter to @Test annotation as we saw in the earlier section.

有时在TestNG中编写功能测试用例时，我们可能会有一个相互依赖的测试，每次测试的执行顺序必须是相同的。为了实现这一点，我们应该使用dependsOnMethods参数来@Testannotation，正如我们在前面的章节中看到的那样。

10. Custom Test Name

10.自定义测试名称

By default, whenever we run a test, the test class and the test method name is printed in console or IDE. JUnit 5 provides a unique feature where we can mention custom descriptive names for class and test methods using @DisplayName annotation.

默认情况下，每当我们运行一个测试时，测试类和测试方法的名称会被打印在控制台或IDE中。JUnit 5提供了一个独特的功能，我们可以使用@DisplayName注解为类和测试方法提到自定义描述性名称。

This annotation doesn’t provide any testing benefits but it brings easy to read and understand test results for a non-technical person too:

这种注释并不提供任何测试的好处，但它也为非技术人员带来了容易阅读和理解的测试结果。

@ParameterizedTest
@ValueSource(strings = { "Hello", "World" })
@DisplayName("Test Method to check that the inputs are not nullable")
void givenString_TestNullOrNot(String word) {
    assertNotNull(word);
}

Whenever we run the test, the output will show the display name instead of the method name.

每当我们运行测试时，输出将显示显示名称而不是方法名称。

Right now, in TestNG there is no way to provide a custom name.

现在，在TestNG中，没有办法提供一个自定义名称。

11. Conclusion

11.结论

Both JUnit and TestNG are modern tools for testing in the Java ecosystem.

JUnit和TestNG都是Java生态系统中的现代测试工具。

In this article, we had a quick look at various ways of writing tests with each of these two test frameworks.

在这篇文章中，我们快速浏览了用这两个测试框架中的每一个编写测试的各种方法。

The implementation of all the code snippets can be found in TestNG and junit-5 Github project.

所有代码片段的实现都可以在TestNG和junit-5 Github项目中找到。