Custom Reporting with TestNG – 使用TestNG的自定义报告

最后修改: 2017年 5月 14日

中文/混合/英文(键盘快捷键：t)

1. Overview

1.概述

In this article, we will discuss generating custom logging and reports using TestNG.

在这篇文章中，我们将讨论使用TestNG生成自定义日志和报告。

TestNG provides its own reporting feature – generating reports in either HTML/XML formats. If tests are run using the maven-surefire-plugin, the report will take the default form defined by the plugin. Apart from built-in reporting, it provides a mechanism for easy customization of logged information and generated reports.

TestNG提供了自己的报告功能–以HTML/XML格式生成报告。如果使用maven-surefire-plugin运行测试，报告将采用该插件定义的默认形式。除了内置的报告，它还提供了一种机制，可以轻松定制记录的信息和生成的报告。

If you’d like to start with the TestNG basics, check out this article.

如果你想从TestNG的基础知识开始学习，请查看这篇文章。

2. Custom Logging

2.自定义日志记录

Before we implement custom logging, let’s look into the default logs by executing mvn test command:

在我们实施自定义日志之前，让我们通过执行mvn test command:来查看默认日志。

Tests run: 11, Failures: 1, Errors: 0, Skipped: 0, 
  Time elapsed: 1.21 sec <<< FAILURE! 
- in TestSuite
whenCalledFromSuite_thanOK(baeldung.com.RegistrationTest)  
Time elapsed: 0.01 sec  <<< FAILURE!
java.lang.AssertionError: Test Failed due to some reason
    at baeldung.com.RegistrationTest.whenCalledFromSuite_thanOK(
      RegistrationTest.java:15)


Results :

Failed tests:
  RegistrationTest.whenCalledFromSuite_thanOK:15 
    Test Failed due to some reason

Tests run: 11, Failures: 1, Errors: 0, Skipped: 0

[ERROR] There are test failures.

These logs do not give us any information about the order of execution, or about when a particular test was started/completed etc.

这些日志没有给我们提供任何关于执行顺序的信息，或者关于某个特定测试何时开始/完成的信息等等。

If we want to know the result of each run along with some custom data, we can implement our own logs and reports. TestNG provides a way of implementing custom reports and logging.

如果我们想知道每次运行的结果以及一些自定义的数据，我们可以实现我们自己的日志和报告。TestNG提供了一种实现自定义报告和日志的方法。

Simply put, we can either implement the org.testng.ITestListener interface for logging or the org.testng.IReporter interface for reporting. These implemented classes get notified for events like start, end, failure etc of tests and suites.

简单地说，我们可以实现org.testng.ITestListener 接口来记录，或者实现org.testng.IReporter 接口来报告。这些实现的类在事件发生时得到通知，如测试和套件的开始、结束、失败等等。

Let’s go ahead and implement some simple custom logging:

让我们继续下去，实现一些简单的自定义日志。

public class CustomisedListener implements ITestListener {
    
    // ...
    @Override
    public void onFinish(ITestContext testContext) {
        LOGGER.info("PASSED TEST CASES");
        testContext.getPassedTests().getAllResults()
          .forEach(result -> {LOGGER.info(result.getName());});
        
        LOGGER.info("FAILED TEST CASES");
        testContext.getFailedTests().getAllResults()
          .forEach(result -> {LOGGER.info(result.getName());});
        
        LOGGER.info(
          "Test completed on: " + testContext.getEndDate().toString());
    }  
 
    //...
}

Notice how we’ve overridden the onFinish() method, which will be invoked when all test executions are complete and all configurations are done. Similarly, we can override other methods – such as onTestStart(), onTestFailure() etc (and find details about these other methods here).

请注意我们是如何覆盖onFinish() 方法的，当所有测试执行完成且所有配置完成时，该方法将被调用。类似地，我们可以覆盖其他方法–例如onTestStart(), onTestFailure() 等（并在这里找到关于这些其他方法的细节）。

Now let’s include this listener in the XML configuration:

现在让我们把这个监听器包括在XML配置中。

<suite name="My test suite">
    <listeners>
        <listener class-name="com.baeldung.reports.CustomisedListener" />
    </listeners>
    <test name="numbersXML">
        <parameter name="value" value="1" />
        <parameter name="isEven" value="false" />
        <classes>
            <class name="baeldung.com.ParametrizedTests" />
        </classes>
    </test>
</suite>

Once executed, the listener will be invoked on every event and will log information as we implemented. This might be useful for debugging our test execution.

一旦执行，监听器将在每个事件中被调用，并按照我们的实现记录信息。这对于调试我们的测试执行可能是有用的。

The output logs:

输出的记录。

...
INFO CUSTOM_LOGS - Started testing on: Sat Apr 22 14:39:43 IST 2017
INFO CUSTOM_LOGS - Testing: 
  givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect
INFO CUSTOM_LOGS - Tested: 
  givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect Time taken:6 ms
INFO CUSTOM_LOGS - Testing: 
  givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect
INFO CUSTOM_LOGS - Failed : 
  givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect
INFO CUSTOM_LOGS - PASSED TEST CASES
INFO CUSTOM_LOGS - givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect
INFO CUSTOM_LOGS - FAILED TEST CASES
INFO CUSTOM_LOGS - 
  givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect
INFO CUSTOM_LOGS - Test completed on: Sat Apr 22 14:39:43 IST 2017
...

The custom logs give us missing information in the default logs.

自定义日志给我们提供了默认日志中缺少的信息。

3. Custom Reports

3.自定义报告

When we run tests using the plugin, it generates reports in HTML/XML formats in target/surefire-reports directory:

当我们使用该插件运行测试时，它会在target/surefire-ports 目录下生成HTML/XML格式的报告。

test report

If we want to run a particular test suite using a TestNG XML file, we need to list it in the surefire-plugin configuration tag:

如果我们想使用TestNG XML文件来运行一个特定的测试套件，我们需要在surefire-plugin configuration tag中列出它。

<configuration>
    <suiteXmlFiles>
        <suiteXmlFile>
            src\test\resources\parametrized_testng.xml
        </suiteXmlFile>
    </suiteXmlFiles>
</configuration>

After custom logging, now let’s try to create some custom reports where we implement the org.testng.IReporter interface and override the generateReport() method:

在自定义日志之后，现在让我们尝试创建一些自定义报告，我们实现org.testng.IReporter接口并重写generateReport() 方法。

public void generateReport(
  List<XmlSuite> xmlSuites, 
  List<ISuite> suites, String outputDirectory) {
 
    String reportTemplate = initReportTemplate();

    String body = suites
      .stream()
      .flatMap(suiteToResults())
      .collect(Collectors.joining());

    String report
      = reportTemplate.replaceFirst("</tbody>", String.format("%s</tbody>", body));
    saveReportTemplate(outputDirectory, report);
}

The overridden method takes three arguments:

该重载方法需要三个参数。

  • xmlSuite – contains a list of all suites mentioned in the XML file
  • suites – a list object, holding all information about the test execution
  • outputDirectory – the directory path where reports are generated

We used the initReportTemplate() method to load an HTML template, suiteToResults() function which invokes the resultsToRow() function to handle the internals of generating the report:

我们使用initReportTemplate()方法来加载HTML模板，suiteToResults()函数调用resultsToRow()函数来处理生成报告的内部程序。

private Function<ISuite, Stream<? extends String>> suiteToResults() {
    return suite -> suite.getResults().entrySet()
      .stream()
      .flatMap(resultsToRows(suite));
}

private Function<Map.Entry<String, ISuiteResult>, 
  Stream<? extends String>> resultsToRows(ISuite suite) {
    return e -> {
        ITestContext testContext = e.getValue().getTestContext();

        Set<ITestResult> failedTests 
          = testContext.getFailedTests().getAllResults();
        Set<ITestResult> passedTests 
          = testContext.getPassedTests().getAllResults();
        Set<ITestResult> skippedTests 
          = testContext.getSkippedTests().getAllResults();

        String suiteName = suite.getName();

        return Stream
          .of(failedTests, passedTests, skippedTests)
          .flatMap(results ->
            generateReportRows(e.getKey(), suiteName, results).stream());
    };
}

and saveReportTemplate() for saving the complete result.

和saveReportTemplate()用于保存完整的结果。

Include the reporter in the XML configuration file:

在XML配置文件中包括报告人。

<suite name="suite">
    <listeners>
        <listener class-name="com.baeldung.reports.CustomisedReports" />
    </listeners>
    <test name="test suite">
        <classes>
	    <class name="baeldung.com.RegistrationTest" />
            <class name="baeldung.com.SignInTest" />
        </classes>
    </test>
</suite>

Here’s the output of our reports:

下面是我们的报告输出。

customReport

In comparison to the default sure-fire HTML report, this report gives a clear and crisp picture of the result, in a single table. Which is more convenient and easy to read.

与默认的肯定的HTML报告相比，这个报告给出了一个清晰明了的结果，在一个表格中。这更方便，更容易阅读。

4. Conclusion

4.总结

In this quick tutorial, we learned about how to generate test reports with Surefire Maven plugin. We also looked into customizing the logs and generating customized reports with TestNG. For more details about TestNG like how to write test cases, suites, etc. refer to our introductory

在这个快速教程中，我们了解了如何用Surefire Maven插件生成测试报告。我们还研究了自定义日志和用TestNG生成自定义报告。关于TestNG的更多细节，如如何编写测试用例、套件等，请参考我们的介绍性说明。

For more details about TestNG, such as how to write test cases, suites, definitely start with our introductory article.

关于TestNG的更多细节，如如何编写测试用例、套件，一定要从我们的介绍性文章开始。

As always the implementation of the snippets can be found over on GitHub.

像往常一样，这些片段的实现可以在GitHub上找到超过。