Configuring Skip Logic in Spring Batch – 在Spring Batch中配置跳过逻辑

最后修改: 2019年 10月 11日

中文/混合/英文(键盘快捷键：t)

1. Introduction

1.绪论

By default, any errors encountered during a Spring Batch job processing will make a corresponding step fail. However, there are many situations where we’d rather like to skip the currently processed item for certain exceptions.

默认情况下，在Spring Batch作业处理过程中遇到的任何错误都会使相应的步骤失败。然而，在很多情况下，我们更希望跳过当前处理的项目来处理某些异常情况。

In this tutorial, we’ll explore two approaches to configure skip logic in the Spring Batch framework.

在本教程中，我们将探讨在Spring Batch框架中配置跳过逻辑的两种方法。

2. Our Use Case

2.我们的用例

For the purpose of examples, we’ll reuse a simple, chunk-oriented job presented already in our Spring Batch introductory article.

为了举例说明，我们将重新使用我们的Spring Batch介绍性文章中已经介绍的一个简单的、面向分块的工作。

This job converts some financial data from a CSV to XML format.

这项工作将一些财务数据从CSV格式转换为XML格式。

2.1. Input Data

2.1.输入数据

First, let’s add a few rows to the original CSV file:

首先，让我们在原始CSV文件中增加几行。

username, user_id, transaction_date, transaction_amount
devendra, 1234, 31/10/2015, 10000
john, 2134, 3/12/2015, 12321
robin, 2134, 2/02/2015, 23411
, 2536, 3/10/2019, 100
mike, 9876, 5/11/2018, -500
, 3425, 10/10/2017, 9999

As we can see, the last three rows contain some invalid data – rows 5 and 7 are missing the username field, and the transaction amount in row 6 is negative.

我们可以看到，最后三行包含一些无效的数据–第5行和第7行缺少用户名字段，第6行的交易金额是负数。

In the later sections, we’ll configure our batch job to skip these corrupted records.

在后面的章节中，我们将配置我们的批处理作业以跳过这些损坏的记录。

3. Configuring Skip Limit and Skippable Exceptions

3.配置跳过限制和可跳过的例外情况

3.1. Using skip and skipLimit

3.1.使用skip和skipLimit

Let’s now discuss the first of two ways to configure our job to skip items in case of a failure — the skip and skipLimit methods:

现在让我们来讨论配置我们的作业在失败时跳过项目的两种方法中的第一种–skip和skipLimit方法。

@Bean
public Step skippingStep(
  ItemProcessor<Transaction, Transaction> processor,
  ItemWriter<Transaction> writer) throws ParseException {
    return stepBuilderFactory
      .get("skippingStep")
      .<Transaction, Transaction>chunk(10)
      .reader(itemReader(invalidInputCsv))
      .processor(processor)
      .writer(writer)
      .faultTolerant()
      .skipLimit(2)
      .skip(MissingUsernameException.class)
      .skip(NegativeAmountException.class)
      .build();
}

First of all, to enable skip functionality, we need to include a call to faultTolerant() during the step-building process.

首先，为了启用跳过功能，我们需要在建立步骤的过程中加入对faultTolerant()的调用。

Within skip() and skipLimit(), we define the exceptions we want to skip and the maximum number of skipped items.

在skip()和skipLimit()中，我们定义了我们想要跳过的异常和跳过的最大项目数。

In the above example, if either a MissingUsernameException or NegativeAmountException is thrown during read, process, or write phase, then the currently processed item will be omitted and counted against the total limit of two.

在上面的例子中，如果在读、处理或写阶段抛出MissingUsernameException或NegativeAmountException，那么当前处理的项目将被省略，并计入总的限制为2。

Consequently, if any exception is thrown a third time, then the whole step will fail.

因此，如果第三次抛出任何异常，那么整个步骤将失败。

3.1. Using noSkip

3.1.使用noSkip

In the previous example, any other exception besides MissingUsernameException and NegativeAmountException makes our step fail.

在前面的例子中，除了MissingUsernameException和NegativeAmountException之外，任何其他异常都会使我们的步骤失败。

In some situations, however, it may be more appropriate to identify exceptions that should make our step fail and skip on any other.

然而，在某些情况下，可能更合适的做法是识别应该使我们的步骤失败的例外情况，并跳过任何其他步骤。

Let’s see how we can configure this using skip, skipLimit, and noSkip:

让我们看看如何使用skip、skipLimit和noSkip进行配置。

@Bean
public Step skippingStep(
  ItemProcessor<Transaction, Transaction> processor,
  ItemWriter<Transaction> writer) throws ParseException {
    return stepBuilderFactory
      .get("skippingStep")
      .<Transaction, Transaction>chunk(10)
      .reader(itemReader(invalidInputCsv))
      .processor(processor)
      .writer(writer)
      .faultTolerant()
      .skipLimit(2)
      .skip(Exception.class)
      .noSkip(SAXException.class)
      .build();
}

With the above configuration, we instruct Spring Batch framework to skip on any Exception (within a configured limit) except SAXException. This means SAXException always causes a step failure.

通过上述配置，我们指示Spring Batch框架跳过任何Exception（在配置的限度内），除了SAXException。这意味着SAXException总是导致步骤失败。

The order of the skip() and noSkip() calls doesn’t matter.

skip()和noSkip()的调用顺序并不重要。

4. Using Custom SkipPolicy

4.使用自定义SkipPolicy

Sometimes we may need a more sophisticated skip-checking mechanism. For that purpose, Spring Batch framework provides the SkipPolicy interface.

有时我们可能需要一个更复杂的跳过检查机制。为此，Spring Batch框架提供了SkipPolicy接口。。

We can then provide our own implementation of skip logic and plug it into our step definition.

然后我们可以提供我们自己的跳过逻辑的实现，并将其插入我们的步骤定义中。

Keeping the preceding example in mind, imagine we still want to define a skip limit of two items and make only MissingUsernameException and NegativeAmountException skippable.

记住前面的例子，想象一下我们仍然想定义一个跳过的限制，即两个项目，并且只让MissingUsernameException和NegativeAmountException可跳。

However, an additional constraint is that we can skip NegativeAmountException, but only if the amount doesn’t exceed a defined limit.

然而，一个额外的约束是，我们可以跳过NegativeAmountException,，但前提是金额不超过定义的限制。

Let’s implement our custom SkipPolicy:

让我们来实现我们的自定义SkipPolicy。

public class CustomSkipPolicy implements SkipPolicy {

    private static final int MAX_SKIP_COUNT = 2;
    private static final int INVALID_TX_AMOUNT_LIMIT = -1000;

    @Override
    public boolean shouldSkip(Throwable throwable, int skipCount) 
      throws SkipLimitExceededException {

        if (throwable instanceof MissingUsernameException && skipCount < MAX_SKIP_COUNT) {
            return true;
        }

        if (throwable instanceof NegativeAmountException && skipCount < MAX_SKIP_COUNT ) {
            NegativeAmountException ex = (NegativeAmountException) throwable;
            if(ex.getAmount() < INVALID_TX_AMOUNT_LIMIT) {
                return false;
            } else {
                return true;
            }
        }

        return false;
    }
}

Now, we can use our custom policy in a step definition:

现在，我们可以在一个步骤定义中使用我们的自定义策略。

    @Bean
    public Step skippingStep(
      ItemProcessor<Transaction, Transaction> processor,
      ItemWriter<Transaction> writer) throws ParseException {
        return stepBuilderFactory
          .get("skippingStep")
          .<Transaction, Transaction>chunk(10)
          .reader(itemReader(invalidInputCsv))
          .processor(processor)
          .writer(writer)
          .faultTolerant()
          .skipPolicy(new CustomSkipPolicy())
          .build();
    }

And, similarly to our previous example, we still need to use faultTolerant() to enable skip functionality.

而且，与我们之前的例子类似，我们仍然需要使用faultTolerant()来启用跳过功能。

This time, however, we do not call skip() or noSkip(). Instead, we use the skipPolicy() method to provide our own implementation of the SkipPolicy interface.

然而，这一次，我们没有调用skip()或noSkip()。相反，我们使用skipPolicy()方法来提供我们自己的SkipPolicy接口的实现。

As we can see, this approach gives us more flexibility, so it can be a good choice in certain use cases.

正如我们所看到的，这种方法为我们提供了更多的灵活性，所以在某些用例中，它可能是一个不错的选择。

5. Conclusion

5.总结

In this tutorial, we presented two ways to make a Spring Batch job fault-tolerant.

在本教程中，我们介绍了两种使Spring Batch作业容错的方法。

Even though using a skipLimit() together with skip() and noSkip() methods seems to be more popular, we may find implementing a custom SkipPolicy to be more convenient in some situations.

尽管与skip()和noSkip()方法一起使用skipLimit()似乎更受欢迎，但我们可能发现在某些情况下实现自定义SkipPolicy更方便。

As usual, all the code examples are available over on GitHub.

像往常一样，所有的代码实例都可以在GitHub上找到。