1. Overview
1.概述
In this tutorial, we’re going to look into distributed background job scheduling and processing in Java using JobRunr and have it integrate with Spring.
在本教程中,我们将使用JobRunr研究Java中的分布式后台作业调度和处理,并让它与Spring集成。
2. About JobRunr
2.关于JobRunr
JobRunr is a library that we can embed in our application and which allows us to schedule background jobs using a Java 8 lambda. We can use any existing method of our Spring services to create a job without the need to implement an interface. A job can be a short or long-running process, and it will be automatically offloaded to a background thread so that the current web request is not blocked.
JobRunr是一个我们可以嵌入到应用程序中的库,它允许我们使用 Java 8 lambda 安排后台作业。我们可以使用 Spring 服务的任何现有方法来创建作业,而无需实现接口。一个作业可以是一个短期或长期运行的进程,它将被自动卸载到一个后台线程中,这样当前的Web请求就不会被阻塞。
To do its job, JobRunr analyses the Java 8 lambda. It serializes it as JSON, and stores it into either a relational database or a NoSQL data store.
为了完成其工作,JobRunr分析了Java 8 lambda。它将其序列化为JSON,并将其存储到一个关系数据库或一个NoSQL数据存储中。
3. JobRunr Features
3.JobRunr的特点
If we see that we’re producing too many background jobs and our server can not cope with the load, we can easily scale horizontally by just adding extra instances of our application. JobRunr will share the load automatically and distribute all jobs over the different instances of our application.
如果我们发现我们产生了太多的后台作业,而我们的服务器无法应付这些负载,我们可以很容易地横向扩展,只需增加我们应用程序的额外实例。JobRunr将自动分担负载,并将所有工作分配到我们应用程序的不同实例中。
It also contains an automatic retry feature with an exponential back-off policy for failed jobs. There is also a built-in dashboard that allows us to monitor all jobs. JobRunr is self-maintaining – succeeded jobs will automatically be deleted after a configurable amount of time so there is no need to perform manual storage cleanup.
它还包含一个自动重试功能,对失败的作业采用指数退避策略。还有一个内置的仪表板,使我们能够监控所有的工作。JobRunr是自我维护的–成功的作业将在可配置的时间后自动删除,因此不需要执行手动存储清理。
4. Setup
4.设置
For the sake of simplicity, we’ll use an in-memory data store to store all job-related information.
为了简单起见,我们将使用一个内存数据存储来存储所有与工作相关的信息。
4.1. Maven Configuration
4.1.Maven配置
Let’s jump straight to the Java code. But before that, we need to have the following Maven dependency declared in our pom.xml file:
让我们直接跳到Java代码。但在此之前,我们需要在pom.xml文件中声明以下Maven依赖项。
<dependency>
<groupId>org.jobrunr</groupId>
<artifactId>jobrunr-spring-boot-starter</artifactId>
<version>3.1.2</version>
</dependency>
4.2. Spring Integration
4.2.Spring集成
Before we jump straight to how to create background jobs, we need to initialize JobRunr. As we’re using the jobrunr-spring-boot-starter dependency, this is easy. We only need to add some properties to the application.properties:
在我们直接跳到如何创建后台作业之前,我们需要初始化JobRunr。由于我们使用的是jobrunr-spring-boot-starter依赖项,这很容易。我们只需要在application.properties中添加一些属性。
org.jobrunr.background-job-server.enabled=true
org.jobrunr.dashboard.enabled=true
The first property tells JobRunr that we want to start an instance of a BackgroundJobServer that is responsible for processing jobs. The second property tells JobRunr to start the embedded dashboard.
第一个属性告诉JobRunr,我们要启动一个BackgroundJobServer的实例,它负责处理作业。第二个属性告诉JobRunr要启动嵌入式仪表盘。
By default, the jobrunr-spring-boot-starter will try to use your existing DataSource in case of a relational database to store all the job-related information.
默认情况下,jobrunr-spring-boot-starter将尝试使用您现有的DataSource(如果是关系型数据库)来存储所有与作业相关的信息。
However, since we’ll use an in-memory data store, we need to provide a StorageProvider bean:
然而,由于我们将使用一个内存数据存储,我们需要提供一个StorageProvider Bean。
@Bean
public StorageProvider storageProvider(JobMapper jobMapper) {
InMemoryStorageProvider storageProvider = new InMemoryStorageProvider();
storageProvider.setJobMapper(jobMapper);
return storageProvider;
}
5. Usage
5.使用方法
Now, let’s find out how to create and schedule background jobs in Spring using JobRunr.
现在,让我们来看看如何使用JobRunr在Spring中创建和安排后台作业。
5.1. Inject Dependencies
5.1.注入依赖关系
When we want to create jobs, we’ll need to inject the JobScheduler and our existing Spring service containing the method for which we want to create jobs, in this case, the SampleJobService:
当我们想要创建工作时,我们需要注入JobScheduler和我们现有的Spring服务,其中包含我们想要创建工作的方法,在本例中是SampleJobService。
@Inject
private JobScheduler jobScheduler;
@Inject
private SampleJobService sampleJobService;
The JobScheduler class from JobRunr allows us to enqueue or schedule new background jobs.
来自JobRunr的JobScheduler类使我们能够列队或安排新的后台作业。
The SampleJobService could be any of our existing Spring services containing a method that might take too long to handle in a web request. It can also be a method that calls some other external services where we want to add resilience as JobRunr will retry the method if an exception occurs.
SampleJobService可以是我们现有的任何一个Spring服务,其中包含一个可能需要太长时间才能在Web请求中处理的方法。它也可以是一个调用其他外部服务的方法,在那里我们希望增加弹性,因为如果发生异常,JobRunr将重试该方法。
5.2. Creating Fire-and-Forget Jobs
5.2 创建火烧眉毛的工作
Now that we have our dependencies, we can create fire-and-forget jobs using the enqueue method:
现在我们有了我们的依赖,我们可以使用enqueue方法创建fire-and-forget作业。
jobScheduler.enqueue(() -> sampleJobService.executeSampleJob());
Jobs can have parameters, just like any other lambda:
工作可以有参数,就像其他的lambda一样。
jobScheduler.enqueue(() -> sampleJobService.executeSampleJob("some string"));
This line makes sure that the lambda – including type, method, and arguments – is serialized as JSON to persistent storage (an RDBMS like Oracle, Postgres, MySql, and MariaDB or a NoSQL database).
这一行确保lambda–包括类型、方法和参数–被序列化为JSON到持久性存储(RDBMS如Oracle、Postgres、MySql和MariaDB或NoSQL数据库)。
A dedicated worker pool of threads running in all the different BackgroundJobServers will then execute these queued background jobs as soon as possible, in a first-in-first-out manner. JobRunr guarantees the execution of your job by a single worker by means of optimistic locking.
在所有不同的BackgroundJobServers中运行的专用线程池将以先入先出的方式尽快执行这些排队的后台作业。JobRunr通过乐观锁定的方式保证你的工作由一个工人执行。
5.3. Scheduling Jobs in the Future
5.3.安排未来的工作
We can also schedule jobs in the future using the schedule method:
我们还可以使用schedule方法在未来安排工作。
jobScheduler.schedule(LocalDateTime.now().plusHours(5), () -> sampleJobService.executeSampleJob());
5.4. Scheduling Jobs Recurrently
5.4.经常性地调度工作
If we want to have recurrent jobs, we need to use the scheduleRecurrently method:
如果我们想有重复性工作,我们需要使用scheduleRecurrently方法。
jobScheduler.scheduleRecurrently(Cron.hourly(), () -> sampleJobService.executeSampleJob());
5.5. Annotating with the @Job Annotation
5.5.用@Job注解进行注解
To control all aspects of a job, we can annotate our service method with the @Job annotation. This allows setting the display name in the dashboard and configuring the number of retries in case a job fails.
为了控制作业的所有方面,我们可以用@Job注解来注释我们的服务方法。这允许在仪表板中设置显示名称,并配置作业失败时的重试次数。
@Job(name = "The sample job with variable %0", retries = 2)
public void executeSampleJob(String variable) {
...
}
We can even use variables that are passed to our job in the display name by means of the String.format() syntax.
我们甚至可以通过String.format()语法,在显示名称中使用传递给我们工作的变量。
If we have very specific use cases where we would want to retry a specific job only on a certain exception, we can write our own ElectStateFilter where we have access to the Job and full control on how to proceed.
如果我们有非常具体的用例,我们只想在某个异常情况下重试一个特定的工作,我们可以编写我们自己的ElectStateFilter,我们可以访问Job并完全控制如何进行。
6. Dashboard
6. 仪表板
JobRunr comes with a built-in dashboard that allows us to monitor our jobs. We can find it at http://localhost:8000 and inspect all the jobs, including all recurring jobs and an estimation of how long it will take until all the enqueued jobs are processed:
JobRunr带有一个内置的仪表板,允许我们监控我们的工作。我们可以在http://localhost:8000找到它,并检查所有的作业,包括所有的重复性作业和对所有排队作业被处理前所需时间的估计。
Bad things can happen, for example, an SSL certificate expired, or a disk is full. JobRunr, by default, will reschedule the background job with an exponential back-off policy. If the background job continues to fail ten times, only then will it go to the Failed state. You can then decide to re-queue the failed job when the root cause has been solved.
不好的事情可能发生,例如,SSL证书过期了,或者磁盘满了。JobRunr,默认情况下,将以指数级的回退策略重新安排后台作业。如果后台作业继续失败十次,那么它才会进入Failed状态。然后你可以决定在解决了根本原因后,重新排定失败的作业。
All of this is visible in the dashboard, including each retry with the exact error message and the complete stack trace of why a job failed:
所有这些都可以在仪表板上看到,包括每次重试的确切错误信息和作业失败的完整堆栈跟踪。
7. Conclusion
7.结语
In this article, we built our first basic scheduler using JobRunr with the jobrunr-spring-boot-starter. The key takeaway from this tutorial is that we were able to create a job with just one line of code and without any XML-based configuration or the need to implement an interface.
在这篇文章中,我们使用JobRunr和jobrunr-spring-boot-starter构建了第一个基本调度器。本教程的主要收获是,我们只用一行代码就能创建一个作业,而且不需要任何基于XML的配置,也不需要实现一个接口。
The complete source code for the example is available over on GitHub.
该示例的完整源代码可在GitHub上获得,。