Download a Large File Through a Spring RestTemplate – 通过Spring RestTemplate下载一个大文件

最后修改: 2019年 6月 16日

中文/混合/英文(键盘快捷键：t)

1. Overview

1.概述

In this tutorial, we’re going to show different techniques on how to download large files with RestTemplate.

在本教程中，我们将展示如何用RestTemplate下载大文件的不同技巧。

2. RestTemplate

2.RestTemplate

RestTemplate is a blocking and synchronous HTTP Client introduced in Spring 3. According to the Spring documentation, it’ll be deprecated in the future since they’ve introduced WebClient as a reactive nonblocking HTTP client in version 5.

RestTemplate是Spring 3中引入的一个阻塞和同步的HTTP客户端。根据Spring文档，由于他们在第5版中引入了WebClient作为一个反应式非阻塞HTTP客户端，所以它将在未来被废弃。

3. Pitfalls

3. 陷阱

Usually, when we download a file, we store it on our file system or load it into memory as a byte array. But when it’s a large file, in-memory loading may lead to an OutOfMemoryError. Hence, we have to store data in a file as we read chunks of response.

通常，当我们下载一个文件时，我们会把它存储在我们的文件系统中，或者把它作为一个字节数组加载到内存中。但是当它是一个大文件时，内存加载可能会导致OutOfMemoryError。因此，我们必须在读取大块的响应时将数据存储在文件中。

Let’s first look at a couple of ways that don’t work:

让我们先看一下几个不起作用的方法。

First, what happens if we return a Resource as our return type:

首先，如果我们返回一个Resource作为我们的返回类型会发生什么。

Resource download() {
    return new ClassPathResource(locationForLargeFile);
}

The reason this doesn’t work is that ResourceHttpMesssageConverter will load the entire response body into a ByteArrayInputStream still adding the memory pressure we wanted to avoid.

这样做不行的原因是，ResourceHttpMesssageConverter会将整个响应体加载到ByteArrayInputStream中，仍然会增加我们想要避免的内存压力。

Second, what if we return an InputStreamResource and configure ResourceHttpMessageConverter#supportsReadStreaming? Well, this doesn’t work either since by the time we can call  InputStreamResource.getInputStream(), we get a “socket closed” error! This is because the “execute” closes the response input stream before the exit.

第二，如果我们返回一个InputStreamResource并配置ResourceHttpMessageConverter#supportsReadStreaming呢？好吧，这也不行，因为当我们可以调用InputStreamResource.getInputStream()时，我们得到一个”socket closed”错误！这是因为“execute“在退出前关闭了响应输入流。

So what can we do to solve the problem? Actually, there are two things here, too:

那么，我们能做什么来解决这个问题呢？其实，这里也有两件事。

  • Write a custom HttpMessageConverter that supports File as a return type
  • Use RestTemplate.execute with a custom ResponseExtractor to store the input stream in a File

In this tutorial, we’ll use the second solution because it is more flexible and also needs less effort.

在本教程中，我们将使用第二种解决方案，因为它更灵活，也需要更少的努力。

4. Download Without Resume

4.下载无简历

Let’s implement a ResponseExtractor to write the body to a temporary file:

让我们实现一个ResponseExtractor，将正文写入一个临时文件：。

File file = restTemplate.execute(FILE_URL, HttpMethod.GET, null, clientHttpResponse -> {
    File ret = File.createTempFile("download", "tmp");
    StreamUtils.copy(clientHttpResponse.getBody(), new FileOutputStream(ret));
    return ret;
});

Assert.assertNotNull(file);
Assertions
  .assertThat(file.length())
  .isEqualTo(contentLength);

Here we have used the StreamUtils.copy to copy the response input stream in a FileOutputStream, but other techniques and libraries are also available.

这里我们使用了StreamUtils.copy来复制响应的输入流在FileOutputStream中，但其他技术和库也可以使用。

5. Download with Pause and Resume

5.带暂停和恢复功能的下载

As we’re going to download a large file, it’s reasonable to consider downloading after we’ve paused for some reason.

由于我们要下载一个大文件，所以考虑在我们因某种原因暂停后再下载是合理的。

So first let’s check if the download URL supports resume:

因此，首先让我们检查一下下载的URL是否支持简历。

HttpHeaders headers = restTemplate.headForHeaders(FILE_URL);

Assertions
  .assertThat(headers.get("Accept-Ranges"))
  .contains("bytes");
Assertions
  .assertThat(headers.getContentLength())
  .isGreaterThan(0);

Then we can implement a RequestCallback to set “Range” header and resume the download:

然后我们可以实现一个RequestCallback来设置 “Range “头并恢复下载。

restTemplate.execute(
  FILE_URL,
  HttpMethod.GET,
  clientHttpRequest -> clientHttpRequest.getHeaders().set(
    "Range",
    String.format("bytes=%d-%d", file.length(), contentLength)),
    clientHttpResponse -> {
        StreamUtils.copy(clientHttpResponse.getBody(), new FileOutputStream(file, true));
    return file;
});

Assertions
  .assertThat(file.length())
  .isLessThanOrEqualTo(contentLength);

If we don’t know the exact content length, we can set the Range header value using String.format:

如果我们不知道确切的内容长度，我们可以使用String.format设置Range头的值。

String.format("bytes=%d-", file.length())

6. Conclusion

6.结论

We’ve discussed problems that can arise when downloading a large file. We also presented a solution while using RestTemplate. Finally, we’ve shown how we can implement a resumable download.

我们已经讨论了下载大文件时可能出现的问题。我们还提出了一个解决方案，同时使用RestTemplate。最后，我们展示了如何实现一个可恢复的下载。

As always the code is available in our GitHub.

像往常一样，代码可以在我们的GitHub中找到。