Storing Files Indexed by a Database – 存储由数据库索引的文件

最后修改: 2020年 11月 2日

中文/混合/英文(键盘快捷键:t)

1. Overview

1.概述

When we are building some sort of content management solution, we need to solve two problems. We need a place to store the files themselves, and we need some sort of database to index them.

当我们正在建立某种内容管理解决方案时,我们需要解决两个问题。我们需要一个地方来存储文件本身,我们需要某种数据库来索引它们。

It’s possible to store the content of the files in the database itself, or we could store the content somewhere else and index it with the database.

有可能将文件内容存储在数据库本身,或者我们可以将内容存储在其他地方,并与数据库建立索引。

In this article, we’re going to illustrate both of these methods with a basic Image Archive Application. We’ll also implement REST APIs for upload and download.

在这篇文章中,我们将用一个基本的图像存档应用程序来说明这两种方法。我们还将实现用于上传和下载的REST APIs。

2. Use Case

2.使用案例

Our Image Archive Application will allow us to upload and download JPEG images.

我们的图像存档应用程序将允许我们上传和下载JPEG图像

When we upload an image, the application will create a unique identifier for it. Then we can use this identifier to download it.

当我们上传图片时,应用程序将为其创建一个唯一的标识符。然后我们可以使用这个标识符来下载它。

We’ll use a relational database, with Spring Data JPA and Hibernate.

我们将使用一个关系型数据库,使用Spring Data JPAHibernate

3. Database Storage

3.数据库存储

Let’s start with our database.

让我们从我们的数据库开始。

3.1. Image Entity

3.1 图像实体

First, let’s create our Image entity:

首先,让我们创建我们的Image实体。

@Entity
class Image {

    @Id
    @GeneratedValue
    Long id;

    @Lob
    byte[] content;

    String name;
    // Getters and Setters
}

The id field is annotated with @GeneratedValue. This means the database will create a unique identifier for each record we add. By indexing the images with these values, we don’t need to worry about multiple uploads of the same image conflicting with each other.

id字段被注释为@GeneratedValue。这意味着数据库将为我们添加的每条记录创建一个唯一的标识符。通过用这些值为图片建立索引,我们不需要担心同一图片的多次上传会相互冲突。

Second, we have the Hibernate @Lob annotation. It’s how we tell JPA our intention of storing a potentially large binary.

其次,我们有Hibernate @Lob 注解。这是我们告诉JPA我们打算存储一个潜在的大二进制文件的方式。

3.2. Image Repository

3.2.图像存储库

Next, we need a repository to connect to the database.

接下来,我们需要一个存储库来连接到数据库

We’ll use the spring JpaRepository:

我们将使用spring JpaRepository

@Repository
interface ImageDbRepository extends JpaRepository<Image, Long> {}

Now we’re ready to save our images.  We just need a way to upload them to our application.

现在我们已经准备好保存我们的图像。 我们只需要一种方法将它们上传到我们的应用程序。

4. REST Controller

4.REST控制器

We will use a MultipartFile to upload our images. Uploading will return the imageId we can use to download the image later.

我们将使用MultipartFile来上传我们的图像。上传将返回imageId,我们可以在以后用来下载图片。

4.1. Image Upload

4.1 图像上传

Let’s start by creating our ImageController to support upload:

让我们开始创建我们的ImageController来支持上传。

@RestController
class ImageController {

    @Autowired
    ImageDbRepository imageDbRepository;

    @PostMapping
    Long uploadImage(@RequestParam MultipartFile multipartImage) throws Exception {
        Image dbImage = new Image();
        dbImage.setName(multipartImage.getName());
        dbImage.setContent(multipartImage.getBytes());

        return imageDbRepository.save(dbImage)
            .getId();
    }
}

The MultipartFile object contains the content and original name of the file. We use this to construct our Image object for storing in the database.

MultipartFile对象包含文件的内容和原始名称。我们用它来构造我们的Image对象,以便存储在数据库中。

This controller returns the generated id as the body of its response.

该控制器将生成的ID作为其响应的主体返回。

4.2. Image Download

4.2 图像下载

Now, let’s add a download route:

现在,让我们添加一个下载路由

@GetMapping(value = "/image/{imageId}", produces = MediaType.IMAGE_JPEG_VALUE)
Resource downloadImage(@PathVariable Long imageId) {
    byte[] image = imageRepository.findById(imageId)
      .orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND))
      .getContent();

    return new ByteArrayResource(image);
}

The imageId path variable contains the id that was generated at upload. If an invalid id is provided, then we’re using ResponseStatusException to return an HTTP response code 404 (Not Found). Otherwise, we’re wrapping the stored file bytes in a ByteArrayResource which allows them to be downloaded.

imageId路径变量包含上传时生成的id。如果提供了一个无效的ID,那么我们将使用ResponseStatusException来返回一个HTTP响应代码404(未找到)。否则,我们将存储的文件字节包裹在一个ByteArrayResource中,允许它们被下载。

5. Database Image Archive Test

5.数据库图像存档测试

Now we’re ready to test our Image Archive.

现在我们准备测试我们的图像存档。

First, let’s build our application:

首先,让我们建立我们的应用程序。

mvn package

Second, let’s start it up:

第二,让我们把它启动起来。

java -jar target/image-archive-0.0.1-SNAPSHOT.jar

5.1. Image Upload Test

5.1.图片上传测试

After our application is running, we’ll use the curl command-line tool to upload our image:

在我们的应用程序运行后,我们将使用curl命令行工具来上传我们的图像

curl -H "Content-Type: multipart/form-data" \
  -F "image=@baeldung.jpeg" http://localhost:8080/image

As the upload service response is the imageId, and this is our first request, the output will be:

由于上传服务的响应是imageId而这是我们的第一个请求,输出将是。

1

5.2. Image Download Test

5.2.图片下载测试

Then we can download our image:

然后我们可以下载我们的图像。

curl -v http://localhost:8080/image/1 -o image.jpeg

The -o image.jpeg option will create a file named image.jpeg and store the response content in it:

-o image.jpeg选项将创建一个名为image.jpeg的文件并在其中存储响应内容。

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /image/1 HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 
< Accept-Ranges: bytes
< Content-Type: image/jpeg
< Content-Length: 9291

We got an HTTP/1.1 200, which means that our download was successful.

我们得到一个HTTP/1.1 200,这意味着我们的下载成功了。

We could also try downloading the image in our browser by hitting http://localhost:8080/image/1.

我们也可以尝试在浏览器中通过点击http://localhost:8080/image/1来下载图片。

6. Separate Content and Location

6.内容和地点分开

So far, we’re capable of uploading and downloading images within a database.

到目前为止,我们有能力在数据库内上传和下载图片。

Another good option is uploading the file content to a different location. Then we save only its filesystem location in the DB.

另一个好的选择是将文件内容上传到一个不同的位置。然后我们保存DB中只有它的文件系统location

For that we’ll need to add a new field to our Image entity:

为此,我们需要为我们的Image实体添加一个新字段。

String location;

This will contain the logical path to the file in some external storage. In our case, it will be the path on our server’s filesystem. 

这将包含该文件在某个外部存储中的逻辑路径。在我们的例子中,它将是我们服务器的文件系统上的路径。

However, we can equally apply this idea to different Stores. For example, we could use cloud storage – Google Cloud Storage or Amazon S3. The location could also use a URI format, for example, s3://somebucket/path/to/file.

然而,我们同样可以将这个想法应用于不同的商店。例如,我们可以使用云存储 – Google云存储Amazon S3。位置也可以使用URI格式,例如,s3://somebucket/path/to/file

Our upload service, rather than writing the bytes of the file to the database, will store the file in the appropriate service – in this case, the filesystem – and will then put the location of the file into the database.

我们的上传服务,不是将文件的字节写入数据库,而是将文件存储在适当的服务中–在这种情况下是文件系统–然后将文件的位置放入数据库。

7. Filesystem Storage

7.文件系统存储

Let’s add the capability to store the images in the filesystem to our solution.

让我们在我们的解决方案中添加在文件系统中存储图像的能力

7.1. Saving in the Filesystem

7.1.保存在文件系统中

First, we need to save our images to the filesystem:

首先,我们需要将我们的图像保存到文件系统。

@Repository
class FileSystemRepository {

    String RESOURCES_DIR = FileSystemRepository.class.getResource("/")
        .getPath();

    String save(byte[] content, String imageName) throws Exception {
        Path newFile = Paths.get(RESOURCES_DIR + new Date().getTime() + "-" + imageName);
        Files.createDirectories(newFile.getParent());

        Files.write(newFile, content);

        return newFile.toAbsolutePath()
            .toString();
    }
}

One important note – we need to make sure that each of our images has a unique location defined server-side at upload time. Otherwise, our uploads may overwrite each other.

一个重要的注意点 – 我们需要确保我们的每张图片在上传时有一个独特的location 服务器端定义的。否则,我们的上传可能会相互覆盖。

The same rule would apply to any cloud storage, where we should create unique keys. In this example, we’ll add the current date in milliseconds format to the image name:

同样的规则也适用于任何云存储,我们应该创建唯一的密钥。在这个例子中,我们将把当前日期以毫秒格式添加到图像名称中。

/workspace/archive-achive/target/classes/1602949218879-baeldung.jpeg

7.2. Retrieving From Filesystem

7.2.从文件系统检索

Now let’s implement the code to fetch our image from the filesystem:

现在让我们实现代码,从文件系统中获取我们的图像。

FileSystemResource findInFileSystem(String location) {
    try {
        return new FileSystemResource(Paths.get(location));
    } catch (Exception e) {
        // Handle access or file not found problems.
        throw new RuntimeException();
    }
}

Here we’re looking for the image using its location. Then we return a FileSystemResource.

这里我们使用图片的location来寻找图片。然后我们返回一个FileSystemResource

Also, we’re catching any exception that may happen while reading our file. We might also wish to throw exceptions with particular HTTP statuses.

另外,我们要捕捉读取文件时可能发生的任何异常。我们也可能希望抛出带有特定HTTP状态的异常。

7.3. Data Streaming and Spring’s Resource

7.3.数据流和Spring的资源

Our findInFileSystem method returns a FileSystemResource, an implementation of Spring’s Resource interface.

我们的 findInFileSystem 方法返回 FileSystemResourceSpring 的 Resource 接口的实现。

It will start reading our file only when we use it. In our case, it’ll be when sending it to the client via the RestController. Also, it’ll stream the file content from the filesystem to the user, saving us from loading all the bytes into memory.

只有当我们使用它时,它才会开始读取我们的文件。在我们的例子中,就是在通过RestController将其发送给客户端时。此外,它还会将文件内容从文件系统中流向用户,省去了我们将所有字节载入内存的麻烦

This approach is a good general solution for streaming files to a client. If we’re using cloud storage instead of the filesystem, we can replace the FileSystemResource for another resource’s implementation, like the InputStreamResource or ByteArrayResource.

这种方法是流式传输文件到客户端的一个很好的通用解决方案。如果我们使用云存储而不是文件系统,我们可以将FileSystemResource替换为其他资源的实现,例如InputStreamResourceByteArrayResource

8. Connecting the File Content and Location

8.连接文件内容和位置

Now that we have our FileSystemRepository, we need to link it with our ImageDbRepository.

现在我们有了FileSystemRepository,我们需要将它与ImageDbRepository连接起来。

8.1. Saving in the Database and Filesystem

8.1.保存在数据库和文件系统中

Let’s create a FileLocationService, starting with our save flow:

让我们创建一个FileLocationService,从我们的保存流程开始。

@Service
class FileLocationService {

    @Autowired
    FileSystemRepository fileSystemRepository;
    @Autowired
    ImageDbRepository imageDbRepository;

    Long save(byte[] bytes, String imageName) throws Exception {
        String location = fileSystemRepository.save(bytes, imageName);

        return imageDbRepository.save(new Image(imageName, location))
            .getId();
    }
}

First, we save the image in the filesystem. Then we save the record containing its location in the database.

首先,我们将图像保存在文件系统中。然后,我们在数据库中保存包含其location的记录

8.2. Retrieving From Database and Filesystem

8.2.从数据库和文件系统检索

Now, let’s create a method to find our image using its id:

现在,让我们创建一个方法,使用它的id找到我们的图像。

FileSystemResource find(Long imageId) {
    Image image = imageDbRepository.findById(imageId)
      .orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND));

    return fileSystemRepository.findInFileSystem(image.getLocation());
}

First, we look for our image in the database. Then we get its location and fetch it from the filesystem.

首先,我们在数据库中寻找我们的图像。然后我们得到它的位置并从文件系统中获取它

If we don’t find the imageId in the database, we’re using ResponseStatusException to return an HTTP Not Found response.

如果我们在数据库中没有找到imageId,我们就使用ResponseStatusException来返回一个HTTP Not Found响应.

9. Filesystem Upload and Download

9.文件系统的上传和下载

Finally, let’s create the FileSystemImageController:

最后,我们来创建FileSystemImageController:

@RestController
@RequestMapping("file-system")
class FileSystemImageController {

    @Autowired
    FileLocationService fileLocationService;

    @PostMapping("/image")
    Long uploadImage(@RequestParam MultipartFile image) throws Exception {
        return fileLocationService.save(image.getBytes(), image.getOriginalFilename());
    }

    @GetMapping(value = "/image/{imageId}", produces = MediaType.IMAGE_JPEG_VALUE)
    FileSystemResource downloadImage(@PathVariable Long imageId) throws Exception {
        return fileLocationService.find(imageId);
    }
}

First, we made our new path start with “/file-system“.

首先,我们让我们的新路径以”/file-system“开始。

Then we created the upload route similar to that in our ImageController, but without the dbImage object.

然后我们创建了类似于ImageController中的上传路线,但没有dbImage对象。

Lastly, we have our download route, which uses the FileLocationService to find the image and returns the FileSystemResource as the HTTP response.

最后,我们的下载路由使用FileLocationService来查找图片,并将FileSystemResource作为HTTP响应返回。

10. Filesystem Image Archive Test

10.文件系统图像存档测试

Now, we can test our filesystem version the same way we did with our database version, though the paths now start with “file-system“:

现在,我们可以像测试数据库版本一样测试我们的文件系统版本,尽管现在的路径以”file-system“开头。

curl -H "Content-Type: multipart/form-data" \
  -F "image=@baeldung.jpeg" http://localhost:8080/file-system/image

1

And then we download:

然后我们下载。

curl -v http://localhost:8080/file-system/image/1 -o image.jpeg

11. Conclusion

11.结语

In this article, we learned how to save file information in a database, with the file content either in the same row or in an external location.

在这篇文章中,我们学习了如何在数据库中保存文件信息,文件内容可以在同一行,也可以在一个外部位置。

We also built and tested a REST API using multipart upload, and we provided a download feature using Resource to allow streaming the file to the caller.

我们还建立并测试了一个使用多部分上传的REST API,我们还使用Resource提供了一个下载功能,允许将文件流式传输给调用者。

As always, the code samples can be found over on GitHub.

一如既往,代码样本可以在GitHub上找到over