1. Introduction
1.绪论
In this tutorial, we’ll learn how to read JSON data from files and import them into MongoDB using Spring Boot. This can be useful for many reasons: restoring data, bulk inserting new data, or inserting default values. MongoDB uses JSON internally to structure its documents, so naturally, that’s what we’ll use to store importable files. Being plain text, this strategy also has the advantage of being easily compressible.
在本教程中，我们将学习如何从文件中读取 JSON 数据并使用 Spring Boot 将它们导入 MongoDB。这在很多情况下都很有用：恢复数据、批量插入新数据或插入默认值。MongoDB 在内部使用 JSON 来构建其文档，因此，我们自然会使用这种方法来存储可导入的文件。作为纯文本，该策略还具有易于压缩的优势。
Moreover, we’ll learn how to validate our input files against our custom types when necessary. Finally, we’ll expose an API so we can use it during runtime in our web app.
此外，我们将学习如何在必要时根据我们的自定义类型验证我们的输入文件。最后，我们将公开一个API，这样我们就可以在网络应用的运行期间使用它。
2. Dependencies
2.依赖性
Let’s add these Spring Boot dependencies to our pom.xml:
让我们把这些Spring Boot依赖项添加到我们的pom.xml中。
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>We’re also going to need a running instance of MongoDB, which requires a properly configured application.properties file.
我们还需要一个运行中的MongoDB实例，这需要一个正确配置的application.properties文件。
3. Importing JSON Strings
3.导入JSON字符串
The simplest way to import JSON into MongoDB is to convert it into an “org.bson.Document” object first. This class represents a generic MongoDB document of no specific type. Therefore we don’t have to worry about creating repositories for all the kinds of objects we might import.
将 JSON 导入 MongoDB 的最简单方法是首先将其转换为”org.bson.Document“对象。该类代表没有特定类型的通用 MongoDB 文档。因此，我们不必担心为我们可能导入的所有类型的对象创建存储库。
Our strategy takes JSON (from a file, resource, or string), converts it into Documents, and saves them using MongoTemplate. Batch operations generally perform better since the amount of round trips is reduced compared to inserting each object individually.
我们的策略采用 JSON（来自文件、资源或字符串），将其转换为Documents，并使用MongoTemplate保存它们。批量操作通常表现更好，因为与单独插入每个对象相比，往返的次数减少了。
Most importantly, we’ll consider our input to have only one JSON object per line break. That way, we can easily delimiter our objects. We’ll encapsulate these functionalities into two classes that we’ll create: ImportUtils and ImportJsonService. Let’s start with our service class:
最重要的是，我们会认为我们的输入在每一行中只有一个JSON对象。这样，我们可以很容易地给我们的对象定界。我们将把这些功能封装在我们将创建的两个类中。ImportUtils和ImportJsonService。让我们从我们的服务类开始。
@Service
public class ImportJsonService {
@Autowired
private MongoTemplate mongo;
}Next, let’s add a method that parses lines of JSON into documents:
接下来，让我们添加一个方法，将几行JSON解析成文档。
private List<Document> generateMongoDocs(List<String> lines) {
List<Document> docs = new ArrayList<>();
for (String json : lines) {
docs.add(Document.parse(json));
}
return docs;
}Then we add a method that inserts a list of Document objects into the desired collection. Also, it’s possible the batch operation partially fails. In that case, we can return the number of inserted documents by checking the cause of the exception:
然后我们添加一个方法，将一个Document对象的列表插入所需的collection中。另外，批处理操作有可能部分失败。在这种情况下，我们可以通过检查exception的cause来返回插入的文档的数量。
private int insertInto(String collection, List<Document> mongoDocs) {
try {
Collection<Document> inserts = mongo.insert(mongoDocs, collection);
return inserts.size();
} catch (DataIntegrityViolationException e) {
if (e.getCause() instanceof MongoBulkWriteException) {
return ((MongoBulkWriteException) e.getCause())
.getWriteResult()
.getInsertedCount();
}
return 0;
}
}Finally, let’s combine those methods. This one takes the input and returns a string showing how many lines were read vs. successfully inserted:
最后，让我们结合这些方法。这个方法接受输入并返回一个字符串，显示有多少行被读取与成功插入。
public String importTo(String collection, List<String> jsonLines) {
List<Document> mongoDocs = generateMongoDocs(jsonLines);
int inserts = insertInto(collection, mongoDocs);
return inserts + "/" + jsonLines.size();
}4. Use Cases
4.使用案例
Now that we’re ready to process input, we can build some use cases. Let’s create the ImportUtils class to help us with that. This class will be responsible for converting input into lines of JSON. It will only contain static methods. Let’s start with the one for reading a simple String:
现在我们已经准备好处理输入，我们可以建立一些用例。让我们创建ImportUtils类来帮助我们完成这项工作。这个类将负责把输入转化为JSON行。它将只包含静态方法。让我们从读取一个简单的String开始。
public static List<String> lines(String json) {
String[] split = json.split("[\\r\\n]+");
return Arrays.asList(split);
}
Since we’re using line breaks as a delimiter, regex works great to break strings into multiple lines. This regex handles both Unix and Windows line endings. Next, a method to convert a File into a list of strings:
由于我们使用换行符作为分隔符，regex在将字符串分成多行方面非常有效。这个regex可以处理Unix和Windows的行结束符。接下来，一个将文件转换为一个字符串列表的方法。
public static List<String> lines(File file) {
return Files.readAllLines(file.toPath());
}Similarly, we finish up with a method to convert a classpath resource into a list:
同样地，我们最后用一个方法将classpath资源转换成一个列表。
public static List<String> linesFromResource(String resource) {
Resource input = new ClassPathResource(resource);
Path path = input.getFile().toPath();
return Files.readAllLines(path);
}4.1. Import File During Startup With a CLI
4.1.在启动期间用CLI导入文件
In our first use case, we’ll implement functionality for importing a file via application arguments. We’ll take advantage of the Spring Boot ApplicationRunner interface to do this at boot time. For instance, we can read command line parameters to define the file to import:
在我们的第一个用例中，我们将实现通过应用程序参数导入文件的功能。我们将利用 Spring Boot ApplicationRunner 接口，在启动时完成这一工作。例如，我们可以读取命令行参数来定义要导入的文件：。
@SpringBootApplication
public class SpringBootJsonConvertFileApplication implements ApplicationRunner {
private static final String RESOURCE_PREFIX = "classpath:";
@Autowired
private ImportJsonService importService;
public static void main(String ... args) {
SpringApplication.run(SpringBootPersistenceApplication.class, args);
}
@Override
public void run(ApplicationArguments args) {
if (args.containsOption("import")) {
String collection = args.getOptionValues("collection")
.get(0);
List<String> sources = args.getOptionValues("import");
for (String source : sources) {
List<String> jsonLines = new ArrayList<>();
if (source.startsWith(RESOURCE_PREFIX)) {
String resource = source.substring(RESOURCE_PREFIX.length());
jsonLines = ImportUtils.linesFromResource(resource);
} else {
jsonLines = ImportUtils.lines(new File(source));
}
String result = importService.importTo(collection, jsonLines);
log.info(source + " - result: " + result);
}
}
}
}
Using getOptionValues() we can process one or more files. These files can be either from our classpath or from our file system. We differentiate them using the RESOURCE_PREFIX. Every argument starting with “classpath:” will be read from our resources folder instead of from the file system. After that, they will all be imported into the desired collection.
使用getOptionValues()我们可以处理一个或多个文件。这些文件可以来自我们的classpath或文件系统。我们使用RESOURCE_PREFIX来区分它们。每个以”classpath:“开头的参数都将从我们的资源文件夹而不是文件系统中读取。之后，它们都将被导入到所需的collection中。
Let’s start using our application by creating a file under src/main/resources/data.json.log:
让我们开始使用我们的应用程序，在src/main/resources/data.json.log下创建一个文件。
{"name":"Book A", "genre": "Comedy"}
{"name":"Book B", "genre": "Thriller"}
{"name":"Book C", "genre": "Drama"}After building, we can use the following example to run it (line breaks added for readability). In our example, two files will be imported, one from the classpath, and one from the file system:
在构建之后，我们可以用下面的例子来运行它（为了便于阅读，添加了换行符）。在我们的例子中，将导入两个文件，一个来自classpath，另一个来自文件系统。
java -cp target/spring-boot-persistence-mongodb/WEB-INF/lib/*:target/spring-boot-persistence-mongodb/WEB-INF/classes \
-Djdk.tls.client.protocols=TLSv1.2 \
com.baeldung.SpringBootPersistenceApplication \
--import=classpath:data.json.log \
--import=/tmp/data.json \
--collection=books4.2. JSON File From HTTP POST Upload
4.2 HTTP POST上传的JSON文件
Additionally, if we create a REST Controller, we’ll have an endpoint to upload and import JSON files. For that, we’ll need a MultipartFile parameter:
此外，如果我们创建一个REST控制器，我们将有一个端点来上传和导入JSON文件。为此，我们需要一个MultipartFile参数。
@RestController
@RequestMapping("/import-json")
public class ImportJsonController {
@Autowired
private ImportJsonService service;
@PostMapping("/file/{collection}")
public String postJsonFile(@RequestPart("parts") MultipartFile jsonStringsFile, @PathVariable String collection) {
List<String> jsonLines = ImportUtils.lines(jsonStringsFile);
return service.importTo(collection, jsonLines);
}
}Now we can import files with a POST like this, where “/tmp/data.json” refers to an existing file:
现在我们可以用POST这样的方式导入文件，其中”/tmp/data.json“指的是一个现有文件。
curl -X POST http://localhost:8082/import-json/file/books -F "parts=@/tmp/books.json"4.3. Mapping JSON to a Specific Java Type
4.3.将JSON映射到一个特定的Java类型
We’ve been using only JSON, not bound to any type, which is one of the advantages of working with MongoDB. Now we want to validate our input. In this case, let’s add an ObjectMapper by making this change to our service:
我们一直只使用JSON，不与任何类型绑定，这是与MongoDB合作的优势之一。现在我们要验证我们的输入。在这种情况下，让我们通过对我们的服务进行这样的修改来添加一个ObjectMapper。
private <T> List<Document> generateMongoDocs(List<String> lines, Class<T> type) {
ObjectMapper mapper = new ObjectMapper();
List<Document> docs = new ArrayList<>();
for (String json : lines) {
if (type != null) {
mapper.readValue(json, type);
}
docs.add(Document.parse(json));
}
return docs;
}That way, if the type parameter is specified, our mapper will try to parse our JSON string as that type. And, with default configuration, will throw an exception if any unknown properties are present. Here’s our simple bean definition for working with a MongoDB repository:
这样，如果type参数被指定，我们的mapper将尝试将我们的JSON字符串解析为该类型。在默认配置下，如果存在任何未知的属性，将抛出一个异常。以下是我们的简单 Bean 定义，用于与 MongoDB 存储库配合工作。
@Document("books")
public class Book {
@Id
private String id;
private String name;
private String genre;
// getters and setters
}And now, to use the improved version of our Document generator, let’s change this method as well:
现在，为了使用改进版的文档生成器，我们也来改变这个方法。
public String importTo(Class<?> type, List<String> jsonLines) {
List<Document> mongoDocs = generateMongoDocs(jsonLines, type);
String collection = type.getAnnotation(org.springframework.data.mongodb.core.mapping.Document.class)
.value();
int inserts = insertInto(collection, mongoDocs);
return inserts + "/" + jsonLines.size();
}Now, instead of passing the name of a collection, we pass a Class. We assume it has the Document annotation as we used in our Book, so it can retrieve the collection name. However, since both the annotation and the Document classes have the same name, we have to specify the whole package.
现在，我们不再传递一个集合的名字，而是传递一个Class。我们假设它有Document注解，就像我们在Book中使用的那样，所以它可以检索到集合的名称。然而，由于注解和Document类都有相同的名字，我们必须指定整个包。
5. Conclusion
5.总结
In this article, we went through breaking JSON input from files, resources, or simple strings and importing them into MongoDB. We centralized this functionality in a service class and a utility class so we could reuse it anywhere. Our use cases included a CLI and a REST option, along with example commands on how to use it.
在这篇文章中，我们经历了从文件、资源或简单字符串中破解JSON输入，并将其导入MongoDB。我们将这一功能集中在一个服务类和一个实用类中，因此我们可以在任何地方重新使用它。我们的使用案例包括一个 CLI 和一个 REST 选项，以及关于如何使用它的示例命令。
And as always, the source code is available over on GitHub.
一如既往，源代码可在GitHub上获得。