1. Overview
1.概述
In this article, we’ll externalize the setup data of an application using CSV files, instead of hardcoding it.
在这篇文章中，我们将使用CSV文件将应用程序的设置数据外置，而不是硬编码。
This setup process is mainly concerned with setting up new data on a fresh system.
这个设置过程主要是在一个新的系统上设置新的数据。
2. A CSV Library
2.一个CSV库
Let’s start by introducing a simple library to work with CSV – the Jackson CSV extension:
让我们首先介绍一个简单的库来处理CSV – Jackson CSV扩展。
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
<version>2.5.3</version>
</dependency>There are of course a host of available libraries to work with CSVs in the Java ecosystem.
当然，在Java生态系统中，有许多可用的库来处理CSVs。
The reason we’re going with Jackson here is that – it’s likely that Jackson is already in use in the application, and the processing we need to read the data is fairly straightforward.
我们在这里使用Jackson的原因是–很可能Jackson已经在应用中使用了，而且我们需要的读取数据的处理是相当简单的。
3. The Setup Data
3.设置数据
Different projects will need to setup different data.
不同的项目将需要设置不同的数据。
In this tutorial, we’re going to be setting up User data – basically preparing the system with a few default users.
在本教程中，我们将设置用户数据–基本上是用一些默认用户来准备系统。
Here’s the simple CSV file containing the users:
这里是包含用户的简单CSV文件。
id,username,password,accessToken
1,john,123,token
2,tom,456,testNote how the first row of the file is the header row – listing out the names of the fields in each row of data.
注意文件的第一行是标题行–列出了每行数据中的字段名称。
3. CSV Data Loader
3.CSV数据加载器
Let’s start by creating a simple data loader to read up data from the CSV files into working memory.
让我们先创建一个简单的数据加载器，将CSV文件中的数据读入工作内存。
3.1. Load a List of Objects
3.1.加载一个对象的列表
We’ll implement the loadObjectList() functionality to load a fully parametrized List of specific Object from the file:
我们将实现loadObjectList()功能，从文件中加载一个完全参数化的特定Object的列表。
public <T> List<T> loadObjectList(Class<T> type, String fileName) {
try {
CsvSchema bootstrapSchema = CsvSchema.emptySchema().withHeader();
CsvMapper mapper = new CsvMapper();
File file = new ClassPathResource(fileName).getFile();
MappingIterator<T> readValues =
mapper.reader(type).with(bootstrapSchema).readValues(file);
return readValues.readAll();
} catch (Exception e) {
logger.error("Error occurred while loading object list from file " + fileName, e);
return Collections.emptyList();
}
}Notes:
注意事项。
- We created the CSVSchema based on first “header” row.
- The implementation is generic enough to handle any type of object.
- If any error occurs, an empty list will be returned.
3.2. Handle Many to Many Relationship
3.2.处理多对多关系
Nested objects are not well supported in Jackson CSV – we’ll need to use an indirect way to load Many to Many relationships.
嵌套对象在Jackson CSV中没有得到很好的支持–我们需要使用一种间接的方式来加载多对多关系。
We’ll represent these similar to simple Join Tables – so naturally we’ll load from disk as a list of arrays:
我们将表示这些类似于简单的Join Tables–所以我们自然会从磁盘上加载一个数组的列表。
public List<String[]> loadManyToManyRelationship(String fileName) {
try {
CsvMapper mapper = new CsvMapper();
CsvSchema bootstrapSchema = CsvSchema.emptySchema().withSkipFirstDataRow(true);
mapper.enable(CsvParser.Feature.WRAP_AS_ARRAY);
File file = new ClassPathResource(fileName).getFile();
MappingIterator<String[]> readValues =
mapper.reader(String[].class).with(bootstrapSchema).readValues(file);
return readValues.readAll();
} catch (Exception e) {
logger.error(
"Error occurred while loading many to many relationship from file = " + fileName, e);
return Collections.emptyList();
}
}Here’s how one of these relationships – Roles <-> Privileges – is represented in a simple CSV file:
下面是这些关系之一–Roles <-> Privileges–在一个简单的CSV文件中的表现。
role,privilege
ROLE_ADMIN,ADMIN_READ_PRIVILEGE
ROLE_ADMIN,ADMIN_WRITE_PRIVILEGE
ROLE_SUPER_USER,POST_UNLIMITED_PRIVILEGE
ROLE_USER,POST_LIMITED_PRIVILEGENote how we’re ignoring the header in this implementation, as we don’t really need that information.
请注意我们在这个实现中忽略了头，因为我们并不真正需要这些信息。
4. Setup Data
4.设置数据
Now, we’ll use a simple Setup bean to do all the work of setting up privileges, roles and users from CSV files:
现在，我们将使用一个简单的Setup bean来完成从CSV文件中设置权限、角色和用户的所有工作。
@Component
public class Setup {
...
@PostConstruct
private void setupData() {
setupRolesAndPrivileges();
setupUsers();
}
...
}4.1. Setup Roles and Privileges
4.1.设置角色和权限
First, let’s load roles and privileges from disk into working memory, and then persist them as part of the setup process:
首先，让我们把角色和权限从磁盘加载到工作内存中，然后把它们作为设置过程的一部分持久化。
public List<Privilege> getPrivileges() {
return csvDataLoader.loadObjectList(Privilege.class, PRIVILEGES_FILE);
}
public List<Role> getRoles() {
List<Privilege> allPrivileges = getPrivileges();
List<Role> roles = csvDataLoader.loadObjectList(Role.class, ROLES_FILE);
List<String[]> rolesPrivileges = csvDataLoader.
loadManyToManyRelationship(SetupData.ROLES_PRIVILEGES_FILE);
for (String[] rolePrivilege : rolesPrivileges) {
Role role = findRoleByName(roles, rolePrivilege[0]);
Set<Privilege> privileges = role.getPrivileges();
if (privileges == null) {
privileges = new HashSet<Privilege>();
}
privileges.add(findPrivilegeByName(allPrivileges, rolePrivilege[1]));
role.setPrivileges(privileges);
}
return roles;
}
private Role findRoleByName(List<Role> roles, String roleName) {
return roles.stream().
filter(item -> item.getName().equals(roleName)).findFirst().get();
}
private Privilege findPrivilegeByName(List<Privilege> allPrivileges, String privilegeName) {
return allPrivileges.stream().
filter(item -> item.getName().equals(privilegeName)).findFirst().get();
}Then we’ll do the persist work here:
然后我们就在这里做坚持不懈的工作。
private void setupRolesAndPrivileges() {
List<Privilege> privileges = setupData.getPrivileges();
for (Privilege privilege : privileges) {
setupService.setupPrivilege(privilege);
}
List<Role> roles = setupData.getRoles();
for (Role role : roles) {
setupService.setupRole(role);
}
}And here is our SetupService:
这里是我们的SetupService。
public void setupPrivilege(Privilege privilege) {
if (privilegeRepository.findByName(privilege.getName()) == null) {
privilegeRepository.save(privilege);
}
}
public void setupRole(Role role) {
if (roleRepository.findByName(role.getName()) == null) {
Set<Privilege> privileges = role.getPrivileges();
Set<Privilege> persistedPrivileges = new HashSet<Privilege>();
for (Privilege privilege : privileges) {
persistedPrivileges.add(privilegeRepository.findByName(privilege.getName()));
}
role.setPrivileges(persistedPrivileges);
roleRepository.save(role); }
}Note how, after we load both Roles and Privileges into working memory, we load their relationships one by one.
请注意，在我们将角色和权限都加载到工作存储器中后，我们是如何逐一加载它们的关系的。
4.2. Setup Initial Users
4.2.设置初始用户
Next – let’s load the users into memory and persist them:
接下来–让我们把用户加载到内存中，并持久化它们。
public List<User> getUsers() {
List<Role> allRoles = getRoles();
List<User> users = csvDataLoader.loadObjectList(User.class, SetupData.USERS_FILE);
List<String[]> usersRoles = csvDataLoader.
loadManyToManyRelationship(SetupData.USERS_ROLES_FILE);
for (String[] userRole : usersRoles) {
User user = findByUserByUsername(users, userRole[0]);
Set<Role> roles = user.getRoles();
if (roles == null) {
roles = new HashSet<Role>();
}
roles.add(findRoleByName(allRoles, userRole[1]));
user.setRoles(roles);
}
return users;
}
private User findByUserByUsername(List<User> users, String username) {
return users.stream().
filter(item -> item.getUsername().equals(username)).findFirst().get();
}Next, let’s focus on persisting the users:
接下来，让我们专注于持久化用户。
private void setupUsers() {
List<User> users = setupData.getUsers();
for (User user : users) {
setupService.setupUser(user);
}
}And here is our SetupService:
这里是我们的SetupService。
@Transactional
public void setupUser(User user) {
try {
setupUserInternal(user);
} catch (Exception e) {
logger.error("Error occurred while saving user " + user.toString(), e);
}
}
private void setupUserInternal(User user) {
if (userRepository.findByUsername(user.getUsername()) == null) {
user.setPassword(passwordEncoder.encode(user.getPassword()));
user.setPreference(createSimplePreference(user));
Set<Role> roles = user.getRoles();
Set<Role> persistedRoles = new HashSet<Role>();
for (Role role : roles) {
persistedRoles.add(roleRepository.findByName(role.getName()));
}
user.setRoles(persistedRoles);
userRepository.save(user);
}
}And here is createSimplePreference() method:
这里是createSimplePreference()方法。
private Preference createSimplePreference(User user) {
Preference pref = new Preference();
pref.setId(user.getId());
pref.setTimezone(TimeZone.getDefault().getID());
pref.setEmail(user.getUsername() + "@test.com");
return preferenceRepository.save(pref);
}Note how, before we save a user, we create a simple Preference entity for it and persist that first.
请注意，在我们保存一个用户之前，我们为它创建了一个简单的Preference实体，并首先将其持久化。
5. Test CSV Data Loader
5.测试CSV数据加载器
Next, let’s perform a simple unit test on our CsvDataLoader:
接下来，让我们对我们的CsvDataLoader执行一个简单的单元测试。
We will test loading list of Users, Roles and Privileges:
我们将测试加载用户、角色和权限的列表。
@Test
public void whenLoadingUsersFromCsvFile_thenLoaded() {
List<User> users = csvDataLoader.
loadObjectList(User.class, CsvDataLoader.USERS_FILE);
assertFalse(users.isEmpty());
}
@Test
public void whenLoadingRolesFromCsvFile_thenLoaded() {
List<Role> roles = csvDataLoader.
loadObjectList(Role.class, CsvDataLoader.ROLES_FILE);
assertFalse(roles.isEmpty());
}
@Test
public void whenLoadingPrivilegesFromCsvFile_thenLoaded() {
List<Privilege> privileges = csvDataLoader.
loadObjectList(Privilege.class, CsvDataLoader.PRIVILEGES_FILE);
assertFalse(privileges.isEmpty());
}Next, let’s test loading some Many to Many relationships via the data loader:
接下来，让我们测试一下通过数据加载器加载一些Many to Many关系。
@Test
public void whenLoadingUsersRolesRelationFromCsvFile_thenLoaded() {
List<String[]> usersRoles = csvDataLoader.
loadManyToManyRelationship(CsvDataLoader.USERS_ROLES_FILE);
assertFalse(usersRoles.isEmpty());
}
@Test
public void whenLoadingRolesPrivilegesRelationFromCsvFile_thenLoaded() {
List<String[]> rolesPrivileges = csvDataLoader.
loadManyToManyRelationship(CsvDataLoader.ROLES_PRIVILEGES_FILE);
assertFalse(rolesPrivileges.isEmpty());
}6. Test Setup Data
6.测试设置数据
Finally, let’s perform a simple unit test on our bean SetupData:
最后，让我们对我们的Bean SetupData执行一个简单的单元测试。
@Test
public void whenGettingUsersFromCsvFile_thenCorrect() {
List<User> users = setupData.getUsers();
assertFalse(users.isEmpty());
for (User user : users) {
assertFalse(user.getRoles().isEmpty());
}
}
@Test
public void whenGettingRolesFromCsvFile_thenCorrect() {
List<Role> roles = setupData.getRoles();
assertFalse(roles.isEmpty());
for (Role role : roles) {
assertFalse(role.getPrivileges().isEmpty());
}
}
@Test
public void whenGettingPrivilegesFromCsvFile_thenCorrect() {
List<Privilege> privileges = setupData.getPrivileges();
assertFalse(privileges.isEmpty());
}7. Conclusion
7.结论
In this quick article we explored an alternative setup method for the initial data that usually needs to be loaded into a system on startup. This is of course just a simple Proof of Concept and a good base to build upon – not a production ready solution.
在这篇快速的文章中，我们探讨了一种替代的设置方法，用于通常需要在启动时加载到系统中的初始数据。当然，这只是一个简单的概念证明，是一个良好的基础–不是一个可用于生产的解决方案。
We’re also going to use this solution in the Reddit web application tracked by this ongoing case study.
我们还将在这个正在进行的案例研究所追踪的Reddit网络应用中使用这个解决方案。