Key Value Store with Chronicle Map – 带纪事地图的键值存储

最后修改: 2019年 7月 3日

中文/混合/英文(键盘快捷键：t)

 1. Overview

1.概述

In this tutorial, we’re going to see how we can use the Chronicle Map for storing key-value pairs. We’ll also be creating short examples to demonstrate its behavior and usage.

在本教程中，我们将了解如何使用Chronicle Map来存储键值对。我们还将创建简短的例子来演示其行为和用法。

2. What Is a Chronicle Map?

2.什么是纪事地图？

Following the documentation, “Chronicle Map is a super-fast, in-memory, non-blocking, key-value store, designed for low-latency, and/or multi-process applications”.

根据文档，“Chronicle Map是一个超快的、内存中的、非阻塞的、键值存储，为低延迟和/或多进程应用而设计。”。

In a nutshell, it’s an off-heap key-value store. The map doesn’t require a large amount of RAM for it to function properly. It can grow based on the available disk capacity. Furthermore, it supports the replication of the data in a multi-master server setup.

简而言之，它是一个离堆键值存储。该地图不需要大量的RAM就能正常运行。它可以根据可用的磁盘容量来增长。此外，它支持在多主服务器设置中的数据复制。

Let’s now see how we can set up and work with it.

现在让我们看看我们如何设置和使用它。

3. Maven Dependency

3.Maven的依赖性

To get started, we’ll need to add the chronicle-map dependency to our project:

为了开始工作，我们需要将chronicle-map依赖项添加到我们的项目中。

<dependency>
    <groupId>net.openhft</groupId>
    <artifactId>chronicle-map</artifactId>
    <version>3.17.2</version>
</dependency>

4. Types of Chronicle Map

4.纪事地图的类型

We can create a map in two ways: either as an in-memory map or as a persisted map.

我们可以用两种方式创建一个地图：要么是内存中的地图，要么是持久化的地图。

Let’s see both of these in detail.

让我们详细看看这两点。

4.1. In-Memory Map

4.1.内存中的地图

An in-memory Chronicle Map is a map store that is created within the physical memory of the server. This means it’s accessible only within the JVM process in which the map store is created.

内存中的Chronicle Map是一个在服务器的物理内存中创建的地图存储。这意味着它只能在创建地图存储的JVM进程中访问。

Let’s see a quick example:

让我们看一个快速的例子。

ChronicleMap<LongValue, CharSequence> inMemoryCountryMap = ChronicleMap
  .of(LongValue.class, CharSequence.class)
  .name("country-map")
  .entries(50)
  .averageValue("America")
  .create();

For the sake of simplicity, we’re creating a map that stores 50 country ids and their names. As we can see in the code snippet, the creation is pretty straightforward except for the averageValue() configuration. This tells the map to configure the average number of bytes taken by map entry values.

为了简单起见，我们要创建一个存储50个国家ID及其名称的地图。正如我们在代码片段中所看到的，除了averageValue()配置外，创建工作是非常直接的。这告诉地图要配置地图条目值所占的平均字节数。

In other words, when creating the map, the Chronicle Map determines the average number of bytes taken by the serialized form of values. It does this by serializing the given average value using the configured value marshallers. It will then allocate the determined number of bytes for the value of each map entry.

换句话说，在创建地图时，Chronicle Map会确定数值的序列化形式所占用的平均字节数。它通过使用配置的值集束器对给定的平均值进行序列化来实现这一点。然后，它将为每个地图条目的值分配确定的字节数。。

One thing we have to note when it comes to the in-memory map is that the data is accessible only when the JVM process is alive. The library will clear the data when the process terminates.

当涉及到内存地图时，我们必须注意的一点是，只有当JVM进程活着时，才能访问这些数据。当进程终止时，库将清除这些数据。

4.2. Persisted Map

4.2.持续的地图

Unlike an in-memory map, the implementation will save a persisted map to disk. Let’s now see how we can create a persisted map:

与内存中的地图不同，该实现会将持久化的地图保存到磁盘上。现在让我们看看如何创建一个持久化的地图。

ChronicleMap<LongValue, CharSequence> persistedCountryMap = ChronicleMap
  .of(LongValue.class, CharSequence.class)
  .name("country-map")
  .entries(50)
  .averageValue("America")
  .createPersistedTo(new File(System.getProperty("user.home") + "/country-details.dat"));

This will create a file called country-details.dat in the folder specified. If this file is already available in the specified path, then the builder implementation will open a link to the existing data store from this JVM process.

这将在指定的文件夹中创建一个名为country-details.dat的文件。如果这个文件在指定的路径中已经存在，那么构建器实现将从这个JVM进程中打开一个指向现有数据存储的链接。

We can make use of the persisted map in cases where we want it to:

我们可以在我们想要的情况下利用持久化的地图。

  • survive beyond the creator process; for example, to support hot application redeployment
  • make it global in a server; for example, to support multiple concurrent process access
  • act as a data store that we’ll save to the disk

5. Size Configuration

5.尺寸配置

It’s mandatory to configure the average value and average key while creating a Chronicle Map, except in the case where our key/value type is either a boxed primitive or a value interface. In our example, we’re not configuring the average key since the key type LongValue is a value interface.

在创建Chronicle Map时，必须配置平均值和平均键，除非我们的键/值类型是盒式原语或值接口。在我们的例子中，我们没有配置平均键，因为键类型LongValue是一个值接口。

Now, let’s see what the options are for configuring the average number of key/value bytes:

现在，让我们看看配置键/值的平均字节数有哪些选项。

  • averageValue() – The value from which the average number of bytes to be allocated for the value of a map entry is determined
  • averageValueSize() – The average number of bytes to be allocated for the value of a map entry
  • constantValueSizeBySample() – The number of bytes to be allocated for the value of a map entry when the size of the value is always the same
  • averageKey() – The key from which the average number of bytes to be allocated for the key of a map entry is determined
  • averageKeySize() – The average number of bytes to be allocated for the key of a map entry
  • constantKeySizeBySample() – The number of bytes to be allocated for the key of a map entry when the size of the key is always the same

6. Key and Value Types

6.键和值类型

There are certain standards that we need to follow when creating a Chronicle Map, especially when defining the key and value. The map works best when we create the key and value using the recommended types.

在创建Chronicle Map时，我们需要遵循一些标准，特别是在定义key和value时。当我们使用推荐的类型创建键和值时，地图的效果最好。

Here are some of the recommended types:

以下是一些推荐的类型。

  • Value interfaces
  • Any class implementing Byteable interface from Chronicle Bytes
  • Any class implementing BytesMarshallable interface from Chronicle Bytes; the implementation class should have a public no-arg constructor
  • byte[] and ByteBuffer
  • CharSequence, String, and StringBuilder
  • Integer, Long, and Double
  • Any class implementing java.io.Externalizable; the implementation class should have a public no-arg constructor
  • Any type implementing java.io.Serializable, including boxed primitive types (except those listed above) and array types
  • Any other type, if custom serializers are provided

7. Querying a Chronicle Map

7.查询编年史地图

Chronicle Map supports single-key queries as well as multi-key queries.

Chronicle Map支持单键查询，也支持多键查询。

7.1. Single-Key Queries

7.1.单键查询

Single-key queries are the operations that deal with a single key. ChronicleMap supports all the operations from the Java Map interface and ConcurrentMap interface:

单键查询是处理单键的操作。ChronicleMap支持Java Map接口和ConcurrentMap接口的所有操作。

LongValue qatarKey = Values.newHeapInstance(LongValue.class);
qatarKey.setValue(1);
inMemoryCountryMap.put(qatarKey, "Qatar");

//...

CharSequence country = inMemoryCountryMap.get(key);

In addition to the normal get and put operations, ChronicleMap adds a special operation, getUsing(), that reduces the memory footprint while retrieving and processing an entry. Let’s see this in action:

除了正常的get和put操作外，ChronicleMap增加了一个特殊的操作，getUsing()，在检索和处理一个条目时减少内存占用。让我们看看这个操作的效果。

LongValue key = Values.newHeapInstance(LongValue.class);
StringBuilder country = new StringBuilder();
key.setValue(1);
persistedCountryMap.getUsing(key, country);
assertThat(country.toString(), is(equalTo("Romania")));

key.setValue(2);
persistedCountryMap.getUsing(key, country);
assertThat(country.toString(), is(equalTo("India")));

Here we’ve used the same StringBuilder object for retrieving values of different keys by passing it to the getUsing() method. It basically reuses the same object for retrieving different entries. In our case, the getUsing() method is equivalent to:

在这里，我们使用同一个StringBuilder对象，通过将其传递给getUsing()方法来检索不同键的值。它基本上是重复使用同一个对象来检索不同的条目。在我们的例子中，getUsing()方法相当于。

country.setLength(0);
country.append(persistedCountryMap.get(key));

7.2. Multi-Key Queries

7.2.多键查询

There may be use cases where we need to deal with multiple keys at the same time. For this, we can use the queryContext() functionality. The queryContext() method will create a context for working with a map entry.

可能有一些用例，我们需要同时处理多个键。为此，我们可以使用queryContext()功能。queryContext()方法将创建一个用于处理地图条目的上下文。

Let’s first create a multimap and add some values to it:

让我们首先创建一个多图，并向其添加一些值。

Set<Integer> averageValue = IntStream.of(1, 2).boxed().collect(Collectors.toSet());
ChronicleMap<Integer, Set<Integer>> multiMap = ChronicleMap
  .of(Integer.class, (Class<Set<Integer>>) (Class) Set.class)
  .name("multi-map")
  .entries(50)
  .averageValue(averageValue)
  .create();

Set<Integer> set1 = new HashSet<>();
set1.add(1);
set1.add(2);
multiMap.put(1, set1);

Set<Integer> set2 = new HashSet<>();
set2.add(3);
multiMap.put(2, set2);

To work with multiple entries, we have to lock those entries to prevent inconsistency that may occur due to a concurrent update:

为了处理多个条目，我们必须锁定这些条目，以防止因并发更新而可能出现的不一致：。

try (ExternalMapQueryContext<Integer, Set<Integer>, ?> fistContext = multiMap.queryContext(1)) {
    try (ExternalMapQueryContext<Integer, Set<Integer>, ?> secondContext = multiMap.queryContext(2)) {
        fistContext.updateLock().lock();
        secondContext.updateLock().lock();

        MapEntry<Integer, Set<Integer>> firstEntry = fistContext.entry();
        Set<Integer> firstSet = firstEntry.value().get();
        firstSet.remove(2);

        MapEntry<Integer, Set<Integer>> secondEntry = secondContext.entry();
        Set<Integer> secondSet = secondEntry.value().get();
        secondSet.add(4);

        firstEntry.doReplaceValue(fistContext.wrapValueAsData(firstSet));
        secondEntry.doReplaceValue(secondContext.wrapValueAsData(secondSet));
    }
} finally {
    assertThat(multiMap.get(1).size(), is(equalTo(1)));
    assertThat(multiMap.get(2).size(), is(equalTo(2)));
}

8. Closing the Chronicle Map

8.关闭纪事地图

Now that we’ve finished working with our maps, let’s call the close() method on our map objects to release the off-heap memory and the resources associated with it:

现在我们已经完成了对地图的操作，让我们在地图对象上调用close()方法来释放堆外内存和与之相关的资源。

persistedCountryMap.close();
inMemoryCountryMap.close();
multiMap.close();

One thing to keep in mind here is that all the map operations must be completed before closing the map. Otherwise, the JVM might crash unexpectedly.

这里需要记住的一点是，在关闭地图之前必须完成所有的地图操作。否则，JVM可能会意外崩溃。

9. Conclusion

9.结论

In this tutorial, we’ve learned how to use a Chronicle Map to store and retrieve key-value pairs. Even though the community version is available with most of the core functionalities, the commercial version has some advanced features like data replication across multiple servers and remote calls.

在本教程中，我们已经学会了如何使用Chronicle Map来存储和检索键值对。即使社区版有大部分的核心功能，商业版也有一些高级功能，如跨多个服务器的数据复制和远程调用。

All the examples we’ve discussed here can be found over the Github project.

我们在这里讨论的所有例子都可以在Github项目中找到。