A Guide to SirixDB – SirixDB指南

1. Overview

1.概述

In this tutorial, we’ll give an overview of what SirixDB is and its most important design goals.

在本教程中，我们将概述什么是SirixDB以及其最重要的设计目标。

Next, we’ll give a walk through a low level cursor-based transactional API.

接下来，我们将介绍一个低级别的基于游标的事务性API。

2. SirixDB Features

2.SirixDB的特点

SirixDB is a log-structured, temporal NoSQL document store, which stores evolutionary data. It never overwrites any data on-disk. Thus, we’re able to restore and query the full revision history of a resource in the database efficiently. SirixDB ensures, that a minimum of storage-overhead is created for each new revision.

SirixDB是一个具有日志结构的时态NoSQL文件存储，它存储的是进化数据。它从不覆盖磁盘上的任何数据。因此，我们能够有效地恢复和查询数据库中资源的全部修订历史。SirixDB确保为每一个新的修订创造最小的存储开销。

Currently, SirixDB offers two built-in native data models, namely a binary XML store as well as a JSON store.

目前，SirixDB提供两种内置的本地数据模型，即二进制XML存储以及JSON存储。

2.1. Design Goals

2.1.设计目标

Some of the most important core principles and design goals are:

一些最重要的核心原则和设计目标是。

• Concurrency – SirixDB contains very few locks and aims to be as suitable for multithreaded systems as possible
• Asynchronous REST API – operations can happen independently; each transaction is bound to a specific revision and only one read-write transaction on a resource is permitted concurrently to N read-only transactions
• Versioning/Revision history – SirixDB stores a revision history of every resource in the database while keeping storage-overhead to a minimum. Read and write performance is tunable. It depends on the versioning type, which we can specify for creating a resource
• Data integrity – SirixDB, like ZFS, stores full checksums of the pages in the parent pages. That means that almost all data corruption can be detected upon reading in the future, as the SirixDB developers aim to partition and replicate databases in the future
• Copy-on-write semantics – similarly to the file systems Btrfs and ZFS, SirixDB uses CoW semantics, meaning that SirixDB never overwrites data. Instead, database page fragments are copied and written to a new location
• Per revision and per record versioning – SirixDB does not only version on a per-page, but also on a per-record basis. Thus, whenever we change a potentially small fraction
  of records in a data page, it does not have to copy the whole page and write it to a new location on a disk or flash drive. Instead, we can specify one of several versioning strategies known from backup systems or a sliding snapshot algorithm during the creation of a database resource. The versioning type we specify is used by SirixDB to version data pages
• Guaranteed atomicity (without a WAL) – the system will never enter an inconsistent state (unless there is hardware failure), meaning that unexpected power-off won’t ever damage the system. This is accomplished without the overhead of a write-ahead-log (WAL)
• Log-structured and SSD friendly – SirixDB batches writes and syncs everything sequentially to a flash drive during commits. It never overwrites committed data

We first want to introduce the low-level API exemplified with JSON data before switching our focus to higher levels in future articles. For instance an XQuery-API for querying both XML and JSON databases or an asynchronous, temporal RESTful API. We can basically use the same low-level API with subtle differences to store, traverse and compare XML resources as well.

我们首先要介绍以JSON数据为例的低级API，然后在未来的文章中把重点转向更高级别的。例如，用于查询XML和JSON数据库的XQuery-API或异步的、时间性的RESTful API。我们基本上可以使用相同的低级别的API，但有细微的差别，来存储、遍历和比较XML资源。

In order to use SirixDB, we at least have to use Java 11.

为了使用SirixDB，我们至少要使用Java 11。。

3. Maven Dependency to Embed SirixDB

3.嵌入SirixDB的Maven依赖项

To follow the examples, we first have to include the sirix-core dependency, for instance, via Maven:

为了遵循这些例子，我们首先要包括sirix-core依赖项，例如，通过Maven。

<dependency>
    <groupId>io.sirix</groupId>
    <artifactId>sirix-core</artifactId>
    <version>0.9.3</version>
</dependency>

Or via Gradle:

或者通过Gradle。

dependencies {
    compile 'io.sirix:sirix-core:0.9.3'
}

4. Tree-Encoding in SirixDB

4.SirixDB中的树形编码

A node in SirixDB references other nodes by a firstChild/leftSibling/rightSibling/parentNodeKey/nodeKey encoding:

SirixDB中的一个节点通过firstChild/leftSibling/rightSibling/parentNodeKey/nodeKey编码引用其他节点。

The numbers in the figure are auto-generated unique, stable node IDs generated with a simple sequential number generator.

图中的数字是自动生成的唯一的、稳定的节点ID，由一个简单的顺序数字发生器生成。

Every node may have a first child, a left sibling, a right sibling, and a parent node. Furthermore, SirixDB is able to store the number of children, the number of descendants and hashes of each node.

每个节点都可能有一个第一个孩子，一个左手兄弟姐妹，一个右手兄弟姐妹，以及一个父节点。此外，SirixDB能够存储每个节点的子代数、子孙数和哈希值。

In the following sections, we’ll introduce the core low-level JSON API of SirixDB.

在下面的章节中，我们将介绍SirixDB的核心低级别的JSON API。

5. Create a Database With a Single Resource

5.用单一资源创建数据库

First, we want to show how to create a database with a single resource. The resource is going to be imported from a JSON file and stored persistently in the internal, binary format of SirixDB:

首先，我们要展示如何用一个单一的资源创建一个数据库。该资源将从一个JSON文件中导入，并以SirixDB的内部二进制格式持久性地存储。

var pathToJsonFile = Paths.get("jsonFile");
var databaseFile = Paths.get("database");

Databases.createJsonDatabase(new DatabaseConfiguration(databaseFile));

try (var database = Databases.openJsonDatabase(databaseFile)) {
    database.createResource(ResourceConfiguration.newBuilder("resource").build());

    try (var manager = database.openResourceManager("resource");
         var wtx = manager.beginNodeTrx()) {
        wtx.insertSubtreeAsFirstChild(JsonShredder.createFileReader(pathToJsonFile));
        wtx.commit();
    }
}

We first create a database. Then we open the database and create the first resource. Various options for creating a resource exist (see the official documentation).

我们首先创建一个数据库。然后我们打开数据库并创建第一个资源。存在各种创建资源的选项（参见官方文档）。

We then open a single read-write transaction on the resource to import the JSON file. The transaction provides a cursor for navigation through moveToX methods. Furthermore, the transaction provides methods to insert, delete or modify nodes. Note that the XML API even provides methods for moving nodes in a resource and copying nodes from other XML resources.

然后我们在资源上打开一个读写事务来导入JSON文件。该事务提供一个光标，用于通过moveToX方法进行导航。此外，该事务提供了插入、删除或修改节点的方法。请注意，XML API甚至提供了移动资源中的节点和从其他XML资源中复制节点的方法。

To properly close the opened read-write transaction, the resource manager and the database we use Java’s try-with-resources statement.

为了正确关闭已打开的读写事务、资源管理器和数据库，我们使用Java的try-with-resources语句。

We exemplified the creation of a database and resource on JSON data, but creating an XML database and resource is almost identical.

我们举例说明了在JSON数据上创建数据库和资源，但创建XML数据库和资源几乎是相同的。

In the next section, we’ll open a resource in a database and show navigational axes and methods.

在下一节，我们将在数据库中打开一个资源，并展示导航轴和方法。

6. Open a Resource in a Database and Navigate

6.在数据库中打开一个资源并进行导航

6.1. Preorder Navigation in a JSON Resource

6.1.JSON资源中的预排序导航

To navigate through the tree structure, we’re able to reuse the read-write transaction after committing. In the following code we’ll, however, open the resource again and begin a read-only transaction on the most recent revision:

为了浏览树状结构，我们能够在提交后重新使用读写事务。但在下面的代码中，我们将再次打开资源，并在最近的修订版上开始一个只读事务。

try (var database = Databases.openJsonDatabase(databaseFile);
     var manager = database.openResourceManager("resource");
     var rtx = manager.beginNodeReadOnlyTrx()) {
    
    new DescendantAxis(rtx, IncludeSelf.YES).forEach((unused) -> {
        switch (rtx.getKind()) {
            case OBJECT:
            case ARRAY:
                LOG.info(rtx.getDescendantCount());
                LOG.info(rtx.getChildCount());
                LOG.info(rtx.getHash());
                break;
            case OBJECT_KEY:
                LOG.info(rtx.getName());
                break;
            case STRING_VALUE:
            case BOOLEAN_VALUE:
            case NUMBER_VALUE:
            case NULL_VALUE:
                LOG.info(rtx.getValue());
                break;
            default:
        }
    });
}

We use the descendant axis to iterate over all nodes in preorder (depth-first). Hashes of nodes are built bottom-up for all nodes per default depending on the resource configuration.

我们使用子孙轴以预排序（深度优先）的方式遍历所有节点。根据资源配置，默认情况下，所有节点的哈希值是自下而上建立的。

Array nodes and Object nodes have no name and no value. We can use the same axis to iterate through XML resources, only the node types differ.

阵列节点和对象节点没有名字，也没有值。我们可以使用同样的轴来遍历XML资源，只是节点类型不同。

SirixDB offers a bunch of axes as for instance all XPath-axes to navigate through XML and JSON resources. Furthermore, it provides a LevelOrderAxis, a PostOrderAxis, a NestedAxis to chain axis and several ConcurrentAxis variants to fetch nodes concurrently and in parallel.

SirixDB提供了一堆轴，例如所有的XPath-axes来浏览XML和JSON资源。此外，它还提供了一个LevelOrderAxis、一个PostOrderAxis、一个NestedAxis链轴和几个ConcurrentAxis变体来并发和并行地获取节点。

In the next section, we’ll show how to use the VisitorDescendantAxis, which iterates in preorder, guided by return types of a node visitor.

在下一节中，我们将展示如何使用VisitorDescendantAxis，它在节点访问者的返回类型的引导下，以预置顺序进行迭代。

6.2. Visitor Descendant Axis

6.2.访客后裔轴

As it’s very common to define behavior based on the different node-types SirixDB uses the visitor pattern.

由于根据不同的节点类型来定义行为是非常常见的，SirixDB使用访问者模式。

We can specify a visitor as a builder argument for a special axis called VisitorDescendantAxis. For each type of node, there’s an equivalent visit-method. For instance, for object key nodes it is the method VisitResult visit(ImmutableObjectKeyNode node).

我们可以指定一个访问者作为一个特殊轴的构建器参数，称为VisitorDescendantAxis。对于每种类型的节点，都有一个等价的访问方法。例如，对于对象键节点，它的方法是VisitResult visit(ImmutableObjectKeyNode node) 。

Each method returns a value of type VisitResult. The only implementation of the VisitResult interface is the following enum:

每个方法都返回一个类型为VisitResult的值。VisitResult接口的唯一实现是下面这个枚举。

public enum VisitResultType implements VisitResult {
    SKIPSIBLINGS,
    SKIPSUBTREE,
    CONTINUE,
    TERMINATE
}

The VisitorDescendantAxis iterates through the tree structure in preorder. It uses the VisitResultTypes to guide the traversal:

VisitorDescendantAxis以预排序方式遍历树状结构。它使用VisitResultTypes来指导遍历。

• SKIPSIBLINGS means that the traversal should continue without visiting the right siblings of the current node the cursor points to
• SKIPSUBTREE means to continue without visiting the descendants of this node
• We use CONTINUE if the traversal should continue in preorder
• We can also use TERMINATE to terminate the traversal immediately

The default implementation of each method in the Visitor interface returns VisitResultType.CONTINUE for each node type. Thus, we only have to implement the methods for the nodes, which we’re interested in. If we’ve implemented a class which implements the Visitor interface called MyVisitor we can use the VisitorDescendantAxis in the following way:

Visitor接口中每个方法的默认实现都会为每个节点类型返回VisitResultType.CONTINUE。因此，我们只需要为我们感兴趣的节点实现这些方法。如果我们实现了一个实现了Visitor接口的类，叫做MyVisitor，我们就可以按以下方式使用VisitorDescendantAxis。

var axis = VisitorDescendantAxis.newBuilder(rtx)
  .includeSelf()
  .visitor(new MyVisitor())
  .build();

while (axis.hasNext()) axis.next();

The methods in MyVisitor are called for each node in the traversal. The parameter rtx is a read-only transaction. The traversal begins with the node the cursor currently points to.

MyVisitor中的方法为遍历中的每个节点被调用。参数rtx是一个只读的事务。遍历从游标当前指向的节点开始。

6.3. Time Travel Axis

6.3.时间旅行轴

var axis = new PastAxis(resourceManager, rtx);
if (axis.hasNext()) {
    var trx = axis.next();
    // Do something with the transactional cursor.
}

new FilterAxis<JsonNodeReadOnlyTrx>(new ChildAxis(rtx), new JsonNameFilter(rtx, "a"))

var axis = new VisitorDescendantAxis.Builder(rtx).includeSelf().visitor(myVisitor).build();
var filter = new JsonNameFilter(rtx, "foobar");
for (var filterAxis = new FilterAxis<JsonNodeReadOnlyTrx>(axis, filter); filterAxis.hasNext();) {
    filterAxis.next();
}

var axis = new PostOrderAxis(rtx);
var axisStream = StreamSupport.stream(axis.spliterator(), false);

axisStream.filter((unusedNodeKey) -> new JsonNameFilter(rtx, "a"))
  .forEach((unused) -> /* Do something with the transactional cursor */);

if (wtx.isObjectKey()) wtx.setObjectKeyName("foo");
if (wtx.isStringValue()) wtx.setStringValue("foo");

wtx.insertObjectRecordAsFirstChild("foo", new StringValue("bar"))
   .moveToParent().trx()
   .insertObjectRecordAsRightSibling("baz", new NullValue());

var json = "{\"foo\": \"bar\",\"baz\": [0, \"bla\", true, null]}";
wtx.insertSubtreeAsFirstChild(JsonShredder.createStringReader(json));

wtx.copySubtreeAsRightSibling(rtx);

resourceManager.beginNodeTrx(TimeUnit.SECONDS, 30);
resourceManager.beginNodeTrx(1000);
resourceManager.beginNodeTrx(1000, TimeUnit.SECONDS, 30);

resourceManager.beginNodeTrx().revertTo(2).commit();

var rtxOpenedByRevisionNumber = resourceManager.beginNodeReadOnlyTrx(2);

var dateTime = LocalDateTime.of(2019, Month.JUNE, 15, 13, 39);
var instant = dateTime.atZone(ZoneId.of("Europe/Berlin")).toInstant();
var rtxOpenedByTimestamp = resourceManager.beginNodeReadOnlyTrx(instant);

DiffFactory.invokeJsonDiff(
  new DiffFactory.Builder(
    resourceManager,
    2,
    1,
    DiffOptimized.HASHED,
    ImmutableSet.of(observer)));

public interface DiffObserver {
    void diffListener(DiffType diffType, long newNodeKey, long oldNodeKey, DiffDepth depth);
    void diffDone();
}

var writer = new StringWriter();
var serializer = new JsonSerializer.Builder(resourceManager, writer).build();
serializer.call();

var serializer = new
JsonSerializer.Builder(resourceManager, writer, 1, 2).build();
serializer.call();

var serializer = new
JsonSerializer.Builder(resourceManager, writer, -1).build();
serializer.call();