Using the JetS3t Java Client With Amazon S3 – 在亚马逊S3中使用JetS3t Java客户端

最后修改: 2018年 4月 17日

中文/混合/英文(键盘快捷键：t)

1. Overview

1.概述

In this tutorial, we’ll use the JetS3t library with Amazon S3.

在本教程中，我们将使用JetS3t库与Amazon S3.。

Simply put, we’ll create buckets, write data to them, read data back, copy it, and then list and delete them.

简单地说，我们将创建桶，向它们写数据，读回数据，复制它，然后列出并删除它们。

2. JetS3t Setup

2.JetS3t设置

2.1. Maven Dependency

2.1.Maven的依赖性

First, we need to add the NATS library and Apache HttpClient to our pom.xml:

首先，我们需要将NATS库和Apache HttpClient添加到我们的pom.xml。

<dependency>
    <groupId>org.lucee</groupId>
    <artifactId>jets3t</artifactId>
    <version>0.9.4.0006L</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.5</version>
</dependency>

Maven Central has the latest version of the JetS3t library and the latest version of HttpClient. The source for JetS3t can be found here.

Maven Central有最新版本的JetS3t库和最新版本的HttpClient。JetS3t的源代码可以在这里找到。

We’ll be using Apache Commons Codec for one of our tests, so we’ll add that to our pom.xml too:

我们将在其中一个测试中使用Apache Commons Codec，所以我们将把它添加到我们的pom.xml中。

<dependency>
    <groupId>org.lucee</groupId>
    <artifactId>commons-codec</artifactId>
    <version>1.10.L001</version>
</dependency>

Maven Central has the latest version here.

Maven Central有最新版本这里。

2.2. Amazon AWS Keys

2.2.亚马逊AWS密钥

We’ll need AWS Access Keys to connect to the S3 storage service. A free account can be created here.

我们需要AWS访问密钥来连接到S3存储服务。可以创建一个免费账户这里。

After we have an account, we need to create a set of security keys.There’s documentation about users and access keys available here.

有了账户之后，我们需要创建一套安全密钥。这里有关于用户和访问密钥的文档，。

JetS3t using Apache Commons logging, so we’ll use it too when we want to print information about what we’re doing.

JetS3t使用Apache Commons日志，所以当我们想打印关于我们正在做的事情的信息时，我们也会使用它。

3. Connecting to a Simple Storage

3.连接到一个简单存储

Now that we have an AWS access key and secret key, we can connect to S3 storage.

现在我们有了AWS访问密钥和秘密密钥，我们可以连接到S3存储。

3.1. Connecting to AWS

3.1.连接到AWS

First, we create AWS credentials and then use them to connect to the service:

首先，我们创建AWS凭证，然后用它们连接到服务。

AWSCredentials awsCredentials 
  = new AWSCredentials("access key", "secret key");
s3Service = new RestS3Service(awsCredentials);

RestS3Serviceis our connection to Amazon S3.It uses HttpClientto communicate with S3 over REST.

RestS3Service是我们与Amazon S3的连接。它使用HttpClient通过REST与S3通信。

3.2. Verifying Connection

3.2.验证连接

We can verify that we’ve successfully connected to the service by listing buckets:

我们可以通过列出水桶来验证我们已经成功连接到服务。

S3Bucket[] myBuckets = s3Service.listAllBuckets();

Depending on whether or not we’ve created buckets before, the array may be empty, but if the operation doesn’t throw an exception, we have a valid connection.

根据我们之前是否创建过水桶，数组可能是空的，但如果操作没有抛出异常，我们就有一个有效的连接。

4. Buckets Management

4.桶的管理

With a connection to Amazon S3, we can create buckets to hold our data. S3 is an object storage system. Data is uploaded as objects and stored in buckets.

通过与亚马逊S3的连接，我们可以创建桶来保存我们的数据。S3是一个对象存储系统。数据以对象的形式上传并存储在桶中。

Since all S3 buckets share the same global namespace, each one must have a unique name.

由于所有的S3桶共享相同的全局名称空间，所以每个桶必须有一个独特的名称。

4.1. Creating a Bucket

4.1.创建一个水桶

Let’s try to create a bucket name “mybucket“:

让我们尝试创建一个名为”mybucket“的桶。

S3Bucket bucket = s3Service.createBucket("mybucket");

This fails with an exception:

这失败了，有一个例外。

org.jets3t.service.S3ServiceException: Service Error Message.
  -- ResponseCode: 409, ResponseStatus: Conflict, XML Error Message:
  <!--?xml version="1.0" encoding="UTF-8"?-->
  <code>BucketAlreadyExists</code> The requested bucket name is not available. 
  The bucket namespace is shared by all users of the system.
  Please select a different name and try again.
  mybucket 07BE34FF3113ECCF 
at org.jets3t.service.S3Service.createBucket(S3Service.java:1586)

The name “mybucket” is, predictably, already taken. For the rest of the tutorial, we’ll make up our names.

可以预见的是，”mybucket“这个名字已经被使用了。在本教程的其余部分，我们将编造我们的名字。

Let’s try again with a different name:

让我们换个名字再试试。

S3Bucket bucket = s3Service.createBucket("myuniquename");
log.info(bucket);

With a unique name, the call succeeds, and we see information about our bucket:

有了唯一的名字，调用就成功了，我们就能看到关于我们的桶的信息。

[INFO] JetS3tClient - S3Bucket
[name=myuniquename,location=US,creationDate=Sat Mar 31 16:47:47 EDT 2018,owner=null]

4.2. Deleting a Bucket

4.2.删除一个水桶

Deleting a bucket is as easy as creating it, except for one thing; buckets must be empty before they can be removed!

删除一个桶和创建一个桶一样简单，除了一件事；桶必须是空的才能被删除！。

s3Service.deleteBucket("myuniquename");

This will throw an exception for a bucket that is not empty.

这将为一个不空的桶抛出一个异常。

4.3. Specifying the Bucket Region

4.3.指定桶的区域

Buckets can be created in a specific data center.For JetS3t the default is Northern Virginia in the United States, or “us-east-1.”

可以在一个特定的数据中心创建桶。对于JetS3t，默认是美国的北弗吉尼亚，或 “us-east-1″。

We can override this by specifying a different region:

我们可以通过指定一个不同的区域来覆盖这一点。

S3Bucket euBucket 
  = s3Service.createBucket("eu-bucket", S3Bucket.LOCATION_EUROPE);
S3Bucket usWestBucket = s3Service
  .createBucket("us-west-bucket", S3Bucket.LOCATION_US_WEST);
S3Bucket asiaPacificBucket = s3Service
  .createBucket("asia-pacific-bucket", S3Bucket.LOCATION_ASIA_PACIFIC);

JetS3t has an extensive list of regions defined as constants.

JetS3t有一个广泛的定义为常数的区域列表。

5. Upload, Download, and Delete Data

5.上传、下载和删除数据

Once we have a bucket, we can add objects to it. Buckets are intended to be long-lasting, and there’s no hard limit on the size or number of objects a bucket can contain.

一旦我们有了一个桶，我们就可以向它添加对象。桶的目的是为了长期使用，而且对一个桶可以包含的对象的大小或数量没有硬性限制。

Data is uploaded to S3 by creating S3Objects.We can upload data a from an InputStream,but JetS3t also provides convenience methods for Stringsand Files.

数据通过创建S3Objects上传至S3。我们可以从InputStream上传数据，但JetS3t也为Strings和Files提供便利方法。

5.1. StringData

5.1.字符串数据

Let’s take a look at Stringsfirst:

让我们先看一下字符串。

S3Object stringObject = new S3Object("object name", "string object");
s3Service.putObject("myuniquebucket", stringObject);

Similar to buckets, objects have names, however, object names only live inside their buckets, so we don’t have to worry about them being globally unique.

与桶类似，对象也有名字，不过，对象的名字只存在于其桶内，所以我们不必担心它们是全局唯一的。

We create the object by passing a name and the data to the constructor. Then we store it with putObject.

我们通过向构造函数传递一个名称和数据来创建对象。然后我们用putObject.存储它。

When we use this method to store Stringswith JetS3t, it sets the correct content type for us.

当我们使用这个方法来存储字符串与JetS3t，它为我们设置了正确的内容类型。

Let’s query S3 for information about our object and look at the content type:

让我们在S3中查询关于我们对象的信息，看看内容类型。

StorageObject objectDetailsOnly 
  = s3Service.getObjectDetails("myuniquebucket", "my string");
log.info("Content type: " + objectDetailsOnly.getContentType() + " length: " 
  + objectDetailsOnly.getContentLength());

ObjectDetailsOnly()retrieves the objects metadata without downloading it. When we log the content type we see:

ObjectDetailsOnly()检索对象的元数据而不下载它。当我们记录内容类型时，我们看到。

[INFO] JetS3tClient - Content type: text/plain; charset=utf-8 length: 9

JetS3t identified the data as text and set the length for us.

JetS3t将数据识别为文本，并为我们设置了长度。

Let’s download the data and compare it to what we uploaded:

让我们下载数据并与我们上传的数据进行比较。

S3Object downloadObject = 
  s3Service.getObject("myuniquebucket, "string object");
String downloadString = new BufferedReader(new InputStreamReader(
  object.getDataInputStream())).lines().collect(Collectors.joining("\n"));
 
assertTrue("string object".equals(downloadString));

Data is retrieved in the same S3Objectwe use to upload it, with the bytes available in a DataInputStream.

数据在我们用来上传的同一个S3Object中被检索，其字节数在DataInputStream中可用。

5.2. File Data

5.2.文件数据

The process for uploading files is similar to Strings:

上传文件的过程与Strings相似。

File file = new File("src/test/resources/test.jpg");
S3Object fileObject = new S3Object(file);
s3Service.putObject("myuniquebucket", fileObject);

When S3Objectsare passed a File they derive their name from the base name of the files they contain:

当S3Objects被传递给File时，它们从它们所包含的文件的基本名称中获得了它们的名字。

[INFO] JetS3tClient - File object name is test.jpg

JetS3t takes the File and uploads it for us.It will attempt to load a mime.types filefrom the classpath and use it to identify the type of file and sent content type appropriately.

JetS3t接收文件并为我们上传。它将尝试从classpath加载mime.types文件，并使用它来识别文件的类型并适当地发送内容类型。

If we retrieve the object info of our file upload and get the content type we see:

如果我们检索我们的文件上传的对象信息并获得内容类型，我们会看到。

[INFO] JetS3tClient - Content type:application/octet-stream

Let’s download our file to a new one and compare the contents:

让我们把我们的文件下载到一个新的文件，并比较其内容。

String getFileMD5(String filename) throws IOException {
    try (FileInputStream fis = new FileInputStream(new File(filename))) {
        return DigestUtils.md5Hex(fis);
    }
}

S3Object fileObject = s3Service.getObject("myuniquebucket", "test.jpg"); 
File newFile = new File("/tmp/newtest.jpg"); 
Files.copy(fileObject.getDataInputStream(), newFile.toPath(), 
  StandardCopyOption.REPLACE_EXISTING);
String origMD5 = getFileMD5("src/test/resources/test.jpg");
String newMD5 = getFileMD5("src/test/resources/newtest.jpg");
assertTrue(origMD5.equals(newMD5));

Similar to Stringswe downloaded the object and used the DataInputStream to create a new file. Then we calculated an MD5 hash for both files and compared them.

与字符串类似，我们下载了对象并使用DataInputStream创建了一个新文件。然后我们为两个文件计算了一个MD5哈希值并进行了比较。

5.3. Streaming Data

5.3.流动数据

When we upload objects other than Stringsor Files,we have a bit more work to do:

当我们上传字符串或文件以外的对象时，我们有更多的工作要做：。

ArrayList<Integer> numbers = new ArrayList<>();
// adding elements to the ArrayList

ByteArrayOutputStream bytes = new ByteArrayOutputStream();
ObjectOutputStream objectOutputStream = new ObjectOutputStream(bytes);
objectOutputStream.writeObject(numbers);

ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes.toByteArray());

S3Object streamObject = new S3Object("stream");
streamObject.setDataInputStream(byteArrayInputStream);
streamObject.setContentLength(byteArrayInputStream.available());
streamObject.setContentType("binary/octet-stream");

s3Service.putObject(BucketName, streamObject);

We need to set our content type and length before uploading.

我们需要在上传前设置我们的内容类型和长度。

Retrieving this stream means reversing the process:

检索这个流意味着反转这个过程。

S3Object newStreamObject = s3Service.getObject(BucketName, "stream");

ObjectInputStream objectInputStream = new ObjectInputStream(
  newStreamObject.getDataInputStream());
ArrayList<Integer> newNumbers = (ArrayList<Integer>) objectInputStream
  .readObject();

assertEquals(2, (int) newNumbers.get(0));
assertEquals(3, (int) newNumbers.get(1));
assertEquals(5, (int) newNumbers.get(2));
assertEquals(7, (int) newNumbers.get(3));

For different data types, the content type property can be used to select a different method for decoding the object.

对于不同的数据类型，可以使用内容类型属性来选择不同的方法对对象进行解码。

6. Copying, Moving and Renaming Data

6.复制、移动和重命名数据

6.1. Copying Objects

6.1.复制对象

Objects can be copied inside S3, without retrieving them.

对象可以在S3内部被复制，而不需要检索它们。

Let’s copy our test file from section 5.2, and verify the result:

让我们从第5.2节复制我们的测试文件，并验证其结果。

S3Object targetObject = new S3Object("testcopy.jpg");
s3Service.copyObject(
  BucketName, "test.jpg", 
  "myuniquebucket", targetObject, false);
S3Object newFileObject = s3Service.getObject(
  "myuniquebucket", "testcopy.jpg");

File newFile = new File("src/test/resources/testcopy.jpg");
Files.copy(
  newFileObject.getDataInputStream(), 
  newFile.toPath(), 
  REPLACE_EXISTING);
String origMD5 = getFileMD5("src/test/resources/test.jpg");
String newMD5 = getFileMD5("src/test/resources/testcopy.jpg");
 
assertTrue(origMD5.equals(newMD5));

We can copy objects inside the same bucket, or between two different ones.

我们可以在同一个桶内复制对象，或者在两个不同的桶之间复制。

If the last argument is true, the copied object will receive new metadata. Otherwise, it will retain the source object’s metadata.

如果最后一个参数为真，被复制的对象将收到新的元数据。否则，它将保留源对象的元数据。

If we want to modify the metadata, we can set the flag to true:

如果我们想修改元数据，我们可以将该标志设置为 “true”。

targetObject = new S3Object("testcopy.jpg");
targetObject.addMetadata("My_Custom_Field", "Hello, World!");
s3Service.copyObject(
  "myuniquebucket", "test.jpg", 
  "myuniquebucket", targetObject, true);

6.2. Moving Objects

6.2.移动物体

Objects can be moved to another S3 bucket in the same region.A move operation is a copy then a delete operation.

对象可以被移动到同一区域的另一个S3桶。移动操作是先复制后删除的操作。

If the copy operation fails, the source object isn’t deleted. If the delete operation fails, the object will still exist in the source and also in the destination location.

如果复制操作失败，源对象不会被删除。如果删除操作失败，该对象将仍然存在于源位置和目标位置。

Moving an object looks similar to copying it:

移动一个物体看起来类似于复制它。

s3Service.moveObject(
  "myuniquebucket",
  "test.jpg",
  "myotheruniquebucket",
  new S3Object("spidey.jpg"),
  false);

6.3. Renaming Objects

6.3.重命名对象

JetS3t has a convenience method for renaming objects.  To change an objects name we merely call it with a new S3Object:

JetS3t有一个方便的方法来重命名对象。要改变一个对象的名字，我们只需用一个新的S3Object来调用它。

s3Service.renameObject(
  "myuniquebucket", "test.jpg", new S3Object("spidey.jpg"));

7. Conclusion

7.结论

In this tutorial, we used JetS3t to connect to Amazon S3. We created and deleted buckets. Then we added different types of data to buckets and retrieved the data. To wrap things up, we copied and moved our data.

在本教程中，我们使用JetS3t连接到Amazon S3。我们创建并删除了桶。然后，我们在桶中添加了不同类型的数据，并检索了数据。为了收尾，我们复制和移动了我们的数据。

Code samples, as always, can be found over on GitHub.

像往常一样，可以在GitHub上找到代码样本。