Upload on S3 with the jclouds library – 用jclouds库在S3上上传

最后修改: 2013年 4月 1日

中文/混合/英文(键盘快捷键：t)

There are several good ways to upload content to an S3 bucket in the Java world – in this article we’ll look at what the jclouds library provides for this purpose.

在Java世界中，有几种将内容上传到S3桶的好方法–在本文中，我们将看看jclouds库为此目的提供什么。

To use jclouds – specifically the APIs discussed in this article, this simple Maven dependency should be added to the pom of the project:

要使用jclouds–特别是本文讨论的API，应将此simple Maven依赖添加到项目的pom。

<dependency>
   <groupId>org.jclouds</groupId>
   <artifactId>jclouds-allblobstore</artifactId>
   <version>1.5.10</version>
</dependency>

1. Uploading to Amazon S3

1.上传至Amazon S3

The first step, in order to access any of these APIs, is to create a BlobStoreContext:

为了访问这些API，第一步是创建一个BlobStoreContext。

BlobStoreContext context = 
  ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
    .buildView(BlobStoreContext.class);

This represents the entry-point to a general key-value storage service, such as Amazon S3 – but not limited to it.

这代表了一般键值存储服务的入口，如Amazon S3 – 但不限于此。

For the more specific S3 only implementation, the context can be created similarly:

对于更具体的只有S3的实现，可以用类似的方法创建上下文。

BlobStoreContext context = 
  ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
    .buildView(S3BlobStoreContext.class);

And even more specifically:

甚至更具体地说。

BlobStoreContext context = 
  ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
    .buildView(AWSS3BlobStoreContext.class);

When the authenticated context is no longer needed, closing it is required to release all resources – threads and connections – associated to it.

当不再需要认证的上下文时，关闭它需要释放与之相关的所有资源 – 线程和连接。

2. The Four S3 APIs of jclouds

2.jclouds的四个S3 APIs

The jclouds library provides four different APIs to upload content to S3 bucket, ranging from simple but inflexible to complex and powerful, all obtained via the BlobStoreContext. Let’s start with the simplest.

jclouds库提供了四种不同的API来上传内容到S3桶，从简单而不灵活到复杂而强大，都是通过BlobStoreContext获得。让我们从最简单的开始。

2.1. Upload via the Map API

2.1.通过Map API上传

The easiest way jclouds can be used to interact with an S3 bucket is by representing that bucket as a Map. The API is obtained from the context:

使用jclouds与S3桶进行交互的最简单方法是将该桶表示为一个Map。API是从上下文中获得的。

InputStreamMap bucket = context.createInputStreamMap("bucketName");

Then, to upload a simple HTML file:

然后，要上传一个简单的HTML文件。

bucket.putString("index1.html", "<html><body>hello world1</body></html>");

The InputStreamMap API exposes several other types of PUT operations – files, raw bytes – both for single and bulk.

InputStreamMap API公开了其他几种类型的PUT操作–文件、原始字节–包括单一和批量。

A simple integration test can be used as an example:

可以用一个简单的集成测试作为例子。

@Test
public void whenFileIsUploadedToS3WithMapApi_thenNoExceptions() {
   BlobStoreContext context = 
      ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
         .buildView(AWSS3BlobStoreContext.class);

   InputStreamMap bucket = context.createInputStreamMap("bucketName");  

   bucket.putString("index1.html", "<html><body>hello world1</body></html>");
   context.close();
}

2.2. Upload via BlobMap

2.2.通过BlobMap上传

Using the simple Map API is straightforward but ultimately limited – for example, there is no way to pass in metadata about the content being uploaded. When more flexibility and customization is necessary, this simplified approach to uploading data to S3 via a Map is no longer enough.

使用简单的地图API是直截了当的，但最终还是有局限性–例如，没有办法传入关于被上传内容的元数据。当需要更多的灵活性和定制时，这种通过地图将数据上传到S3的简化方法已经不够了。

The next API we’ll look at is the Blob Map API – this is obtained from the context:

我们要看的下一个API是Blob Map API – 这是从上下文中获得的。

BlobMap bucket = context.createBlobMap("bucketName");

The API allows the client to access more lower level details, such as Content–Length, Content-Type, Content-Encoding, eTag hash and others; to upload new content in the bucket:

该API允许客户端访问更多低级别的细节，如Content–Length, Content-Type, Content-Encoding, eTag hash和其他；在桶中上传新内容。

Blob blob = bucket.blobBuilder().name("index2.html").
   payload("<html><body>hello world2</body></html>").
      contentType("text/html").calculateMD5().build();

The API also allows setting a variety of payloads on the create request.

该API还允许在创建请求上设置各种有效载荷。

A simple integration test for uploading a basic HTML file to S3 via the Blob Map API:

通过Blob Map API向S3上传基本HTML文件的简单集成测试。

@Test
public void whenFileIsUploadedToS3WithBlobMap_thenNoExceptions() throws IOException {
   BlobStoreContext context = 
      ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
         .buildView(AWSS3BlobStoreContext.class);

   BlobMap bucket = context.createBlobMap("bucketName");

   Blob blob = bucket.blobBuilder().name("index2.html").
      payload("<html><body>hello world2</body></html>").
         contentType("text/html").calculateMD5().build();
   bucket.put(blob.getMetadata().getName(), blob);

   context.close();
}

2.3. Upload via BlobStore

2.3.通过BlobStore上传

The previous APIs had no way to upload content using multipart upload – this makes them ill suited when working with large files. This limitation is addressed by the next API we’re going to look at – the synchronous BlobStore API.

之前的API无法使用多部分上传来上传内容，这使得它们不适合处理大文件。我们要研究的下一个API解决了这一限制，即同步BlobStore API。

This is obtained from the context:

这是从上下文中得到的。

BlobStore blobStore = context.getBlobStore();

To use the multipart support and upload a file to S3:

要使用多部分支持并上传文件到S3。

Blob blob = blobStore.blobBuilder("index3.html").
   payload("<html><body>hello world3</body></html>").contentType("text/html").build();
blobStore.putBlob("bucketName", blob, PutOptions.Builder.multipart());

The payload builder is the same one that was being used by the BlobMap API, so the same flexibility in specifying lower level metadata information about the blob is available here. The difference is the PutOptions supported by the PUT operation of the API – namely the multipart support.

有效载荷生成器与BlobMap API所使用的相同，因此在指定有关blob的低级元数据信息方面，这里也有同样的灵活性。不同之处在于API的PUT操作所支持的PutOptions，即multipart support。

The previous integration test now has multipart enabled:

以前的集成测试现在启用了多部分。

@Test
public void whenFileIsUploadedToS3WithBlobStore_thenNoExceptions() {
   BlobStoreContext context = 
      ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
         .buildView(AWSS3BlobStoreContext.class);

   BlobStore blobStore = context.getBlobStore();

   Blob blob = blobStore.blobBuilder("index3.html").
      payload("<html><body>hello world3</body></html>").contentType("text/html").build();
   blobStore.putBlob("bucketName", blob, PutOptions.Builder.multipart());
   context.close();
}

2.4. Upload via AsyncBlobStore

2.4.通过AsyncBlobStore上传

While the previous BlobStore API was synchronous, there is also an asynchronous API for BlobStore – AsyncBlobStore. The API is similarly obtained from the context:

虽然之前的BlobStore API是同步的，但也有异步API用于BlobStore – AsyncBlobStore。该API类似于从上下文中获得。

AsyncBlobStore blobStore = context.getAsyncBlobStore();

The only difference between the two is that the async API is returning ListenableFuture for the PUT asynchronous operation:

两者之间唯一的区别是，异步API为PUT异步操作返回ListenableFuture。

Blob blob = blobStore.blobBuilder("index4.html").
   .payload("<html><body>hello world4</body></html>").build();
blobStore.putBlob("bucketName", blob)<strong>.get()>;

The integration test displaying this operation is similar to the synchronous one:

显示这一操作的集成测试与同步测试类似。

@Test
public void whenFileIsUploadedToS3WithBlobStore_thenNoExceptions() {
   BlobStoreContext context = 
      ContextBuilder.newBuilder("aws-s3").credentials(identity, credentials)
         .buildView(AWSS3BlobStoreContext.class);

   BlobStore blobStore = context.getBlobStore();

   Blob blob = blobStore.blobBuilder("index4.html").
      payload("<html><body>hello world4</body></html>").contentType("text/html").build();
   Future<String> putOp = blobStore.putBlob("bucketName", blob, PutOptions.Builder.multipart());
   putOp.get();
   context.close();
}

3. Conclusion

3.结论

In this article, we analyzed the four APIs that the jclouds library provides to upload content to Amazon S3. These four APIs are generic and they work with other key-value storage services as well – such as Microsoft Azure Storage for example.

在这篇文章中，我们分析了jclouds库为上传内容到Amazon S3所提供的四个API。这四个API是通用的，它们也适用于其他键值存储服务–例如Microsoft Azure Storage。

In the next article we’ll look at the Amazon specific S3 API available in jclouds – the AWSS3Client. We’ll implement the operation of uploading a large file, dynamically calculate the optimal number of parts for any given file, and perform the upload of all parts in parallel.

在下一篇文章中，我们将看看jclouds中可用的亚马逊特定S3 API – AWSS3Client。我们将实现上传大文件的操作，动态地计算任何给定文件的最佳部件数量，并并行地执行所有部件的上传。