org.apache.hadoop.fs

Interface MultipartUploader

All Superinterfaces:

AutoCloseable, Closeable, org.apache.hadoop.fs.statistics.IOStatisticsSource

@InterfaceAudience.Public
 @InterfaceStability.Unstable
public interface MultipartUploader
extends Closeable, org.apache.hadoop.fs.statistics.IOStatisticsSource

MultipartUploader is an interface for copying files multipart and across multiple nodes.

The interface extends IOStatisticsSource so that there is no need to cast an instance to see if is a source of statistics. However, implementations MAY return null for their actual statistics.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Void>	abort(UploadHandle uploadId, Path filePath) Aborts a multipart upload.
CompletableFuture<Integer>	abortUploadsUnderPath(Path path) Best effort attempt to aborts multipart uploads under a path.
CompletableFuture<PathHandle>	complete(UploadHandle uploadId, Path filePath, Map<Integer,PartHandle> handles) Complete a multipart upload.
CompletableFuture<PartHandle>	putPart(UploadHandle uploadId, int partNumber, Path filePath, InputStream inputStream, long lengthInBytes) Put part as part of a multipart upload.
CompletableFuture<UploadHandle>	startUpload(Path filePath) Initialize a multipart upload.

Methods inherited from interface java.io.Closeable

close

Methods inherited from interface org.apache.hadoop.fs.statistics.IOStatisticsSource

getIOStatistics

Method Detail

startUpload

CompletableFuture<UploadHandle> startUpload(Path filePath)
                                     throws IOException

Initialize a multipart upload.

Parameters:

filePath - Target path for upload.

Returns:

unique identifier associating part uploads.

Throws:

IOException - IO failure

putPart

CompletableFuture<PartHandle> putPart(UploadHandle uploadId,
                                      int partNumber,
                                      Path filePath,
                                      InputStream inputStream,
                                      long lengthInBytes)
                               throws IOException

Put part as part of a multipart upload. It is possible to have parts uploaded in any order (or in parallel).

Parameters:

uploadId - Identifier from startUpload(Path).

partNumber - Index of the part relative to others.

filePath - Target path for upload (as startUpload(Path)).

inputStream - Data for this part. Implementations MUST close this stream after reading in the data.

lengthInBytes - Target length to read from the stream.

Returns:

unique PartHandle identifier for the uploaded part.

Throws:

IOException - IO failure

complete

CompletableFuture<PathHandle> complete(UploadHandle uploadId,
                                       Path filePath,
                                       Map<Integer,PartHandle> handles)
                                throws IOException

Complete a multipart upload.

Parameters:

uploadId - Identifier from startUpload(Path).

filePath - Target path for upload (as startUpload(Path).

handles - non-empty map of part number to part handle. from putPart(UploadHandle, int, Path, InputStream, long).

Returns:

unique PathHandle identifier for the uploaded file.

Throws:

IOException - IO failure

abort

CompletableFuture<Void> abort(UploadHandle uploadId,
                              Path filePath)
                       throws IOException

Aborts a multipart upload.

Parameters:

uploadId - Identifier from startUpload(Path).

filePath - Target path for upload (same as startUpload(Path).

Returns:

a future; the operation will have completed

Throws:

IOException - IO failure

abortUploadsUnderPath

CompletableFuture<Integer> abortUploadsUnderPath(Path path)
                                          throws IOException

Best effort attempt to aborts multipart uploads under a path. Not all implementations support this, and those which do may be vulnerable to eventually consistent listings of current uploads -some may be missed.

Parameters:

path - path to abort uploads under.

Returns:

a future to the number of entries aborted; -1 if aborting is unsupported

Throws:

IOException - IO failure