[
https://issues.apache.org/jira/browse/HADOOP-14520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Georgi Chalakov updated HADOOP-14520:
-------------------------------------
Description:
Block Compaction for WASB allows uploading new blocks for every hflush/hsync
call. When the number of blocks is above 32000, next hflush/hsync triggers the
block compaction process. Block compaction replaces a sequence of blocks with
one block. From all the sequences with total length less than 4M, compaction
chooses the longest one. It is a greedy algorithm that preserve all potential
candidates for the next round. Block Compaction for WASB increases data
durability and allows using block blobs instead of page blobs. By default,
block compaction is disabled. Similar to the configuration for page blobs, the
client needs to specify HDFS folders where block compaction over block blobs is
enabled.
Results for HADOOP-14520-04.patch
tested endpoint: fs.azure.account.key.hdfs4.blob.core.windows.net
Tests run: 701, Failures: 0, Errors: 0, Skipped: 119
was:
Block Compaction for WASB allows uploading new blocks for every hflush/hsync
call. When the number of blocks is above 32000, next hflush/hsync triggers the
block compaction process. Block compaction replaces a sequence of blocks with
one block. From all the sequences with total length less than 4M, compaction
chooses the longest one. It is a greedy algorithm that preserve all potential
candidates for the next round. Block Compaction for WASB increases data
durability and allows using block blobs instead of page blobs. By default,
block compaction is disabled. Similar to the configuration for page blobs, the
client needs to specify HDFS folders where block compaction over block blobs is
enabled.
Results for HADOOP-14520-01.patch
tested endpoint: fs.azure.account.key.hdfs4.blob.core.windows.net
Tests run: 704, Failures: 0, Errors: 0, Skipped: 119
> Block compaction for WASB (Block Blobs Instead of Page Plobs)
> -------------------------------------------------------------
>
> Key: HADOOP-14520
> URL: https://issues.apache.org/jira/browse/HADOOP-14520
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/azure
> Affects Versions: 3.0.0-alpha3
> Reporter: Georgi Chalakov
> Assignee: Georgi Chalakov
> Attachments: HADOOP-14520-01.patch, HADOOP-14520-01-test.txt,
> HADOOP-14520-03.patch
>
>
> Block Compaction for WASB allows uploading new blocks for every hflush/hsync
> call. When the number of blocks is above 32000, next hflush/hsync triggers
> the block compaction process. Block compaction replaces a sequence of blocks
> with one block. From all the sequences with total length less than 4M,
> compaction chooses the longest one. It is a greedy algorithm that preserve
> all potential candidates for the next round. Block Compaction for WASB
> increases data durability and allows using block blobs instead of page blobs.
> By default, block compaction is disabled. Similar to the configuration for
> page blobs, the client needs to specify HDFS folders where block compaction
> over block blobs is enabled.
> Results for HADOOP-14520-04.patch
> tested endpoint: fs.azure.account.key.hdfs4.blob.core.windows.net
> Tests run: 701, Failures: 0, Errors: 0, Skipped: 119
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
