[
https://issues.apache.org/jira/browse/HADOOP-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179511#comment-16179511
]
Georgi Chalakov edited comment on HADOOP-14906 at 9/25/17 6:28 PM:
-------------------------------------------------------------------
I am not sure that this is related to the block compaction change. The debug
message shows no directories in the list for block blobs with compaction. I
posted the code where we check whether the file is in one of those directories
and if it is not we skip BlockBlobAppendStream.
{quote}
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]:
azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction
directories:
{quote}
{code:title=hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java|borderStyle=solid}
if (isBlockBlobWithCompactionKey(key)) {
BlockBlobAppendStream blockBlobOutputStream = new BlockBlobAppendStream(
(CloudBlockBlobWrapper) blob,
keyEncoded,
this.uploadBlockSizeBytes,
true,
getInstrumentedContext());
outputStream = blockBlobOutputStream;
} else {
outputStream = openOutputStream(blob);
}
{code}
was (Author: georgi):
I am not sure that this is related to the block compaction change. The debug
message shows no directories in the list for block blobs with compaction. I
posted the code where we check whether the file is in one of those directories
and if it is not we skip BlockBlobAppendStream.
{noformat}
2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]:
azure.AzureNativeFileSystemStore
(AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction
directories:
{noformat}
{code:title=hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java|borderStyle=solid}
if (isBlockBlobWithCompactionKey(key)) {
BlockBlobAppendStream blockBlobOutputStream = new BlockBlobAppendStream(
(CloudBlockBlobWrapper) blob,
keyEncoded,
this.uploadBlockSizeBytes,
true,
getInstrumentedContext());
outputStream = blockBlobOutputStream;
} else {
outputStream = openOutputStream(blob);
}
{code}
> ITestAzureConcurrentOutOfBandIo failing with checksum errors on write
> ---------------------------------------------------------------------
>
> Key: HADOOP-14906
> URL: https://issues.apache.org/jira/browse/HADOOP-14906
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 2.9.0, 3.1.0
> Environment: UK BT ASDL connection, 1.8.0_121-b13, azure storage
> ireland
> Reporter: Steve Loughran
>
> {{ITestAzureConcurrentOutOfBandIo}} is consistently raising an IOE with the
> text "The MD5 value specified in the request did not match with the MD5 value
> calculated by the server"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
