[ https://issues.apache.org/jira/browse/HADOOP-14906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179511#comment-16179511 ]
Georgi Chalakov edited comment on HADOOP-14906 at 9/25/17 6:27 PM: ------------------------------------------------------------------- I am not sure that this is related to the block compaction change. The debug message shows no directories in the list for block blobs with compaction. I posted the code where we check whether the file is in one of those directories and if it is not we skip BlockBlobAppendStream. {noformat} 2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction directories: {noformat} {code:title=hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java|borderStyle=solid} if (isBlockBlobWithCompactionKey(key)) { BlockBlobAppendStream blockBlobOutputStream = new BlockBlobAppendStream( (CloudBlockBlobWrapper) blob, keyEncoded, this.uploadBlockSizeBytes, true, getInstrumentedContext()); outputStream = blockBlobOutputStream; } else { outputStream = openOutputStream(blob); } {code} was (Author: georgi): I am not sure that this is related to the block compaction change. The debug message shows no directories in the list for block blobs with compaction. I posted the code where we check whether the file is in one of those directories and if it is not we skip BlockBlobAppendStream. 2017-09-25 14:47:17,484 DEBUG [JUnit-testReadOOBWrites]: azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:initialize(550)) - Block blobs with compaction directories: {code:title=hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java|borderStyle=solid} if (isBlockBlobWithCompactionKey(key)) { BlockBlobAppendStream blockBlobOutputStream = new BlockBlobAppendStream( (CloudBlockBlobWrapper) blob, keyEncoded, this.uploadBlockSizeBytes, true, getInstrumentedContext()); outputStream = blockBlobOutputStream; } else { outputStream = openOutputStream(blob); } {code} > ITestAzureConcurrentOutOfBandIo failing with checksum errors on write > --------------------------------------------------------------------- > > Key: HADOOP-14906 > URL: https://issues.apache.org/jira/browse/HADOOP-14906 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure > Affects Versions: 2.9.0, 3.1.0 > Environment: UK BT ASDL connection, 1.8.0_121-b13, azure storage > ireland > Reporter: Steve Loughran > > {{ITestAzureConcurrentOutOfBandIo}} is consistently raising an IOE with the > text "The MD5 value specified in the request did not match with the MD5 value > calculated by the server" -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org