Xiaolei Liu created HDFS-15183:
----------------------------------
Summary: For AzureNativeFS, when BlockCompaction is enabled,
FileSystem.create(path).close() would throw exception.
Key: HDFS-15183
URL: https://issues.apache.org/jira/browse/HDFS-15183
Project: Hadoop HDFS
Issue Type: Bug
Components: fs/azure
Affects Versions: 3.2.1, 2.9.2
Environment: macOS Mojave 10.14.6
Reporter: Xiaolei Liu
For AzureNativeFS, when BlockCompaction is enabled,
FileSystem.create(path).close() would throw blob not existed exception.
Block Compaction Setting: fs.azure.block.blob.with.compaction.dir
Exception is thrown from close(), this would happen when no write happened.
When actually write any content in the file, same context close() won't trigger
the exception.
When BlockCompaction is not enabled, this issue won't happen.
Call Stack:
org.apache.hadoop.fs.azure.AzureException: Source blob
_$azuretmpfolder$/956457df-4a3e-4285-bc68-29f68b9b36c4test1911.log does not
exist.
org.apache.hadoop.fs.azure.AzureException: Source blob
_$azuretmpfolder$/956457df-4a3e-4285-bc68-29f68b9b36c4test1911.log does not
exist.
at
org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2648)
at
org.apache.hadoop.fs.azure.AzureNativeFileSystemStore.rename(AzureNativeFileSystemStore.java:2608)
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsOutputStream.restoreKey(NativeAzureFileSystem.java:1199)
at
org.apache.hadoop.fs.azure.NativeAzureFileSystem$NativeAzureFsOutputStream.close(NativeAzureFileSystem.java:1068)
at
org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:72)
at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:106)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
