[
https://issues.apache.org/jira/browse/IMPALA-9437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236256#comment-17236256
]
Arpit Agarwal commented on IMPALA-9437:
---------------------------------------
Perhaps we could make it simpler for Impala by returning the fixed block size
from configuration.
> Cannot get Ozone file block size after opening the file
> -------------------------------------------------------
>
> Key: IMPALA-9437
> URL: https://issues.apache.org/jira/browse/IMPALA-9437
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Sahil Takiar
> Priority: Major
>
> When creating a tmp file on HDFS, {{HdfsTableSink::CreateNewTmpFile}} first
> opens the file, and then stats the file ({{hdfsGetPathInfo}}) before actually
> writing any data to the file or closing the file. HDFS seems to allow this
> behavior. However, Ozone, S3A, and ABFS do not. Impala does this for HDFS in
> order to get the block size of the opened file. According to
> {{HdfsTableSink}} it is possible for HDFS to create a block size with a
> different one than requested by Impala. So in order to track the correct
> block size for a file, the file needs to be stat'ed after opening it. For S3A
> and ABFS this isn't a big deal, because they aren't block based filesystem,
> but Ozone is. So we should investigate the impact of not having this
> capability and consider adding it to the Ozone client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
