[
https://issues.apache.org/jira/browse/IMPALA-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated IMPALA-8581:
---------------------------------
Labels: newbie (was: )
> Switch hdfs commandline calls in dataload and tests to use "-d" on s3 where
> applicable
> --------------------------------------------------------------------------------------
>
> Key: IMPALA-8581
> URL: https://issues.apache.org/jira/browse/IMPALA-8581
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 3.3.0
> Reporter: Joe McDonnell
> Priority: Minor
> Labels: newbie
>
> When doing a put or copyFromLocal via the HDFS commandline, it puts the file
> into a temporary file and then moves it into place. The "-d" option skips the
> temporary file and copies the file directly into place. On S3, this avoids an
> extra copy and can also avoid consistency issues. We should look into using
> the "-d" option across our tests. Our usages of the HDFS commandline
> generally don't require atomicity of this file copy, so using "-d" is
> unlikely to impact HDFS tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
