[
https://issues.apache.org/jira/browse/CAMEL-8768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hassan Faouaz updated CAMEL-8768:
---------------------------------
Description:
If you need to produce files into an existing HDFS2 path, the default behavoir
is overwrite the path, which will delete all existing files on HDFS. If
overwrite option is disabled, then the component will complain that the
existing HDFS directory exists and will not work.
The propose solution is to add the following if statement to ignore check if
the HDFS directory exists. The overwrite option should only be used for files
not directories.
code snippet in HdfsOutputStream.java and patch is attached
if (ret.info.getFileSystem().exists(new Path(ret.actualPath))) {
//only check of not directory
if (!ret.info.getFileSystem().isDirectory(new
Path(ret.actualPath))) {
if (configuration.isOverwrite()) {
ret.info.getFileSystem().delete(new
Path(ret.actualPath), true);
} else {
throw new RuntimeCamelException("The file already
exists");
}
}
}
was:
If you need to produce files into an existing HDFS2 path, the default behavoir
is overwrite the path, which will delete all existing files on HDFS. If
overwrite option is disabled, then the component will complain that the
existing HDFS directory exists and will not work.
The propose solution is to add the following if statement to ignore check if
the HDFS directory exists. The overwrite option should only be used for files
not directories.
code snippet in HdfsOutputStream.java:
if (ret.info.getFileSystem().exists(new Path(ret.actualPath))) {
//only check of not directory
if (!ret.info.getFileSystem().isDirectory(new
Path(ret.actualPath))) {
if (configuration.isOverwrite()) {
ret.info.getFileSystem().delete(new
Path(ret.actualPath), true);
} else {
throw new RuntimeCamelException("The file already
exists");
}
}
}
> hdfs2 component overwrite option is also being applied to directory
> filesystem path
> -----------------------------------------------------------------------------------
>
> Key: CAMEL-8768
> URL: https://issues.apache.org/jira/browse/CAMEL-8768
> Project: Camel
> Issue Type: Bug
> Components: camel-hdfs
> Affects Versions: 2.15.2
> Reporter: Hassan Faouaz
> Priority: Critical
> Labels: hadoop
> Attachments: patchfile.txt
>
>
> If you need to produce files into an existing HDFS2 path, the default
> behavoir is overwrite the path, which will delete all existing files on HDFS.
> If overwrite option is disabled, then the component will complain that the
> existing HDFS directory exists and will not work.
> The propose solution is to add the following if statement to ignore check if
> the HDFS directory exists. The overwrite option should only be used for files
> not directories.
> code snippet in HdfsOutputStream.java and patch is attached
> if (ret.info.getFileSystem().exists(new Path(ret.actualPath))) {
> //only check of not directory
> if (!ret.info.getFileSystem().isDirectory(new
> Path(ret.actualPath))) {
> if (configuration.isOverwrite()) {
> ret.info.getFileSystem().delete(new
> Path(ret.actualPath), true);
> } else {
> throw new RuntimeCamelException("The file already
> exists");
> }
> }
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
