[
https://issues.apache.org/jira/browse/CAMEL-18776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17641303#comment-17641303
]
Claus Ibsen commented on CAMEL-18776:
-------------------------------------
We should backport to 3.18 and maybe also 3.14 LTSs
> camel-hdfs - Fix HdfsNormalFileHandler to handle temporary file path correctly
> ------------------------------------------------------------------------------
>
> Key: CAMEL-18776
> URL: https://issues.apache.org/jira/browse/CAMEL-18776
> Project: Camel
> Issue Type: Bug
> Components: camel-hdfs
> Reporter: Kengo Seki
> Priority: Minor
> Fix For: 3.18.5, 3.20.0
>
>
> Currently, HDFS consumer doesn't seem to work with normal files.
> Given the following file on HDFS,
> {code}
> $ hdfs dfs -ls /tmp/input
> Found 1 items
> -rw-r--r-- 3 sekikn supergroup 4 2022-11-30 20:40
> /tmp/input/foo.txt
> $ hdfs dfs -cat /tmp/input/foo.txt
> bar
> {code}
> I tried to run the following route so as to copy normal files from HDFS to
> local filesystem.
> {code}
> $ cat put.java
> import org.apache.camel.builder.RouteBuilder;
> public class put extends RouteBuilder {
> @Override
> public void configure() throws Exception {
> from("hdfs:192.168.2.2:9000/tmp/input/?repeatCount=1").log("${body}");
> }
> }
> {code}
> But I came across the following error.
> {code}
> $ camel run put.java
> ...
> 2022-11-30 20:46:25.151 WARN 2718863 --- [9000/tmp/input/]
> apache.camel.component.hdfs.HdfsConsumer : Failed polling endpoint:
> hdfs://192.168.2.2:9000/tmp/input/?repeatCount=1. Will try again at next
> poll. Caused by: [org.apache.camel.RuntimeCamelException -
> java.io.FileNotFoundException:
> /tmp/foo.txt.opened5427239805734101292.hdfs/foo.txt.opened (Not a directory)]
> org.apache.camel.RuntimeCamelException: java.io.FileNotFoundException:
> /tmp/foo.txt.opened5427239805734101292.hdfs/foo.txt.opened (Not a directory)
> at
> org.apache.camel.component.hdfs.HdfsNormalFileHandler.createInputStream(HdfsNormalFileHandler.java:98)
> ~[camel-hdfs-3.19.0.jar:3.19.0]
> at
> org.apache.camel.component.hdfs.HdfsNormalFileHandler.createInputStream(HdfsNormalFileHandler.java:38)
> ~[camel-hdfs-3.19.0.jar:3.19.0]
> at
> org.apache.camel.component.hdfs.HdfsFileType.createInputStream(HdfsFileType.java:46)
> ~[camel-hdfs-3.19.0.jar:3.19.0]
> at
> org.apache.camel.component.hdfs.HdfsInputStream.createInputStream(HdfsInputStream.java:71)
> ~[camel-hdfs-3.19.0.jar:3.19.0]
> at
> org.apache.camel.component.hdfs.HdfsConsumer.asHdfsFile(HdfsConsumer.java:246)
> ~[camel-hdfs-3.19.0.jar:3.19.0]
> ...
> Caused by: java.io.FileNotFoundException:
> /tmp/foo.txt.opened5427239805734101292.hdfs/foo.txt.opened (Not a directory)
> at java.io.FileInputStream.open0(Native Method) ~[?:?]
> at java.io.FileInputStream.open(FileInputStream.java:219) ~[?:?]
> at java.io.FileInputStream.<init>(FileInputStream.java:157) ~[?:?]
> at
> org.apache.camel.component.hdfs.HdfsNormalFileHandler.createInputStream(HdfsNormalFileHandler.java:94)
> ~[camel-hdfs-3.19.0.jar:3.19.0]
> ... 27 more
> {code}
> Temporary file was correctly created as follows, but HdfsNormalFileHandler
> seems to have mistaken its path. The correct one is
> "/tmp/foo.txt.opened5427239805734101292.hdfs", but there's unnecessary
> trailing string "/foo.txt.opened" in the log above.
> {code}
> $ ls /tmp/foo.txt*
> /tmp/foo.txt.opened5427239805734101292.hdfs
> $ cat /tmp/foo.txt.opened5427239805734101292.hdfs
> bar
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
