[jira] [Updated] (NIFI-7997) ListFTP does not use HTTP proxy
[
https://issues.apache.org/jira/browse/NIFI-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Richard Zuidhof updated NIFI-7997:
--
Description:
After configuring a HTTP type proxy server in the ListFTP processor Nifi still
tries to connect directly to the FTP server instead of the proxy server for the
initial connection. Subsequent commands like DIR or GET are tunneled through
the HTTP proxy but without a direct connection to the FTP server this will
never succeeded. There is no issue using a SOCKS proxy.
You can reproduce this very easily, even without a real proxy. Just enter a non
existing ftp host and set proxy type to HTTP and use a fake proxy host and
port. When using a non-existing ftp hostname you will get this error. This
shows Nifi wants to connect directly instead of via the proxy:
{{2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.ListFTP
ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on
remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go}}
When using an unreachable IP (like 10.123.1.1) you will see this:
{{2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.ListFTP
ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on
remote host due to Connection timed out (Connection timed out):
java.net.ConnectException: Connection timed out (Connection timed out)
java.net.ConnectException: Connection timed out (Connection timed out) at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
java.net.Socket.connect(Socket.java:607) at
org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at
org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at
org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600)
at
org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233)
at
org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196)
at
org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106)
at
org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472)
at
org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)}}
was:
After configuring a HTTP type proxy server in the ListFTP processor Nifi still
tries to connect directly to the FTP server instead of the proxy server for the
initial connection. Subsequent commands like DIR or GET are tunneled through
the HTTP proxy but without a direct connection to the FTP server this will
never succeeded. There is no issue using a SOCKS proxy.
You can reproduce this very easily, even without a real proxy. Just enter a non
existing ftp host and set proxy type to HTTP and use a fake proxy host and
port. When using a non-existing ftp hostname you will get this error. This
shows Nifi wants to connect directly instead of via the proxy:
2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.ListFTP
ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on
remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go
When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11
12:43:10,862 ERROR [Timer-Driven Process Thread-10]
o.a.nifi.processors.standard.ListFTP
ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on
remote
[jira] [Updated] (NIFI-7997) ListFTP does not use HTTP proxy
[ https://issues.apache.org/jira/browse/NIFI-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Zuidhof updated NIFI-7997: -- Description: After configuring a HTTP type proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy. You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy: 2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on remote host due to Connection timed out (Connection timed out): java.net.ConnectException: Connection timed out (Connection timed out) java.net.ConnectException: Connection timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) was: After configuring a proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy. You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy: 2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on remote host due to
[jira] [Created] (NIFI-7997) ListFTP does not use HTTP proxy
Richard Zuidhof created NIFI-7997: - Summary: ListFTP does not use HTTP proxy Key: NIFI-7997 URL: https://issues.apache.org/jira/browse/NIFI-7997 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.11.4 Reporter: Richard Zuidhof After configuring a proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy. You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy: 2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000--e04e112b] Failed to perform listing on remote host due to Connection timed out (Connection timed out): java.net.ConnectException: Connection timed out (Connection timed out) java.net.ConnectException: Connection timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005)
