Hi James,
Thanks for the good questions and areas to look at!
We are using Nifi version 0.7.0
No. We have not configured the proxy host, port, or endpoint override url.
These are small json objects and should not use Multi-part upload. All of
the json object have the same fields and should be around the same size.
I was able to send a file to the S3 bucket using aws cli on all three
instances of the Nifi cluster, but have not pin pointed with one is
producing the error for Nifi.
>From the Nifi-app.log file I have the following error message:
2016-09-09 16:01:46,292 ERROR [Timer-Driven Process Thread-4]
o.a.nifi.processors.aws.s3.PutS3Object
com.amazonaws.AmazonClientException: Unable to execute HTTP request:
Connection timed out
at
com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:516)
~[aws-java-sdk-core-1.10.32.jar:na]
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317)
~[aws-java-sdk-core-1.10.32.jar:na]
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3595)
~[aws-java-sdk-s3-1.10.32.jar:na]
at
com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1382)
~[aws-java-sdk-s3-1.10.32.jar:na]
at
org.apache.nifi.processors.aws.s3.PutS3Object$1.process(PutS3Object.java:446)
~[nifi-aws-processors-0.7.0.jar:0.7.0]
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
~[na:na]
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
~[na:na]
at
org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:400)
~[nifi-aws-processors-0.7.0.jar:0.7.0]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
[nifi-api-0.7.0.jar:0.7.0]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
[nifi-framework-core-0.7.0.jar:0.7.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-0.7.0.jar:0.7.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-0.7.0.jar:0.7.0]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
[nifi-framework-core-0.7.0.jar:0.7.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_101]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_101]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_101]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
~[na:1.8.0_101]
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[na:1.8.0_101]
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[na:1.8.0_101]
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[na:1.8.0_101]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
~[na:1.8.0_101]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_101]
I also get warning errors like the one below :
2016-09-09 16:02:00,192 WARN [Framework Task Thread
Thread-4-SendThread(ip-10-204-53-236.dqa.capitalone.com:2181)]
org.apache.zookeeper.ClientCnxn Session 0x0 for server null, unexpected
error, closing\
socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.8.0_101]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
~[na:1.8.0_101]
at
org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
~[na:na]
at
org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
~[na:na]
Thanks!
--
View this message in context:
http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Debugging-AWS-PutS3-processor-tp13311p13314.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
