Hi Andy,
Thanks for chiming in. I've gathered some info down here:
The LogAttribute before InvokeHTTP:
2017-01-26 09:15:10,373 INFO [Timer-Driven Process Thread-4]
o.a.n.processors.standard.LogAttribute
LogAttribute[id=d9d45caf-0159-1000-d4a6-5127a
fdeaf64] logging for flow file
StandardFlowFileRecord[uuid=dffe5b52-b5f1-4597-8540-dc0baaff29ad,claim=StandardContentClaim
[resourceClaim=Standar
dResourceClaim[id=1485197201540-1, container=default, section=1], offset=37389,
length=10],offset=0,name=258752766597833,size=10]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Thu Jan 26 09:14:57 CET 2017'
Key: 'lineageStartDate'
Value: 'Thu Jan 26 09:14:57 CET 2017'
Key: 'fileSize'
Value: '10'
FlowFile Attribute Map Content
Key: 'filename'
Value: '258752766597833'
Key: 'path'
Value: './'
Key: 'uuid'
Value: 'dffe5b52-b5f1-4597-8540-dc0baaff29ad'
--------------------------------------------------
After:
2017-01-26 09:15:48,572 INFO [Timer-Driven Process Thread-6]
o.a.n.processors.standard.LogAttribute
LogAttribute[id=d9d49cf0-0159-1000-c5cd-c0810449e070] logging for flow file
StandardFlowFileRecord[uuid=dffe5b52-b5f1-4597-8540-dc0baaff29ad,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1485197201540-1, container=default,
section=1], offset=37389, length=10],offset=0,name=258752766597833,size=10]
--------------------------------------------------
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Thu Jan 26 09:14:57 CET 2017'
Key: 'lineageStartDate'
Value: 'Thu Jan 26 09:14:57 CET 2017'
Key: 'fileSize'
Value: '10'
FlowFile Attribute Map Content
Key: 'filename'
Value: '258752766597833'
Key: 'path'
Value: './'
Key: 'uuid'
Value: 'dffe5b52-b5f1-4597-8540-dc0baaff29ad'
--------------------------------------------------
The exception InvokeHTTP is raising (in this case a ConnectException, I can
make it a SocketTimeout with some extra work but I think new attributes should
be present in this case as well)
2017-01-26 09:15:10,975 INFO [StandardProcessScheduler Thread-1]
o.a.n.c.s.TimerDrivenSchedulingAgent Stopped scheduling LogAttribute[id=d9d45caf
-0159-1000-d4a6-5127afdeaf64] to run
2017-01-26 09:15:11,100 INFO [Flow Service Tasks Thread-2]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controll
er.FlowController@68275864 // Another save pending = true
2017-01-26 09:15:11,955 INFO [Flow Service Tasks Thread-2]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controll
er.FlowController@68275864 // Another save pending = false
2017-01-26 09:15:16,228 INFO [StandardProcessScheduler Thread-8]
o.a.n.c.s.TimerDrivenSchedulingAgent Scheduled InvokeHTTP[id=d9d48225-0159-1000-
9958-4bed80de9fa6] to run with 1 threads
2017-01-26 09:15:16,842 INFO [Flow Service Tasks Thread-2]
o.a.nifi.controller.StandardFlowService Saved flow controller
org.apache.nifi.controll
er.FlowController@68275864 // Another save pending = false
2017-01-26 09:15:17,551 ERROR [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.InvokeHTTP
InvokeHTTP[id=d9d48225-0159-1000-9958-4bed8
0de9fa6] Routing to Failure due to exception: java.net.ConnectException: Failed
to connect to /127.0.0.1:8000: java.net.ConnectException: Failed
to connect to /127.0.0.1:8000
2017-01-26 09:15:17,553 ERROR [Timer-Driven Process Thread-8]
o.a.nifi.processors.standard.InvokeHTTP
java.net.ConnectException: Failed to connect to /127.0.0.1:8000
at
com.squareup.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:139)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.internal.io.RealConnection.connect(RealConnection.java:108)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:283)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
~[okhttp-2.7.1.jar:na]
at com.squareup.okhttp.Call.getResponse(Call.java:286)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:243)
~[okhttp-2.7.1.jar:na]
at
com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:205)
~[okhttp-2.7.1.jar:na]
at com.squareup.okhttp.Call.execute(Call.java:80) ~[okhttp-2.7.1.jar:na]
at
org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:624)
~[nifi-standard-processors-1.0.0.jar:1.0.0]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
[nifi-api-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064)
[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0.jar:1.0.0]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
[nifi-framework-core-1.0.0.jar:1.0.0]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_77]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_77]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_77]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_77]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_77]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_77]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
The InvokeHTTP config is standard except:
HTTP Method: POST
URL 127.0.0.1:8000 (nothing is running there)
Read timeout: 5 secs
Content-type: text/plain
The test flow is very simple, the real one is more complicated.
[cid:[email protected]]
From: Andy LoPresto [mailto:[email protected]]
Sent: Wednesday, January 25, 2017 10:47 PM
To: [email protected]
Subject: Re: InvokeHTTP and SocketTimeoutException
Giovanni,
You are correct; the flowfile should have new attributes
(invokehttp.status.code, etc.) on failure. Can you put a LogAttribute processor
before and after the InvokeHTTP processor and print all of the attributes of
the flowfile, then copy them here? If you can also share the configuration of
your InvokeHTTP processor, that will help us diagnose the issue. Thanks.
Andy LoPresto
[email protected]<mailto:[email protected]>
[email protected]<mailto:[email protected]>
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
On Jan 25, 2017, at 7:44 AM, Giovanni Lanzani
<[email protected]<mailto:[email protected]>>
wrote:
Hi all,
I'm using InvokeHTTP to do a POST request to a server. I'm trying to simulate
the server accepting connection but NOT giving response within the limit I set
in NiFi.
The processor fails with SocketTimeoutException and the FlowFile is rerouted to
the Failure relationship (up until this: all is good!).
The issue is that the exception was not written in the attributes of the
FlowFile. No attributes are modified when the failure is occurring. By reading
the docs for the failure relationship, I was under the impression that:
"The original FlowFile [...] will have new attributes detailing the request".
But I think that maybe that is not correct?! (I'm using NiFi 1.0.0)
Thanks in advance for everybody that might chime in! Sorry in advance if this
was a super dump question like my last one.
Cheers,
Giovanni
