[jira] [Comment Edited] (CAMEL-10238) Camel-salesforce component never tries to reconnect after a disconnect
[
https://issues.apache.org/jira/browse/CAMEL-10238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455466#comment-15455466
]
Rajesh A edited comment on CAMEL-10238 at 9/1/16 3:01 PM:
--
Missed to attach another log, this one is the backed up one due to log overload.
was (Author: rajesh734):
Missed to attach the another log, this one is the backed up one due log
overload.
> Camel-salesforce component never tries to reconnect after a disconnect
> --
>
> Key: CAMEL-10238
> URL: https://issues.apache.org/jira/browse/CAMEL-10238
> Project: Camel
> Issue Type: Bug
> Components: camel-salesforce
>Affects Versions: 2.17.2, 2.17.3, 2.18.0
>Reporter: Rajesh A
>Assignee: Dhiraj Bokde
> Attachments: streaming-api-validation-jetty-debug.log,
> streaming-api-validation-jetty-debug.log,
> streaming-api-validation-jetty-debug.log,
> streaming-api-validation-jetty-debug.log.zip
>
>
> My connection to salesforce-streaming api gets disconnect automatically after
> 2 hours. This is because salesforce automatically disconnects the connection
> from server side. However, I was expecting camel-salesforce component to
> reconnect automatically after disconnect. But, it does not reconnect and I do
> not have a hold or a way to reconnect. Seems to be a defect and a blocker to
> me.
> Here is the trace log
> {code}
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=84, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> connectionType=long-polling}]
> [36mo.a.c.c.s.i.s.SubscriptionHelper$3 [0;39m [2m:[0;39m Received
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> successful=true}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Processing
> /meta/connect {clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect,
> id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> CONNECTED
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=86,
> connectionType=long-polling}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> UNCONNECTED
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Messages failed
> [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=86,
> connectionType=long-polling}]
> java.io.EOFException: HttpConnectionOverHTTP@12f0e719(l:/10.172.131.200:50574
> <->
> r:my-proxy.com/x.x.x.x:xx,closed=false)[HttpChannelOverHTTP@2a4927(exchange=HttpExchange@6ae1ae35
> req=TERMINATED/null@null
> res=PENDING/null@null)[send=HttpSenderOverHTTP@51fea1e0(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator{s=START}],recv=HttpReceiverOverHTTP@44234ce9(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0
> of -1}]]]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:277)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1309)
> [jetty-http-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:182)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:129)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:89)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:122)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
>
[jira] [Comment Edited] (CAMEL-10238) Camel-salesforce component never tries to reconnect after a disconnect
[
https://issues.apache.org/jira/browse/CAMEL-10238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15445664#comment-15445664
]
Rajesh A edited comment on CAMEL-10238 at 8/29/16 12:05 PM:
Dhiraj, still getting the same issue. I have attached the log here. This is for
version 2.18.0-SNAPSHOT
was (Author: rajesh734):
Dhiraj, still getting the same issue. I have attached the log here.
> Camel-salesforce component never tries to reconnect after a disconnect
> --
>
> Key: CAMEL-10238
> URL: https://issues.apache.org/jira/browse/CAMEL-10238
> Project: Camel
> Issue Type: Bug
> Components: camel-salesforce
>Affects Versions: 2.17.2, 2.17.3, 2.18.0
>Reporter: Rajesh A
>Assignee: Dhiraj Bokde
> Attachments: streaming-api-validation-jetty-debug.log
>
>
> My connection to salesforce-streaming api gets disconnect automatically after
> 2 hours. This is because salesforce automatically disconnects the connection
> from server side. However, I was expecting camel-salesforce component to
> reconnect automatically after disconnect. But, it does not reconnect and I do
> not have a hold or a way to reconnect. Seems to be a defect and a blocker to
> me.
> Here is the trace log
> {code}
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=84, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> connectionType=long-polling}]
> [36mo.a.c.c.s.i.s.SubscriptionHelper$3 [0;39m [2m:[0;39m Received
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> successful=true}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Processing
> /meta/connect {clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect,
> id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> CONNECTED
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=86,
> connectionType=long-polling}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> UNCONNECTED
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Messages failed
> [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=86,
> connectionType=long-polling}]
> java.io.EOFException: HttpConnectionOverHTTP@12f0e719(l:/10.172.131.200:50574
> <->
> r:my-proxy.com/x.x.x.x:xx,closed=false)[HttpChannelOverHTTP@2a4927(exchange=HttpExchange@6ae1ae35
> req=TERMINATED/null@null
> res=PENDING/null@null)[send=HttpSenderOverHTTP@51fea1e0(req=QUEUED,snd=COMPLETED,failure=null)[HttpGenerator{s=START}],recv=HttpReceiverOverHTTP@44234ce9(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0
> of -1}]]]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:277)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1309)
> [jetty-http-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:182)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:129)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:89)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:122)
> [jetty-client-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
> [jetty-io-9.2.14.v20151106.jar:9.2.14.v20151106]
> at
>
[jira] [Comment Edited] (CAMEL-10238) Camel-salesforce component never tries to reconnect after a disconnect
[
https://issues.apache.org/jira/browse/CAMEL-10238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15436518#comment-15436518
]
Dhiraj Bokde edited comment on CAMEL-10238 at 8/25/16 8:51 AM:
---
Rajesh, I have pushed the fixes to master branch. Please test in your
environment and let me know whether it fixes your issue.
Also, note the following changes in behavior:
* Since subscriptions are created and handled asynchronously, the consumer
throws {{SalesforceException}} when it is unable to subscribe for any reason.
The error message will be {{Error subscribing to : }}. The
consumer won't attempt to re-subscribe and will not receive any messages from
Salesforce after this error. The underlying cause has to be fixed and the
consumer/route restarted to attempt to subscribe to the channel again.
* For hard disconnects, it will try to reconnect with a pause between attempts
based on backoff options in CometD, which are 1 second and 30 seconds for
increment and max respectively. If needed CometD options could be made
configurable in the future to make the backoff customizable.
* If it can't reconnect after the pause between attempts exceeds max backoff,
all consumers will throw a SalesforceException with a message {{Aborting
Salesforce reconnect due to: }}. At which point, all consumers are
again disconnected and the component will have to be restarted to try to
reconnect to Salesforce.
Hopefully this makes the consumer more robust and the error handling intuitive.
Let me know if you have any feedback or other ideas.
was (Author: dhirajsb):
Rajesh, I have pushed the fixes to master branch. Please test in your
environment and let me know whether it fixes your issue.
Also, note the following changes in behavior:
* Since subscriptions are created and handled asynchronously, the consumer
throws {{SalesforceException}} when it is unable to subscribe for any reason.
The error message will be {{Error subscribing to : }}. The
consumer won't attempt to re-subscribe and will not receive any messages from
Salesforce after this error. The underlying cause has to be fixed and the
consumer/route restarted to attempt to subscribe to the channel again.
* For hard disconnects, it will try to reconnect with a pause between attempts
based on backoff options in CometD, which are 1 second and 30 seconds for
increment and max respectively. If needed CometD options could be made
configurable in the future to make the backoff customizable.
* If it can't reconnect after the pause between attempts exceeds max backoff,
all consumers will throw a SalesforceException with a message {{Aborting
Salesforce reconnect due to: }}. At which point, all consumers are
again disconnected and consumers/routes will have to be restarted to try to
reconnect to Salesforce.
Hopefully this makes the consumer more robust and the error handling intuitive.
Let me know if you have any feedback or other ideas.
> Camel-salesforce component never tries to reconnect after a disconnect
> --
>
> Key: CAMEL-10238
> URL: https://issues.apache.org/jira/browse/CAMEL-10238
> Project: Camel
> Issue Type: Bug
> Components: camel-salesforce
>Affects Versions: 2.17.2, 2.17.3, 2.18.0
>Reporter: Rajesh A
>Assignee: Dhiraj Bokde
>
> My connection to salesforce-streaming api gets disconnect automatically after
> 2 hours. This is because salesforce automatically disconnects the connection
> from server side. However, I was expecting camel-salesforce component to
> reconnect automatically after disconnect. But, it does not reconnect and I do
> not have a hold or a way to reconnect. Seems to be a defect and a blocker to
> me.
> Here is the trace log
> {code}
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=84, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> connectionType=long-polling}]
> [36mo.a.c.c.s.i.s.SubscriptionHelper$3 [0;39m [2m:[0;39m Received
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> successful=true}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Processing
> /meta/connect {clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect,
> id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> CONNECTED
> [36mo.a.c.c.s.i.s.SubscriptionHelper
[jira] [Comment Edited] (CAMEL-10238) Camel-salesforce component never tries to reconnect after a disconnect
[
https://issues.apache.org/jira/browse/CAMEL-10238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15428600#comment-15428600
]
Dhiraj Bokde edited comment on CAMEL-10238 at 8/19/16 6:33 PM:
---
Hi Rajesh,
Please attach logs as files next time. From your log it looks like CometD tries
to reconnect after the 2 hour expiry by sending new handshake messages (I
tested this and it worked in my local tests), but in your test OAuth token has
also expired and Salesforce responds with the following message but HTTP status
200:
{code}
{ext={replay=true}, advice={reconnect=none}, channel=/meta/handshake, id=104,
error=401::Authentication invalid, successful=false}
{code}
The 401 is buried in the body, so {{SalesforceSecurityHandler}} in 2.18.x
doesn't see it, and {{reconnect=none}} is causing CometD to terminate the
connection. I'll also check what's going on in 2.17.x.
I will create a test for this scenario and provide a way to handle it. BTW, is
this consumer the only route in your application? I'm wondering why this wasn't
seen earlier by other users.
was (Author: dhirajsb):
Hi Rajesh,
Please attach logs as files next time. From your log it looks like CometD tries
to reconnect after the 2 hour expiry by sending new handshake messages (I
tested this and it worked in my local tests), but in your test OAuth token has
also expired and Salesforce responds with the following message but HTTP status
200:
{code}
{ext={replay=true}, advice={reconnect=none}, channel=/meta/handshake, id=104,
error=401::Authentication invalid, successful=false}
{code}
The 401 is buried in the body, so {{SalesforceSecurityHandler}} in 2.18.x
doesn't see it, and {{reconnect=none}} is causing CometD to terminate the
connection. I'll also check what's going on in 2.1.7.x.
I will create a test for this scenario and provide a way to handle it. BTW, is
this consumer the only route in your application? I'm wondering why this wasn't
seen earlier by other users.
> Camel-salesforce component never tries to reconnect after a disconnect
> --
>
> Key: CAMEL-10238
> URL: https://issues.apache.org/jira/browse/CAMEL-10238
> Project: Camel
> Issue Type: Bug
> Components: camel-salesforce
>Affects Versions: 2.17.2, 2.17.3, 2.18.0
>Reporter: Rajesh A
>Assignee: Dhiraj Bokde
>
> My connection to salesforce-streaming api gets disconnect automatically after
> 2 hours. This is because salesforce automatically disconnects the connection
> from server side. However, I was expecting camel-salesforce component to
> reconnect automatically after disconnect. But, it does not reconnect and I do
> not have a hold or a way to reconnect. Seems to be a defect and a blocker to
> me.
> Here is the trace log
> {code}
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=84, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> connectionType=long-polling}]
> [36mo.a.c.c.s.i.s.SubscriptionHelper$3 [0;39m [2m:[0;39m Received
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=85,
> successful=true}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Processing
> /meta/connect {clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect,
> id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> CONNECTED
> [36mo.a.c.c.s.i.s.SubscriptionHelper[0;39m [2m:[0;39m
> [CHANNEL:META_CONNECT]: {clientId=3u9riwg6ag3r5dd3ay86i444f,
> channel=/meta/connect, id=85, successful=true}
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Connecting,
> transport
> org.apache.camel.component.salesforce.internal.streaming.SubscriptionHelper$3@4e0cc334
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Sending
> messages [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=86,
> connectionType=long-polling}]
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m State update:
> CONNECTED -> UNCONNECTED
> [36morg.cometd.client.BayeuxClient [0;39m [2m:[0;39m Messages failed
> [{clientId=3u9riwg6ag3r5dd3ay86i444f, channel=/meta/connect, id=86,
> connectionType=long-polling}]
> java.io.EOFException: HttpConnectionOverHTTP@12f0e719(l:/10.172.131.200:50574
> <->
> r:my-proxy.com/x.x.x.x:xx,closed=false)[HttpChannelOverHTTP@2a4927(exchange=HttpExchange@6ae1ae35
