Hi Ron, Did you tell XCC to be HTTP compliant? It's not by default.
http://docs.marklogic.com/guide/xcc/concepts#id_28335 Enable HTTP 1.1 Compliance Enabling HTTP compliant mode guarantees the traffic between your XDBC App Server and your XCC client is compliant with the HTTP 1.1 protocol. This enables properly configured load balancers to detect the SessionID cookie generated by MarkLogic Server and use it to enforce session affinity. To enable this mode for a Java application, set the xcc.httpcompliant system property to true on the Java command line. For example: java -Dxcc.httpcompliant=true ... To enable this mode for a .NET application, set the xcc.httpcompliant environment variable to True. Setting xcc.httpcompliant to true is incompatible with enabling content entity resolution using ContentCreateOptions.setResolveEntities (Java) or the ResolveEntities property (.NET). If xcc.httpcompliant is not set explicitly, xcc.httpcompliant is false. You must also configure your load balancer to use the value in the SessionID cookie for session affinity. Some routers or load balancers may need to have xcc.httpcompliant enabled to allow any traffic through, regardless of session affinity issues. Configure the Load Balancer In addition to setting xcc.httpcompliant to true, you must configure your load balancer to use the SessionID cookie generated by MarkLogic Server for session affinity. You might also need to enable session affinity or sticky sessions in your load balancer. The exact configuration steps depend on the load balancer; see your load balancer documentation for details. Your load balancer must be HTTP 1.1 compliant and support cookie-based session affinity to use this feature of XCC. A SessionID cookie looks similar to the following: SessionID=25b877c32807aa9f -jh- > On Jul 1, 2016, at 10:10 PM, Ron Hitchens <[email protected]> wrote: > > > We’re having quite a bit of trouble with ELBs in AWS. There is an ELB in > front of the service/app nodes, and another ELB between them and the three > node MarkLogic cluster. > > We’ve had problems with the ELB in front of ML marking nodes down if they > are not responsive enough, which results in 504 responses on calls to ML > after that. We’ve had to crank up the timeouts to fairly high levels to > avoid one slow query from causing subsequent queries to fail. > > That’s annoying but manageable. What’s really problematic is apparent data > corruption when using XCC, specifically CORB. > > For most calls to MarkLogic, we’re using our bespoke REST interface using > URL rewriting on an HTTP appserver. We don’t use XCC normally. But we’re > doing some one time batch reprocessing with CORB. Pointing CORB at one of > the E-nodes works fine. But if we point it at the ELB (which works fine for > the REST calls) then we randomly get failures like the one below. > > Since I originally wrote XCC, I know that this error message is the result > of the data stream being out of sync with what XCC is expecting to see. It’s > looking for a sub-header in the partitioned response that describes the XDM > type of the next result item. It’s seeing something else instead. > > Are there known issues with ELBs and XCC/XDBC and/or CORB? Is there an ELB > setting get around this? I have a feeling that the ELB may be munging the > response in some way that is tripping up XCC. > > Any help appreciated. > > ======= > > INFO: monitoring 1,171,599 tasks > Jun 30, 2016 3:35:23 PM com.marklogic.developer.corb.Manager populateQueue > INFO: received first uri: > urn:cambridge.org:id:content-bundle:S0360966900011993:version:1 > 2016-06-30 15:35:23.454 SEVERE [1] (StreamingResultSequence.next): > IOException instantiating ResultItem 240: Nodes of type 'rsion' are not > supported in XCC result sequences > java.io.IOException: Nodes of type 'rsion' are not supported in XCC result > sequences > > com.marklogic.xcc.impl.AbstractResultSequence.nodeFactory(AbstractResultSequence.java:248) > > com.marklogic.xcc.impl.AbstractResultSequence.instantiateResultItem(AbstractResultSequence.java:202) > > com.marklogic.xcc.impl.StreamingResultSequence.next(StreamingResultSequence.java:147) > > com.marklogic.xcc.impl.StreamingResultSequence.next(StreamingResultSequence.java:166) > > com.marklogic.developer.corb.QueryUrisLoader.next(QueryUrisLoader.java:158) > com.marklogic.developer.corb.Manager.populateQueue(Manager.java:719) > com.marklogic.developer.corb.Manager.run(Manager.java:472) > com.marklogic.developer.corb.Manager.main(Manager.java:162) > Jun 30, 2016 3:35:23 PM com.marklogic.developer.corb.Manager stop > INFO: cleaning up > Jun 30, 2016 3:35:23 PM com.marklogic.developer.corb.QueryUrisLoader close > INFO: closing uris session > Jun 30, 2016 3:35:23 PM com.marklogic.developer.corb.Monitor run > SEVERE: interrupted: exiting > java.lang.InterruptedException > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088) > at > java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) > at > java.util.concurrent.ExecutorCompletionService.poll(ExecutorCompletionService.java:202) > at com.marklogic.developer.corb.Monitor.monitorResults(Monitor.java:96) > at com.marklogic.developer.corb.Monitor.run(Monitor.java:75) > at java.lang.Thread.run(Thread.java:745) > > 2016-06-30 15:35:23.475 WARNING [10] (AbstractRequestController.runRequest): > Cannot obtain connection: null > java.nio.channels.ClosedByInterruptException > > java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:659) > java.nio.channels.SocketChannel.open(SocketChannel.java:189) > > com.marklogic.xcc.impl.SocketPoolProvider.obtainConnection(SocketPoolProvider.java:93) > > com.marklogic.xcc.impl.handlers.AbstractRequestController.runRequest(AbstractRequestController.java:85) > > com.marklogic.xcc.impl.SessionImpl.submitRequestInternal(SessionImpl.java:437) > com.marklogic.xcc.impl.SessionImpl.submitRequest(SessionImpl.java:432) > > com.marklogic.developer.corb.AbstractTask.invokeModule(AbstractTask.java:233) > com.marklogic.developer.corb.AbstractTask.call(AbstractTask.java:148) > com.marklogic.developer.corb.AbstractTask.call(AbstractTask.java:56) > java.util.concurrent.FutureTask.run(FutureTask.java:266) > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > java.util.concurrent.FutureTask.run(FutureTask.java:266) > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > java.lang.Thread.run(Thread.java:745) > Jun 30, 2016 3:35:23 PM com.marklogic.developer.corb.AbstractTask > handleRequestException > WARNING: Encountered ServerConnectionException from Marklogic Server. > Retrying attempt 1 after 60 seconds..: null at URI: > urn:cambridge.org:id:content-bundle:S0360966900011993:version:1 > > --- > Ron Hitchens {[email protected]} +44 7879 358212 > > _______________________________________________ > General mailing list > [email protected] > Manage your subscription at: > http://developer.marklogic.com/mailman/listinfo/general _______________________________________________ General mailing list [email protected] Manage your subscription at: http://developer.marklogic.com/mailman/listinfo/general
