Hi Cihad, There are NTLMEngineImpl tests that exercise precisely the case that is failing. I'm therefore becoming convinced that there is something very odd about your installation. Are you using a non-standard JVM, for instance?
Karl On Fri, Mar 17, 2017 at 10:28 AM, Karl Wright <[email protected]> wrote: > Hi Cihad, > > Could you also check out and build the latest 4.5.x httpclient, from this > branch? > > https://svn.apache.org/repos/asf/httpcomponents/httpclient/ > branches/pull-66 > > You will need maven for this but otherwise you can build it any way you > like. Replace the "httpclient-4.5.1.jar" in the lib directory with the jar > you build, and then you can rebuild MCF. See if you still get the error. > If you do, it should be possible to chase it down more readily. > > Thanks, > Karl > > > On Fri, Mar 17, 2017 at 9:57 AM, Cihad Guzel <[email protected]> wrote: > >> No. I don't use any custom library. >> >> I try with manifoldcf trunk on my notebook. I install sharepoint 2013 on >> ms server 2012 for testing with default configuration. >> >> 17 Mar 2017 16:05 tarihinde "Karl Wright" <[email protected]> yazdı: >> >> Hmm, I can see no way this can happen. Are you by any chance using a >>> modified version of the HttpClient library? >>> Karl >>> >>> >>> On Fri, Mar 17, 2017 at 8:09 AM, Karl Wright <[email protected]> wrote: >>> >>>> Hi Cihad, >>>> >>>> This is very interesting because the problem is coming from >>>> Httpclient's NTLM engine. The allocated packet size for the Type 1 message >>>> is being exceeded, which I didn't think was even possible. >>>> >>>> This may be a result of credentials that you have supplied being >>>> strange in some way. Let me look at the Httpclient code and get back to >>>> you. >>>> >>>> Karl >>>> >>>> >>>> On Fri, Mar 17, 2017 at 7:57 AM, Cihad Guzel <[email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> I try sharepoint connector with Active Directory in debug mode. I saw >>>>> ArrayIndexOutOfBoundException in manifoldcf.log file. Any bugs? >>>>> >>>>> DEBUG 2017-03-17 14:30:48,386 (Worker thread '0') - SharePoint: >>>>> Getting version of '/Documents2//Step by step Installation of SharePoint >>>>> 2013 on Windows Server 2012 R2 part 1 - SharePoint Community.pdf' >>>>> DEBUG 2017-03-17 14:30:48,466 (Worker thread '0') - SharePoint: >>>>> Checking whether to include document '/Documents2/Step by step >>>>> Installation >>>>> of SharePoint 2013 on Windows Server 2012 R2 part 1 - SharePoint >>>>> Community.pdf' >>>>> DEBUG 2017-03-17 14:30:48,466 (Worker thread '0') - SharePoint: File >>>>> '/Documents2/Step by step Installation of SharePoint 2013 on Windows >>>>> Server >>>>> 2012 R2 part 1 - SharePoint Community.pdf' exactly matched rule path >>>>> '/Documents2/*' >>>>> DEBUG 2017-03-17 14:30:48,467 (Worker thread '0') - SharePoint: >>>>> Including file '/Documents2/Step by step Installation of SharePoint 2013 >>>>> on >>>>> Windows Server 2012 R2 part 1 - SharePoint Community.pdf' >>>>> DEBUG 2017-03-17 14:30:48,468 (Worker thread '0') - SharePoint: >>>>> Finding metadata to include for document/item '/Documents2/Step by step >>>>> Installation of SharePoint 2013 on Windows Server 2012 R2 part 1 - >>>>> SharePoint Community.pdf'. >>>>> DEBUG 2017-03-17 14:30:48,510 (Worker thread '0') - SharePoint: In >>>>> getFieldValues; fieldNames=[Ljava.lang.String;@69f1a61a, site='', >>>>> docLibrary='{1B694C45-DF1F-44E7-9814-F5096E85A126}', >>>>> docId='/Documents2/Step by step Installation of SharePoint 2013 on Windows >>>>> Server 2012 R2 part 1 - SharePoint Community.pdf', dspStsWorks=false >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: >>>>> Getting version of '/Documents2//' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: >>>>> Getting version of '/Documents2//CXFCA3100080010.pdf' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: >>>>> Checking whether to include document '/Documents2/CXFCA3100080010.pdf' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: File >>>>> '/Documents2/CXFCA3100080010.pdf' exactly matched rule path >>>>> '/Documents2/*' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: >>>>> Including file '/Documents2/CXFCA3100080010.pdf' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: >>>>> Checking whether to include library '/Documents2' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: >>>>> Finding metadata to include for document/item >>>>> '/Documents2/CXFCA3100080010.p >>>>> df'. >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: >>>>> Library '/Documents2' partially matched file rule path '/Documents2/*' - >>>>> including >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: >>>>> Document identifier is a library: '/Documents2' >>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: In >>>>> getDocLibID; parentSite='', parentSiteDecoded='', docLibrary='Documents2' >>>>> DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - SharePoint: >>>>> Getting version of '/' >>>>> DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - SharePoint: >>>>> Checking whether to include site '/' >>>>> DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - SharePoint: Site >>>>> '/' partially matched file rule path '/Documents2/*' - including >>>>> DEBUG 2017-03-17 14:30:48,548 (Worker thread '4') - SharePoint: In >>>>> getFieldValues; fieldNames=[Ljava.lang.String;@6f447d2e, site='', >>>>> docLibrary='{1B694C45-DF1F-44E7-9814-F5096E85A126}', >>>>> docId='/Documents2/CXFCA3100080010.pdf', dspStsWorks=false >>>>> DEBUG 2017-03-17 14:30:48,560 (Worker thread '2') - SharePoint: >>>>> Document identifier is a site: '' >>>>> DEBUG 2017-03-17 14:30:48,560 (Worker thread '2') - SharePoint: In >>>>> getSites; parentSite='' >>>>> DEBUG 2017-03-17 14:30:50,398 (Worker thread '4') - SharePoint: Got a >>>>> remote exception getting field values for site library >>>>> {1B694C45-DF1F-44E7-9814-F5096E85A126} document >>>>> [/Documents2/CXFCA3100080010.pdf] - retrying >>>>> AxisFault >>>>> faultCode: {http://schemas.xmlsoap.org/so >>>>> ap/envelope/}Server.userException >>>>> faultSubcode: >>>>> faultString: java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> faultActor: >>>>> faultNode: >>>>> faultDetail: >>>>> {http://xml.apache.org/axis/}stackTrace:java.lang.ArrayIndex >>>>> OutOfBoundsException: 41 >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte >>>>> (NTLMEngineImpl.java:911) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addULon >>>>> g(NTLMEngineImpl.java:941) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes >>>>> ponse(NTLMEngineImpl.java:1043) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL >>>>> MEngineImpl.java:148) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT >>>>> LMEngineImpl.java:1628) >>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme >>>>> .java:139) >>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc >>>>> hemeBase.java:138) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe >>>>> nticator.java:239) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp >>>>> onse(HttpAuthenticator.java:202) >>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl >>>>> ientExec.java:262) >>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol >>>>> Exec.java:184) >>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect >>>>> Exec.java:110) >>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int >>>>> ernalHttpClient.java:184) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:82) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:107) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:55) >>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend >>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) >>>>> >>>>> {http://xml.apache.org/axis/}hostname:cg-DELL >>>>> >>>>> java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> at org.apache.axis.AxisFault.makeFault(AxisFault.java:101) >>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend >>>>> er.invoke(CommonsHTTPSender.java:263) >>>>> at org.apache.axis.strategies.InvocationStrategy.visit(Invocati >>>>> onStrategy.java:32) >>>>> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118) >>>>> at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83) >>>>> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165) >>>>> at org.apache.axis.client.Call.invokeEngine(Call.java:2784) >>>>> at org.apache.axis.client.Call.invoke(Call.java:2767) >>>>> at org.apache.axis.client.Call.invoke(Call.java:2443) >>>>> at org.apache.axis.client.Call.invoke(Call.java:2366) >>>>> at org.apache.axis.client.Call.invoke(Call.java:1812) >>>>> at com.microsoft.schemas.sharepoint.soap.ListsSoapStub.getListI >>>>> tems(ListsSoapStub.java:1841) >>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SPSProxy >>>>> Helper.getFieldValues(SPSProxyHelper.java:2134) >>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SharePoi >>>>> ntRepository.processDocuments(SharePointRepository.java:1433) >>>>> at org.apache.manifoldcf.crawler.system.WorkerThread.run(Worker >>>>> Thread.java:399) >>>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte >>>>> (NTLMEngineImpl.java:911) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addULon >>>>> g(NTLMEngineImpl.java:941) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes >>>>> ponse(NTLMEngineImpl.java:1043) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL >>>>> MEngineImpl.java:148) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT >>>>> LMEngineImpl.java:1628) >>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme >>>>> .java:139) >>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc >>>>> hemeBase.java:138) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe >>>>> nticator.java:239) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp >>>>> onse(HttpAuthenticator.java:202) >>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl >>>>> ientExec.java:262) >>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol >>>>> Exec.java:184) >>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect >>>>> Exec.java:110) >>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int >>>>> ernalHttpClient.java:184) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:82) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:107) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:55) >>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend >>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) >>>>> DEBUG 2017-03-17 14:30:50,398 (Worker thread '0') - SharePoint: Got a >>>>> remote exception getting field values for site library >>>>> {1B694C45-DF1F-44E7-9814-F5096E85A126} document [/Documents2/Step by >>>>> step Installation of SharePoint 2013 on Windows Server 2012 R2 part 1 - >>>>> SharePoint Community.pdf] - retrying >>>>> AxisFault >>>>> faultCode: {http://schemas.xmlsoap.org/so >>>>> ap/envelope/}Server.userException >>>>> faultSubcode: >>>>> faultString: java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> faultActor: >>>>> faultNode: >>>>> faultDetail: >>>>> {http://xml.apache.org/axis/}stackTrace:java.lang.ArrayIndex >>>>> OutOfBoundsException: 41 >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte >>>>> (NTLMEngineImpl.java:911) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUSho >>>>> rt(NTLMEngineImpl.java:933) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes >>>>> ponse(NTLMEngineImpl.java:1040) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL >>>>> MEngineImpl.java:148) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT >>>>> LMEngineImpl.java:1628) >>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme >>>>> .java:139) >>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc >>>>> hemeBase.java:138) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe >>>>> nticator.java:239) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp >>>>> onse(HttpAuthenticator.java:202) >>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl >>>>> ientExec.java:262) >>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol >>>>> Exec.java:184) >>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect >>>>> Exec.java:110) >>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int >>>>> ernalHttpClient.java:184) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:82) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:107) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:55) >>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend >>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) >>>>> >>>>> {http://xml.apache.org/axis/}hostname:cg-DELL >>>>> >>>>> java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> at org.apache.axis.AxisFault.makeFault(AxisFault.java:101) >>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend >>>>> er.invoke(CommonsHTTPSender.java:263) >>>>> at org.apache.axis.strategies.InvocationStrategy.visit(Invocati >>>>> onStrategy.java:32) >>>>> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118) >>>>> at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83) >>>>> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165) >>>>> at org.apache.axis.client.Call.invokeEngine(Call.java:2784) >>>>> at org.apache.axis.client.Call.invoke(Call.java:2767) >>>>> at org.apache.axis.client.Call.invoke(Call.java:2443) >>>>> at org.apache.axis.client.Call.invoke(Call.java:2366) >>>>> at org.apache.axis.client.Call.invoke(Call.java:1812) >>>>> at com.microsoft.schemas.sharepoint.soap.ListsSoapStub.getListI >>>>> tems(ListsSoapStub.java:1841) >>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SPSProxy >>>>> Helper.getFieldValues(SPSProxyHelper.java:2134) >>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SharePoi >>>>> ntRepository.processDocuments(SharePointRepository.java:1433) >>>>> at org.apache.manifoldcf.crawler.system.WorkerThread.run(Worker >>>>> Thread.java:399) >>>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte >>>>> (NTLMEngineImpl.java:911) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUSho >>>>> rt(NTLMEngineImpl.java:933) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes >>>>> ponse(NTLMEngineImpl.java:1040) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL >>>>> MEngineImpl.java:148) >>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT >>>>> LMEngineImpl.java:1628) >>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme >>>>> .java:139) >>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc >>>>> hemeBase.java:138) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe >>>>> nticator.java:239) >>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp >>>>> onse(HttpAuthenticator.java:202) >>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl >>>>> ientExec.java:262) >>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol >>>>> Exec.java:184) >>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect >>>>> Exec.java:110) >>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int >>>>> ernalHttpClient.java:184) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:82) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:107) >>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos >>>>> eableHttpClient.java:55) >>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend >>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654) >>>>> WARN 2017-03-17 14:30:50,404 (Worker thread '4') - Service >>>>> interruption reported for job 1489681799610 connection 'sp2013 crawl': >>>>> Remote procedure exception: ; nested exception is: >>>>> java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> WARN 2017-03-17 14:30:50,405 (Worker thread '0') - Service >>>>> interruption reported for job 1489681799610 connection 'sp2013 crawl': >>>>> Remote procedure exception: ; nested exception is: >>>>> java.lang.ArrayIndexOutOfBoundsException: 41 >>>>> DEBUG 2017-03-17 14:31:09,041 (Worker thread '2') - SharePoint: >>>>> getSites xml response: <ns1:Webs xmlns:ns1="http://schemas.micr >>>>> osoft.com/sharepoint/soap/"/> >>>>> DEBUG 2017-03-17 14:31:09,043 (Worker thread '2') - SharePoint: In >>>>> getDocumentLibraries; parentSite='', parentSiteDecoded='' >>>>> >>>>> -- >>>>> Teşekkürler >>>>> Cihad Güzel >>>>> >>>> >>>> >>> >
