Hi Karl, Thanks so much for your prompt and detailed response.
On the first issue, we do not actually need the support for Basic auth; we were just experimenting to get any of the auth modes to work with the ManifoldCF's SharePoint plugin. As of right now, NTLM doesn't work either (I included the HTTP client debug info + the stack trace, those were generated with the auth provider set to NTLM). On the second issue, here's the breakdown of the versions: 1. The server version is: SP 2010, the Microsoft.SharePoint.dll version is: 14.0.4762.1000. Configuration database version: 14.0.4762.1000. Which I believe is known as SharePoint 2010, RTM / Farm Build Version. 2. The repository configuration is using "4.0" in the version pulldown. 3. The SharePoint plugin is SP 2010. It's from the sharepoint-2010 directory, with the 12 kb size of the MC wsp file (rather than 11 kb in the SP 2010 subdirectory). 4. The build of ManifoldCF is 1.3. 5. The system was built from sources using the following commands: ant make-core-deps, then ant build. 6. The system was started by: cd dist/example; java -jar start.jar. Please also note that we've tried the Derby-based persistence first and then the Postgres-based persistence. The results appear to be the same although the symptoms seem to fluctuate between the 401 error and the service not found error. I believe this is most likely due to difference in gestures in the UI; re-saving the repository config seems to generate different errors than firing off a crawl. Given all this, do you have any ideas on what might be causing the failures and/or any additional ways of troubleshooting? I can generate fuller logs for specific UI gestures, if need be. - Dmitry On Mon, Sep 2, 2013 at 5:36 PM, Karl Wright <[email protected]> wrote: > Hi Dimitry, > > The SharePoint connector does not support basic auth at this time. It's > so rare that heretofore we've never encountered it. NTLM is what everyone > seems to use. However, it is straightforward to add this logic. If you > want it, please create a Jira ticket for it. > > As for the "ListsSoap not found" error, there may be a mismatch between > the version of SharePoint you have and the MCPermissions service version > you installed. There is a SharePoint 2007 version, which only is > appropriate for SharePoint 2007, and when you select 3.0 (2007) in the > version pullldown. There is also a 2010 plugin, which is different, which > is used when you select 4.0 (2010) in the version pulldown. > > Thanks, > Karl > > > > On Mon, Sep 2, 2013 at 5:18 PM, Dmitry Goldenberg > <[email protected]>wrote: > >> I get the attached stack trace on a GetListCollection request as below. >> >> I have configured a 4.0 SP repository connection. The username is >> <domain>\<user> and it's got a valid password; I am able to login to the SP >> instance with those (admin) credentials. >> >> I've verified that I have the MCPermissionsService installed. >> >> I noticed that one time I was able to seemingly go past this error after >> switching from Basic auth to NTLM. However, then I started getting a >> different error -- No service named ListsSoap is available. >> >> If I uninstall the MC service and try to use the 2.0 or 3.0 type of >> connection, I'm getting errors also. >> >> Any help or insights or troubleshooting tips would be appreciated. The >> http client DEBUG dump is also included below as the second part of the >> attached info. Thanks. >> >> >> ============================================================================ >> >> >> faultString: The request failed with HTTP status 401: >> Unauthorized. >> faultActor: >> http://ec2-23-20-55-72.compute-1.amazonaws.com/_vti_bin/MCPermissions.asmx >> faultNode: >> faultDetail: >> {}Error:<ErrorNumber>1000</ErrorNumber><ErrorMessage>The request failed >> with HTTP status 401: >> Unauthorized.</ErrorMessage><ErrorSource>System.Web.Services</ErrorSource> >> The request failed with HTTP status 401: >> Unauthorized. >> at >> org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:222) >> at >> org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:129) >> at >> org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087) >> at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown >> Source) at >> org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown >> Source) at >> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown >> Source) at >> org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown >> Source) at >> org.apache.xerces.parsers.XML11Configuration.parse(Unknown >> Source) at >> org.apache.xerces.parsers.XML11Configuration.parse(Unknown >> Source) at >> org.apache.xerces.parsers.XMLParser.parse(Unknown >> Source) at >> org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown >> Source) at >> org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown >> Source) at >> org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown >> Source) at >> org.apache.axis.encoding.DeserializationContext.parse(DeserializationContext.java:227) >> at >> org.apache.axis.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:696) >> at >> org.apache.axis.Message.getSOAPEnvelope(Message.java:435) >> at >> org.apache.axis.handlers.soap.MustUnderstandChecker.invoke(MustUnderstandChecker.java:62) >> at >> org.apache.axis.client.AxisClient.invoke(AxisClient.java:206) >> at >> org.apache.axis.client.Call.invokeEngine(Call.java:2784) >> at >> org.apache.axis.client.Call.invoke(Call.java:2767) >> at >> org.apache.axis.client.Call.invoke(Call.java:2443) >> at >> org.apache.axis.client.Call.invoke(Call.java:2366) >> at >> org.apache.axis.client.Call.invoke(Call.java:1812) >> at >> com.microsoft.sharepoint.webpartpages.PermissionsSoapStub.getPermissionCollection(PermissionsSoapStub.java:200) >> at >> org.apache.manifoldcf.crawler.connectors.sharepoint.SPSProxyHelper.checkConnection(SPSProxyHelper.java:1466) >> at >> org.apache.manifoldcf.crawler.connectors.sharepoint.SharePointRepository.check(SharePointRepository.java:377) >> at >> org.apache.jsp.viewconnection_jsp._jspService(viewconnection_jsp.java:268) >> at >> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> at >> javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388) >> at >> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) >> at >> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) >> at >> javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >> at >> org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:288) >> at >> org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115) >> at >> org.apache.jasper.runtime.PageContextImpl.doForward(PageContextImpl.java:709) >> at >> org.apache.jasper.runtime.PageContextImpl.forward(PageContextImpl.java:680) >> at >> org.apache.jsp.execute_jsp._jspService(execute_jsp.java:328) >> at >> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) >> at >> javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388) >> at >> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313) >> at >> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260) >> at >> javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:480) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >> at >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) >> at >> org.eclipse.jetty.server.Server.handle(Server.java:349) >> at >> org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441) >> at >> org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:936) >> at >> org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:801) >> at >> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:224) >> at >> org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:51) >> at >> org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586) >> at >> org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533) >> >> >> >> >> ====================================================================================== >> >> >> [2013-09-02 16:58:07,986]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.DefaultHttpClient - Attempt 1 to execute request >> >> [2013-09-02 16:58:07,986]-[Thread-609]-[DEBUG] >> org.apache.http.impl.conn.DefaultClientConnection - Sending request: POST >> /_vti_bin/MCPermissions.asmx HTTP/1.1 >> >> [2013-09-02 16:58:07,987]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "POST /_vti_bin/MCPermissions.asmx HTTP/1.1[\r][\n]" >> >> [2013-09-02 16:58:07,987]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Content-Type: text/xml; charset=utf-8[\r][\n]" >> >> [2013-09-02 16:58:07,987]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Accept: */*[\r][\n]" >> >> [2013-09-02 16:58:07,988]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "SOAPAction: " >> http://microsoft.com/sharepoint/webpartpages/GetPermissionCollection >> "[\r][\n]" >> >> [2013-09-02 16:58:07,988]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "User-Agent: Axis/1.4[\r][\n]" >> >> [2013-09-02 16:58:07,988]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Content-Length: 420[\r][\n]" >> >> [2013-09-02 16:58:07,988]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Host: ec2-23-20-55-72.compute-1.amazonaws.com[\r][\n]" >> >> [2013-09-02 16:58:07,989]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Connection: Keep-Alive[\r][\n]" >> >> [2013-09-02 16:58:07,989]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Cookie: >> WSS_KeepSessionAuthenticated={5ecf7334-276b-471a-ae98-8591c7280476}[\r][\n]" >> >> [2013-09-02 16:58:07,989]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "Cookie2: $Version=1[\r][\n]" >> >> [2013-09-02 16:58:07,989]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "[\r][\n]" >> >> [2013-09-02 16:58:07,989]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> POST /_vti_bin/MCPermissions.asmx HTTP/1.1 >> >> [2013-09-02 16:58:07,990]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Content-Type: text/xml; charset=utf-8 >> >> [2013-09-02 16:58:07,990]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Accept: */* >> >> [2013-09-02 16:58:07,990]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> SOAPAction: " >> http://microsoft.com/sharepoint/webpartpages/GetPermissionCollection"; >> >> [2013-09-02 16:58:07,990]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> User-Agent: Axis/1.4 >> >> [2013-09-02 16:58:07,991]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Content-Length: 420 >> >> [2013-09-02 16:58:07,991]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Host: ec2-23-20-55-72.compute-1.amazonaws.com >> >> [2013-09-02 16:58:07,991]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Connection: Keep-Alive >> >> [2013-09-02 16:58:07,991]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Cookie: >> WSS_KeepSessionAuthenticated={5ecf7334-276b-471a-ae98-8591c7280476} >> >> [2013-09-02 16:58:07,992]-[Thread-609]-[DEBUG] org.apache.http.headers - >> >> Cookie2: $Version=1 >> >> [2013-09-02 16:58:07,992]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:07,993]-[Thread-609]-[DEBUG] org.apache.http.wire - >> >> "<?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv=" >> http://schemas.xmlsoap.org/soap/envelope/"; xmlns:xsd=" >> http://www.w3.org/2001/XMLSchema"; xmlns:xsi=" >> http://www.w3.org/2001/XMLSchema-instance";><soapenv:Body><GetPermissionCollection >> xmlns="http://microsoft.com/sharepoint/webpartpages/ >> "><objectName>/</objectName><objectType>Web</objectType></GetPermissionCollection></soapenv:Body></soapenv:Envelope>" >> >> [2013-09-02 16:58:08,023]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "HTTP/1.1 401 Unauthorized[\r][\n]" >> >> [2013-09-02 16:58:08,023]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "Server: Microsoft-IIS/7.5[\r][\n]" >> >> [2013-09-02 16:58:08,023]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "SPRequestGuid: 65ed9f96-b358-4a17-9d65-b58f55d19b17[\r][\n]" >> >> [2013-09-02 16:58:08,024]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "WWW-Authenticate: NTLM[\r][\n]" >> >> [2013-09-02 16:58:08,024]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "X-Powered-By: ASP.NET[\r][\n]" >> >> [2013-09-02 16:58:08,024]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "MicrosoftSharePointTeamServices: 14.0.0.4762[\r][\n]" >> >> [2013-09-02 16:58:08,024]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "Date: Mon, 02 Sep 2013 20:58:04 GMT[\r][\n]" >> >> [2013-09-02 16:58:08,025]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "Content-Length: 0[\r][\n]" >> >> [2013-09-02 16:58:08,025]-[Thread-609]-[DEBUG] org.apache.http.wire - << >> "[\r][\n]" >> >> [2013-09-02 16:58:08,026]-[Thread-609]-[DEBUG] >> org.apache.http.impl.conn.DefaultClientConnection - Receiving response: >> HTTP/1.1 401 Unauthorized >> >> [2013-09-02 16:58:08,026]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << HTTP/1.1 401 Unauthorized >> >> [2013-09-02 16:58:08,026]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << Server: Microsoft-IIS/7.5 >> >> [2013-09-02 16:58:08,026]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << SPRequestGuid: 65ed9f96-b358-4a17-9d65-b58f55d19b17 >> >> [2013-09-02 16:58:08,027]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << WWW-Authenticate: NTLM >> >> [2013-09-02 16:58:08,027]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << X-Powered-By: ASP.NET >> >> [2013-09-02 16:58:08,027]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << MicrosoftSharePointTeamServices: 14.0.0.4762 >> >> [2013-09-02 16:58:08,028]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << Date: Mon, 02 Sep 2013 20:58:04 GMT >> >> [2013-09-02 16:58:08,028]-[Thread-609]-[DEBUG] org.apache.http.headers - >> << Content-Length: 0 >> >> [2013-09-02 16:58:08,028]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.DefaultHttpClient - Connection can be kept >> alive indefinitely >> >> [2013-09-02 16:58:08,029]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.DefaultHttpClient - Authentication required >> >> [2013-09-02 16:58:08,029]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.DefaultHttpClient - >> ec2-23-20-55-72.compute-1.amazonaws.com:80 requested authentication >> >> [2013-09-02 16:58:08,029]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.TargetAuthenticationStrategy - Authentication >> schemes in the order of preference: [negotiate, Kerberos, NTLM, Digest, >> Basic] >> >> [2013-09-02 16:58:08,030]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for >> negotiate authentication scheme not available >> >> [2013-09-02 16:58:08,030]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for >> Kerberos authentication scheme not available >> >> [2013-09-02 16:58:08,030]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for >> Digest authentication scheme not available >> >> [2013-09-02 16:58:08,031]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.TargetAuthenticationStrategy - Challenge for >> Basic authentication scheme not available >> >> [2013-09-02 16:58:08,032]-[Thread-609]-[DEBUG] >> org.apache.http.impl.client.DefaultHttpClient - Selected authentication >> options: [NTLM] >> >> [2013-09-02 16:58:08,032]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:08,033]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:08,033]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:08,033]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:08,033]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:08,033]-[Thread-609]-[DEBUG] org.apache.axis.SOAPPart - >> Enter: SOAPPart::saveChanges >> >> [2013-09-02 16:58:08,034]-[Thread-609]-[DEBUG] >> org.apache.http.client.protocol.RequestAddCookies - CookieSpec selected: >> best-match >> >> [2013-09-02 16:58:08,034]-[Thread-609]-[DEBUG] >> org.apache.http.client.protocol.RequestAddCookies - Cookie [version: >> 0][name: WSS_KeepSessionAuthenticated][value: >> {5ecf7334-276b-471a-ae98-8591c7280476}][domain: >> ec2-23-20-55-72.compute-1.amazonaws.com][path: /][expiry: null] match [ >> ec2-23-20-55-72.compute-1.amazonaws.com:80/_vti_bin/MCPermissions.asmx] >> >> [2013-09-02 16:58:08,035]-[Thread-609]-[DEBUG] >> org.apache.http.client.protocol.RequestAuthCache - Auth cache not set in >> the context >> >> [2013-09-02 16:58:08,035]-[Thread-609]-[DEBUG] >> org.apache.http.client.protocol.RequestTargetAuthentication - Target auth >> state: CHALLENGED >> >> [2013-09-02 16:58:08,035]-[Thread-609]-[DEBUG] >> org.apache.http.client.protocol.RequestTargetAuthentication - Generating >> response to an authentication challenge using ntlm scheme >> >> [2013-09-02 16:58:08,036]-[Thread-609]-[DEBUG] >> org.apache.http.client.protocol.RequestProxyAuthentication - Proxy auth >> state: UNCHALLENGED >> > >
