The connector already should be ignoring certain kinds of errors and simply skipping the documents. The issue with 403 errors is that these *may* indicate bad credentials so we do not have any way of distinguishing these cases. A 503 error might be more easy to unambiguously deal with.
Karl On Fri, May 24, 2019 at 9:32 AM Julien <julien.massi...@francelabs.com> wrote: > Hi Karl, > > I recently experienced a particular case with the SharePoint connector > (2016) : > > During a job, it may occur, for some reasons (related to the SharePoint > server configuration, not the job), that some resources of a site are not > available (for instance if it requires some credentials to open a > resource). In that case, the SP connector gets a 403 or a 503 response code > from the SharePoint. The problem is that whenever it gets this kind of > response code, the job is aborted with an error. > This is problematic as it means that it will be painfull to complete a > crawl of an entire SP site as we cannot predict the resources that will not > be available. We could add filtering rules in order to avoid the resource > causing the error, but if we are talking about thousands or more of such > resources, it does not really scale from a human perspective. > Since the response codes are clearly identified (403 and 503), could we > envision that instead of aborting the job, we continue the job and log > something into the repo history ? > > Regards, > Julien Massiera > > > > --- > L'absence de virus dans ce courrier électronique a été vérifiée par le > logiciel antivirus Avast. > https://www.avast.com/antivirus >
