Re: Job interrupted
Ok, I found the 'hard fail' situation. Here is a patch to fix it:
Index:
connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.java
===
---
connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.java
(revision 1881006)
+++
connectors/jcifs/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/sharedrive/SharedDriveConnector.java
(working copy)
@@ -1349,7 +1349,7 @@
Logging.connectors.warn("JCIFS: 'File in Use' response when
"+activity+" for "+documentIdentifier+": retrying...",se);
// 'File in Use' skip the document and keep going
throw new ServiceInterruption("Timeout or other service
interruption: "+se.getMessage(),se,currentTime + 30L,
-currentTime + 3 * 60 * 6L,-1,true);
+currentTime + 3 * 60 * 6L,-1,false);
}
else if (se.getMessage().indexOf("cannot find") != -1 ||
se.getMessage().indexOf("cannot be found") != -1)
{
I'll commit to trunk as well.
Karl
On Mon, Aug 24, 2020 at 9:19 AM Karl Wright wrote:
> Ok, then let me examine the code and see why it's not catching it.
> Karl
>
>
> On Mon, Aug 24, 2020 at 8:49 AM Bisonti Mario
> wrote:
>
>> Yes, I see only that exception inside the manifoldcf.log and the job
>> stops with:
>>
>>
>>
>>
>>
>> Error: Repeated service interruptions - failure processing document: The
>> process cannot access the file because it is being used by another process.
>>
>>
>>
>>
>>
>> *Da:* Karl Wright
>> *Inviato:* lunedì 24 agosto 2020 12:27
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job interrupted
>>
>>
>>
>> Well, we look for certain kinds of exceptions from JCIFS and allow the
>> job to continue if we can't succeed. You have to be sure though that the
>> failure was from *that* exception. The reason I point that out is because
>> we have already a check for that, I believe.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Mon, Aug 24, 2020 at 5:55 AM Bisonti Mario
>> wrote:
>>
>> Yes, but after I obtain:
>>
>>
>>
>> Error: Repeated service interruptions - failure processing document: The
>> process cannot access the file because it is being used by another process.
>>
>>
>>
>> And the job stops
>>
>>
>>
>>
>>
>> *Da:* Karl Wright
>> *Inviato:* lunedì 24 agosto 2020 11:52
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job interrupted
>>
>>
>>
>> Hi,
>> That's a warning. The job will keep running and the document will be
>> retried later.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Mon, Aug 24, 2020 at 5:24 AM Bisonti Mario
>> wrote:
>>
>> Hallo.
>>
>> I have some problems about job interrupted.
>>
>> The job execute a windows share scan
>>
>>
>>
>> After many errors, sometimes it stops
>>
>>
>>
>> I see in the manifoldcf.log many errors:
>>
>>
>>
>>
>>
>> at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610)
>> [mcf-jcifs-connector.jar:?]
>>
>> at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2020-08-24T11:17:25,501 (Worker thread '59') - JCIFS: 'File in Use'
>> response when getting document version for smb://
>> fileserver.net/Workgroups/Dir/Dir2/finename.xlsx
>> <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Ffileserver.net%2FWorkgroups%2FDir%2FDir2%2Ffinename.xlsx=01%7C01%7CMario.Bisonti%40vimar.com%7Cd726636fb2744bb0882c08d848185962%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1=lvpKI2hFeY40s4vgbQViO%2BfxXQBivrz4CFD3kHNKy2Q%3D=0>:
>> retrying...
>>
>> jcifs.smb.SmbException: The process cannot access the file because it is
>> being used by another process.
>>
>> at
>> jcifs.smb.SmbTransportImpl.checkStatus2(SmbTransportImpl.java:1441)
>> ~[jcifs-ng-2.1.2.jar:?]
>>
>> at
>> jcifs.smb.SmbTransportImpl.checkStatus(SmbTransportImpl.java:1552)
>> ~[jcifs-ng-2.1.2.jar:?]
>>
>> at
>> jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:1007)
>> ~[jcifs-ng-2.1.2.jar:?]
>>
&g
Re: Job interrupted
Ok, then let me examine the code and see why it's not catching it. Karl On Mon, Aug 24, 2020 at 8:49 AM Bisonti Mario wrote: > Yes, I see only that exception inside the manifoldcf.log and the job stops > with: > > > > > > Error: Repeated service interruptions - failure processing document: The > process cannot access the file because it is being used by another process. > > > > > > *Da:* Karl Wright > *Inviato:* lunedì 24 agosto 2020 12:27 > *A:* user@manifoldcf.apache.org > *Oggetto:* Re: Job interrupted > > > > Well, we look for certain kinds of exceptions from JCIFS and allow the job > to continue if we can't succeed. You have to be sure though that the > failure was from *that* exception. The reason I point that out is because > we have already a check for that, I believe. > > > > Karl > > > > > > On Mon, Aug 24, 2020 at 5:55 AM Bisonti Mario > wrote: > > Yes, but after I obtain: > > > > Error: Repeated service interruptions - failure processing document: The > process cannot access the file because it is being used by another process. > > > > And the job stops > > > > > > *Da:* Karl Wright > *Inviato:* lunedì 24 agosto 2020 11:52 > *A:* user@manifoldcf.apache.org > *Oggetto:* Re: Job interrupted > > > > Hi, > That's a warning. The job will keep running and the document will be > retried later. > > > > Karl > > > > > > On Mon, Aug 24, 2020 at 5:24 AM Bisonti Mario > wrote: > > Hallo. > > I have some problems about job interrupted. > > The job execute a windows share scan > > > > After many errors, sometimes it stops > > > > I see in the manifoldcf.log many errors: > > > > > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > > WARN 2020-08-24T11:17:25,501 (Worker thread '59') - JCIFS: 'File in Use' > response when getting document version for smb:// > fileserver.net/Workgroups/Dir/Dir2/finename.xlsx > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Ffileserver.net%2FWorkgroups%2FDir%2FDir2%2Ffinename.xlsx=01%7C01%7CMario.Bisonti%40vimar.com%7Cd726636fb2744bb0882c08d848185962%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1=lvpKI2hFeY40s4vgbQViO%2BfxXQBivrz4CFD3kHNKy2Q%3D=0>: > retrying... > > jcifs.smb.SmbException: The process cannot access the file because it is > being used by another process. > > at > jcifs.smb.SmbTransportImpl.checkStatus2(SmbTransportImpl.java:1441) > ~[jcifs-ng-2.1.2.jar:?] > > at > jcifs.smb.SmbTransportImpl.checkStatus(SmbTransportImpl.java:1552) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:1007) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:409) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeImpl.send(SmbTreeImpl.java:472) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send0(SmbTreeConnection.java:399) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send(SmbTreeConnection.java:314) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send(SmbTreeConnection.java:294) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeHandleImpl.send(SmbTreeHandleImpl.java:130) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeHandleImpl.send(SmbTreeHandleImpl.java:117) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1747) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1716) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1710) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.queryPath(SmbFile.java:763) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.exists(SmbFile.java:844) > ~[jcifs-ng-2.1.2.jar:?] > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileExists(SharedDriveConnector.java:2188) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > > WARN 2020-08-24T11:17:25,502 (Worker thread '59') - Service interruption > reported for job 1533797717712 connection 'WinShare': Timeout or other > service interruption: The process cannot access the file because it is > being used by another process. > > > > > > What could I check? > > > > Thanks a lot > > Mario > >
Re: Job interrupted
Well, we look for certain kinds of exceptions from JCIFS and allow the job to continue if we can't succeed. You have to be sure though that the failure was from *that* exception. The reason I point that out is because we have already a check for that, I believe. Karl On Mon, Aug 24, 2020 at 5:55 AM Bisonti Mario wrote: > Yes, but after I obtain: > > > > Error: Repeated service interruptions - failure processing document: The > process cannot access the file because it is being used by another process. > > > > And the job stops > > > > > > *Da:* Karl Wright > *Inviato:* lunedì 24 agosto 2020 11:52 > *A:* user@manifoldcf.apache.org > *Oggetto:* Re: Job interrupted > > > > Hi, > That's a warning. The job will keep running and the document will be > retried later. > > > > Karl > > > > > > On Mon, Aug 24, 2020 at 5:24 AM Bisonti Mario > wrote: > > Hallo. > > I have some problems about job interrupted. > > The job execute a windows share scan > > > > After many errors, sometimes it stops > > > > I see in the manifoldcf.log many errors: > > > > > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > > WARN 2020-08-24T11:17:25,501 (Worker thread '59') - JCIFS: 'File in Use' > response when getting document version for smb:// > fileserver.net/Workgroups/Dir/Dir2/finename.xlsx > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Ffileserver.net%2FWorkgroups%2FDir%2FDir2%2Ffinename.xlsx=01%7C01%7CMario.Bisonti%40vimar.com%7Ca26fd37fa4af4fe8b96708d848135dc1%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1=iMDk%2FqLW6FLe3gPsqwKVba6OFJw7HZd5XoRTUQGH7tg%3D=0>: > retrying... > > jcifs.smb.SmbException: The process cannot access the file because it is > being used by another process. > > at > jcifs.smb.SmbTransportImpl.checkStatus2(SmbTransportImpl.java:1441) > ~[jcifs-ng-2.1.2.jar:?] > > at > jcifs.smb.SmbTransportImpl.checkStatus(SmbTransportImpl.java:1552) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:1007) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:409) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeImpl.send(SmbTreeImpl.java:472) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send0(SmbTreeConnection.java:399) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send(SmbTreeConnection.java:314) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send(SmbTreeConnection.java:294) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeHandleImpl.send(SmbTreeHandleImpl.java:130) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeHandleImpl.send(SmbTreeHandleImpl.java:117) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1747) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1716) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1710) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.queryPath(SmbFile.java:763) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.exists(SmbFile.java:844) > ~[jcifs-ng-2.1.2.jar:?] > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileExists(SharedDriveConnector.java:2188) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > > WARN 2020-08-24T11:17:25,502 (Worker thread '59') - Service interruption > reported for job 1533797717712 connection 'WinShare': Timeout or other > service interruption: The process cannot access the file because it is > being used by another process. > > > > > > What could I check? > > > > Thanks a lot > > Mario > >
Re: Job interrupted
Hi, That's a warning. The job will keep running and the document will be retried later. Karl On Mon, Aug 24, 2020 at 5:24 AM Bisonti Mario wrote: > Hallo. > > I have some problems about job interrupted. > > The job execute a windows share scan > > > > After many errors, sometimes it stops > > > > I see in the manifoldcf.log many errors: > > > > > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > > WARN 2020-08-24T11:17:25,501 (Worker thread '59') - JCIFS: 'File in Use' > response when getting document version for smb:// > fileserver.net/Workgroups/Dir/Dir2/finename.xlsx: retrying... > > jcifs.smb.SmbException: The process cannot access the file because it is > being used by another process. > > at > jcifs.smb.SmbTransportImpl.checkStatus2(SmbTransportImpl.java:1441) > ~[jcifs-ng-2.1.2.jar:?] > > at > jcifs.smb.SmbTransportImpl.checkStatus(SmbTransportImpl.java:1552) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTransportImpl.sendrecv(SmbTransportImpl.java:1007) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTransportImpl.send(SmbTransportImpl.java:1523) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbSessionImpl.send(SmbSessionImpl.java:409) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeImpl.send(SmbTreeImpl.java:472) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send0(SmbTreeConnection.java:399) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send(SmbTreeConnection.java:314) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeConnection.send(SmbTreeConnection.java:294) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeHandleImpl.send(SmbTreeHandleImpl.java:130) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbTreeHandleImpl.send(SmbTreeHandleImpl.java:117) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1747) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1716) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.withOpen(SmbFile.java:1710) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.queryPath(SmbFile.java:763) > ~[jcifs-ng-2.1.2.jar:?] > > at jcifs.smb.SmbFile.exists(SmbFile.java:844) > ~[jcifs-ng-2.1.2.jar:?] > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileExists(SharedDriveConnector.java:2188) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:610) > [mcf-jcifs-connector.jar:?] > > at > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) > [mcf-pull-agent.jar:?] > > WARN 2020-08-24T11:17:25,502 (Worker thread '59') - Service interruption > reported for job 1533797717712 connection 'WinShare': Timeout or other > service interruption: The process cannot access the file because it is > being used by another process. > > > > > > What could I check? > > > > Thanks a lot > > Mario > >
