If that *fails*, let's try running the standard load tests on your machine. To do that, download the sources, and do the following:
ant make-core-deps ant load-hs That should take many hours, but if it fails you'll know there's something fundamentally wrong with your environment. Karl On Sat, Jul 13, 2019 at 4:39 PM Karl Wright <[email protected]> wrote: > >>>>>> > ERROR 2019-07-13T16:20:34,259 (Seeding thread) - Exception tossed: > Unexpected job status: 33 > org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected job > status: 33 > <<<<<< > > This is something I have never seen in any of the ManifoldCF tests ever > done. > > I am completely at a lost to explain why you are having tremendous > difficulties with very basic crawls. > > Could you do me the following favor: download the latest distribution, and > run the single-process example with NO configuration changes. Use an > Oracle JDK. Do your crawl and see if you get the same behavior. > > Thanks, > Karl > > > > On Sat, Jul 13, 2019 at 3:56 PM Cihad Guzel <[email protected]> wrote: > >> In addition my all job are waiting as "End notification" status now. >> >> Cihad Güzel >> >> >> Cihad Guzel <[email protected]>, 13 Tem 2019 Cmt, 22:27 tarihinde şunu >> yazdı: >> >>> Hi Karl, >>> >>> No, this is different new setup. But I use same version for mfc and >>> database. I am tring new setup for my every testing. I didn't see any >>> repeated or non-repeated error logs like before. >>> >>> Then, I have build the jdbc connector from trunk branch and changed the >>> jdbc-connector.jar with new version. Now, There are non-repeated new error >>> log as follow: >>> >>> ERROR 2019-07-13T16:20:34,259 (Seeding thread) - Exception tossed: >>> Unexpected job status: 33 >>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Unexpected >>> job status: 33 >>> at >>> org.apache.manifoldcf.crawler.jobs.JobManager.resetSeedJob(JobManager.java:7934) >>> ~[mcf-pull-agent.jar:?] >>> at >>> org.apache.manifoldcf.crawler.system.SeedingThread.run(SeedingThread.java:242) >>> [mcf-pull-agent.jar:?] >>> >>> Cihad Guzel >>> >>> >>> Karl Wright <[email protected]>, 13 Tem 2019 Cmt, 18:46 tarihinde şunu >>> yazdı: >>> >>>> You previously reported errors of the kind that ManifoldCF throws when >>>> it finds that the database seemingly lost transactional integrity. >>>> My question is whether you are still using the same database setup >>>> where you previously got those errors? >>>> >>>> The ArrayIndexOutOfBounds exception applied to JDBC connector metadata >>>> indexing. If you are using the JDBC connector with metadata without the >>>> patch then it could explain your problem also. But you would be seeing >>>> exceptions thrown over and over again in your logs. >>>> >>>> Karl >>>> >>>> >>>> On Sat, Jul 13, 2019 at 11:12 AM Cihad Guzel <[email protected]> wrote: >>>> >>>>> Hi Karl, >>>>> >>>>> If you are talking about is >>>>> https://issues.apache.org/jira/browse/CONNECTORS-1613, my setup >>>>> doesn't include this change because I use mfc 2.12. Are you suggesting I >>>>> use a trunk version? >>>>> >>>>> Cihad Güzel >>>>> >>>>> Karl Wright <[email protected]>, 13 Tem 2019 Cmt, 17:27 tarihinde >>>>> şunu yazdı: >>>>> >>>>>> Is this the same setup where you were getting errors because of >>>>>> inconsistent database states? >>>>>> That could lead to this problem, you know. >>>>>> >>>>>> Karl >>>>>> >>>>>> >>>>>> On Sat, Jul 13, 2019 at 10:14 AM Cihad Guzel <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Karl, >>>>>>> >>>>>>> I also have a job waiting for 12 days as an "Aborting" status. >>>>>>> >>>>>>> Status: Aborting >>>>>>> Start time: 7/1/19 4:01:46 PM >>>>>>> Documents: 10003 >>>>>>> Active: 10003 >>>>>>> Processed: 10002 >>>>>>> >>>>>>> Cihad Guzel >>>>>>> >>>>>>> >>>>>>> Cihad Guzel <[email protected]>, 13 Tem 2019 Cmt, 16:53 tarihinde >>>>>>> şunu yazdı: >>>>>>> >>>>>>>> Hi Karl, >>>>>>>> >>>>>>>> I tried quick-start single process model. After your suggestion , i >>>>>>>> have tried multiprocess-zk-example for zookeeper-based locking. But I >>>>>>>> have >>>>>>>> the same problem. >>>>>>>> >>>>>>>> Status: Aborting >>>>>>>> Start time: 7/13/19 3:42:10 PM >>>>>>>> Documents: 10003 >>>>>>>> Active: 10003 >>>>>>>> Processed: 1021 >>>>>>>> >>>>>>>> I'm waiting for over an hour for the jdbc job to stop. I have not >>>>>>>> any error logs in my manifolcf log. >>>>>>>> >>>>>>>> Cihad Güzel >>>>>>>> >>>>>>>> Cihad Güzel >>>>>>>> >>>>>>>> >>>>>>>> Karl Wright <[email protected]>, 8 Tem 2019 Pzt, 13:23 tarihinde >>>>>>>> şunu yazdı: >>>>>>>> >>>>>>>>> Are you using file-based locking? >>>>>>>>> If so, I would suggest strongly migrating to zookeeper-based >>>>>>>>> locking. >>>>>>>>> But if you are using the file-based locking, please execute the >>>>>>>>> "lock clean procedure" as follows: >>>>>>>>> >>>>>>>>> - shut down all manifoldcf processes, including the web UI >>>>>>>>> - run the lock-clean script >>>>>>>>> - start the processes again >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Karl >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jul 8, 2019 at 5:05 AM Cihad Guzel <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Karl, >>>>>>>>>> >>>>>>>>>> Nothing. I don't have any error log. >>>>>>>>>> >>>>>>>>>> 8 Tem 2019 Pzt 03:18 tarihinde Karl Wright <[email protected]> >>>>>>>>>> şunu yazdı: >>>>>>>>>> >>>>>>>>>>> Hi Cihad, >>>>>>>>>>> >>>>>>>>>>> What does your manifoldcf log have in it? Any errors? >>>>>>>>>>> >>>>>>>>>>> Karl >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Jul 7, 2019 at 3:52 PM Cihad Guzel <[email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Karl, >>>>>>>>>>>> >>>>>>>>>>>> I mistakenly wrote "Stopping" instead of "Aborting". My job is >>>>>>>>>>>> waiting as "Aborting" status. I have also the same problem while >>>>>>>>>>>> restarting. I am waiting for 2 days for one job. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Cihad Guzel >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Cihad Guzel <[email protected]>, 7 Tem 2019 Paz, 22:42 >>>>>>>>>>>> tarihinde şunu yazdı: >>>>>>>>>>>> >>>>>>>>>>>>> Hi Karl, >>>>>>>>>>>>> >>>>>>>>>>>>> I have a few jobs. I stopped all of them but only one job is >>>>>>>>>>>>> waiting as "stopping" status. >>>>>>>>>>>>> >>>>>>>>>>>>> I know that some large jobs is waited long time. But, I have >>>>>>>>>>>>> only 1000 rows on database. So, all of jobs crawled small number >>>>>>>>>>>>> of >>>>>>>>>>>>> documents. But , It doesn't make much sense to stay status of >>>>>>>>>>>>> "stopping" >>>>>>>>>>>>> for a long time. How can I identify a problem? >>>>>>>>>>>>> >>>>>>>>>>>>> Postgresql version: 9.4 >>>>>>>>>>>>> Manifoldcf version: 2.12 >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Cihad Guzel >>>>>>>>>>>>> >>>>>>>>>>>>
