Hi Lalit, In theory the combined war will run with external locking, it's just that nobody ever thought that would be how people would use it, and therefore it hasn't been tested etc.
I would suggest that for a development setup, you use the Jetty simple example. For a production setup, you should adopt the standard distributed arrangement as follows: - a deployed mcf-crawler-ui war - a deployed mcf-authority-service war - an (optional) deployed mcf-api-service war - one or more agents processes This is how it is done in both kinds of multiprocess examples, and is described thoroughly in the how-to-build-and-deploy document online. Thanks, Karl On Wed, Jul 9, 2014 at 10:56 AM, lalit jangra <[email protected]> wrote: > Thanks Karl, > > My idea was as its name suggests mcf-combined-war as i was trying to take > advantage of agent process inside war. My assumption is wrong here as i can > see here. > > In this case, which war should i use - mcf-crawler-ui.war? If i use this > or any of other wars, then should i start agent process? > > Please suggest. > > Regards. > > > On Wed, Jul 9, 2014 at 3:48 PM, Karl Wright <[email protected]> wrote: > >> Hi Lalit, >> >> Probably it is not a database issue, but how locking is configured. >> Typically the combined war is run in a non-clustered environment with no >> external synchronization required (as it is in the example I pointed you >> at). We don't currently have an example that uses combined war with >> external locking -- that would not make much sense. Can you tell me why >> you are trying to do it that way? >> >> Karl >> >> >> >> On Wed, Jul 9, 2014 at 10:46 AM, lalit jangra <[email protected]> >> wrote: >> >>> Hi Karl, >>> >>> if i try ./start-combined.sh, the job runs and i could see it finishing >>> gracefully with updating index in solr. >>> >>> I assume there is some DB issue here? Any suggestions? >>> >>> Regards. >>> >>> >>> On Wed, Jul 9, 2014 at 3:17 PM, Karl Wright <[email protected]> wrote: >>> >>>> Hi Lalit, >>>> >>>> Please do the following: In an unmodified dist/example directory, run >>>> the start-combined script. Open the UI, create the connections, and try to >>>> run the job. Does it work for you? >>>> >>>> >>>> Karl >>>> >>>> >>>> >>>> On Wed, Jul 9, 2014 at 10:09 AM, lalit jangra <[email protected] >>>> > wrote: >>>> >>>>> Thanks Karl, >>>>> >>>>> I have reconfigured everything and this time i am starting tomcat and >>>>> deploying mcf-combined-war. I created output connection, repository >>>>> connection & job. Still when i start my job, it does not go ahead and >>>>> stuck >>>>> at starting up status. >>>>> >>>>> >>>>> Regards. >>>>> >>>>> >>>>> >>>>> On Wed, Jul 9, 2014 at 12:49 PM, Karl Wright <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Lalit, >>>>>> >>>>>> You do not use combined.war AND an agents process at the same time. >>>>>> The combined war INCLUDES the agents process. >>>>>> >>>>>> Karl >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Jul 9, 2014 at 6:18 AM, lalit jangra < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> Hi Karl, >>>>>>> >>>>>>> As per your suggestion, i built a new MCF 1.5.1 instance, con from >>>>>>> source , configured it and pointed to local postgresql DB in >>>>>>> properties.xml under /multiprocess-file-example. Then i tool a brand >>>>>>> new >>>>>>> tomcat and configured for MCF in it in setenv.sh for properties.xml >>>>>>> using >>>>>>> "-Dorg.apache.manifoldcf.configfile" argument and deployed >>>>>>> mcf-combined.war >>>>>>> using app manager. >>>>>>> >>>>>>> First i initialized DB using ./initialize.sh command & the moment i >>>>>>> started agent using ./start-agents.sh command under >>>>>>> /multiprocess-file-example, i got the error, even when tomcat is not >>>>>>> started at this time. I created a job which never runs & get stuck. >>>>>>> >>>>>>> >>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed: >>>>>>> Service 'A' of type >>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is already >>>>>>> active >>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service >>>>>>> 'A' of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>> is >>>>>>> already active >>>>>>> >>>>>>> But if i try to run jetty bundle using start.jar under example, no >>>>>>> such error but it does not fulfil my requirements. >>>>>>> >>>>>>> Regards. >>>>>>> >>>>>>> >>>>>>> On Wed, Jul 9, 2014 at 10:03 AM, Karl Wright <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> So, lalit, if you run the multiprocess example without any changes, >>>>>>>> do >>>>>>>> you see this? I don't. >>>>>>>> >>>>>>>> Karl >>>>>>>> >>>>>>>> Sent from my Windows Phone >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: lalit jangra >>>>>>>> Sent: 7/9/2014 12:17 AM >>>>>>>> To: Karl Wright >>>>>>>> Cc: [email protected] >>>>>>>> Subject: Re: Apache ManifoldCF job stuck up >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks Karl, >>>>>>>> >>>>>>>> >>>>>>>> I am currently running only single agent process on single machine >>>>>>>> without clustering. I have two environments and i could see this >>>>>>>> issue >>>>>>>> coming up at both places. >>>>>>>> >>>>>>>> While trying to start agent, i could see below error. When i am >>>>>>>> trying >>>>>>>> to start agent, it says below error and exits. But there is no agent >>>>>>>> process already running. >>>>>>>> >>>>>>>> >>>>>>>> [root@server1 multiprocess-file-example]# ./start-agents.sh & >>>>>>>> >>>>>>>> [1] 5020 >>>>>>>> >>>>>>>> [root@server1 multiprocess-file-example]# Running... >>>>>>>> >>>>>>>> Configuration file successfully read >>>>>>>> >>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service >>>>>>>> 'A' >>>>>>>> of type 'AGENT' is already active >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.AgentRun.doExecute(AgentRun.java:54) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.BaseAgentsInitializationCommand.execute(BaseAgentsInitializationCommand.java:37) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.AgentRun.main(AgentRun.java:93) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> [1]+ Exit 1 ./start-agents.sh >>>>>>>> >>>>>>>> >>>>>>>> If i am able to successfully start agent using ./start-agents.sh, >>>>>>>> still it throws same error but i have no other process running. >>>>>>>> >>>>>>>> Regards. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jul 8, 2014 at 2:38 PM, Karl Wright <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hi lalit, >>>>>>>> >>>>>>>> This occurs when you have more than one agents process with the same >>>>>>>> process id using the same shared file system directory / zookeeper >>>>>>>> cluster. There is no other way it can occur. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Karl >>>>>>>> >>>>>>>> Sent from my Windows Phone >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -----Original Message----- >>>>>>>> From: lalit jangra >>>>>>>> Sent: 7/8/2014 8:38 AM >>>>>>>> To: [email protected] >>>>>>>> Subject: Re: Apache ManifoldCF job stuck up >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Thanks Karl, >>>>>>>> >>>>>>>> >>>>>>>> I have tried steps you suggested and it worked on one instance. >>>>>>>> >>>>>>>> But on another instance i am still not able to resolve this issue. >>>>>>>> Along with steps you mentioned, i tried recreating DB instance >>>>>>>> again, >>>>>>>> setting up new MCF instance , cleaning locks and then starting >>>>>>>> agents >>>>>>>> first & then tomcat. But issue still persists. >>>>>>>> >>>>>>>> If i try to run ./start-agents.sh, i get this error for agent A. >>>>>>>> >>>>>>>> >>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed: >>>>>>>> Service 'A' of type >>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is >>>>>>>> already >>>>>>>> active >>>>>>>> >>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service >>>>>>>> 'A' >>>>>>>> of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>>> is >>>>>>>> already active >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> But if i try to run ./start-agents-2.sh, i see similar error but >>>>>>>> agent B >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ERROR 2014-07-08 13:32:19,823 (Agents thread) - Exception tossed: >>>>>>>> Service 'B' of type >>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is >>>>>>>> already >>>>>>>> active >>>>>>>> >>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service >>>>>>>> 'B' >>>>>>>> of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>>> is >>>>>>>> already active >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Regards. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jul 7, 2014 at 4:55 PM, Karl Wright <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi Lalit, >>>>>>>> >>>>>>>> >>>>>>>> If you are using file synchronization, you cannot expect MCF to >>>>>>>> clean >>>>>>>> up itself unless you shut it down cleanly. You should be either >>>>>>>> using >>>>>>>> ^C or plain kill, NEVER kill -9. kill -9 will leave dangling >>>>>>>> locks. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> To clean up dangling locks: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> - shut ALL manifoldcf processes and web apps down >>>>>>>> >>>>>>>> >>>>>>>> - run the lock-clean script >>>>>>>> >>>>>>>> >>>>>>>> - start up the processes again >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Zookeeper synchronization, by the way, does not have this kind of >>>>>>>> problem. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Karl >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Jul 7, 2014 at 11:49 AM, lalit jangra < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> >>>>>>>> I configured MCF 1.5.1 to run with Postgresql DB and tomcat 7. >>>>>>>> Initially i created all connections, created a an alfresco job and >>>>>>>> it >>>>>>>> all worked fine. >>>>>>>> >>>>>>>> >>>>>>>> Next for updates, i stopped, tomcat and agent process running. Then >>>>>>>> i >>>>>>>> updated CmisRepositoryConnector.java with my own code and run "ant >>>>>>>> build" at root of MCF. It updated all code and jars file. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Also properties.xml is reset under /dist/multiprocess-file/example >>>>>>>> which i updated to connect to Postgresql DB and logging >>>>>>>> configurations. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I started tomcat then agent process. Finally i started job to crawl >>>>>>>> alfresco but it got stuck and moving on. I checked into >>>>>>>> /dist/multiprocess-file/example/logs/manifoldcf.log file and could >>>>>>>> see >>>>>>>> below error. >>>>>>>> >>>>>>>> >>>>>>>> ERROR 2014-07-07 16:09:04,936 (Agents thread) - Exception tossed: >>>>>>>> Service '' of type >>>>>>>> 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' is >>>>>>>> already >>>>>>>> active >>>>>>>> >>>>>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Service >>>>>>>> '' >>>>>>>> of type 'AGENT_org.apache.manifoldcf.crawler.system.CrawlerAgent' >>>>>>>> is >>>>>>>> already active >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:156) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.BaseLockManager.registerServiceBeginServiceActivity(BaseLockManager.java:120) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.core.lockmanager.LockManager.registerServiceBeginServiceActivity(LockManager.java:69) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon.checkAgents(AgentsDaemon.java:270) >>>>>>>> >>>>>>>> at >>>>>>>> org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsThread.run(AgentsDaemon.java:208) >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> I created another job but that got stuck too.Is DB got corrupt due >>>>>>>> to rebuild? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Also is this the right way to build MCF (I hope its correct way). >>>>>>>> Now >>>>>>>> what should i do to fix this issue? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Please help. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Lalit Jangra. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Lalit Jangra. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Lalit Jangra. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Lalit Jangra. >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Regards, >>>>> Lalit Jangra. >>>>> >>>> >>>> >>> >>> >>> -- >>> Regards, >>> Lalit Jangra. >>> >> >> > > > -- > Regards, > Lalit Jangra. >
