Oh, wait - you messed with the file, right? Probably you changed the
order of the entries around by doing so...
The current 0.5 code outputs the right order:
java.util.zip.ZipEntry outputEntry = new
java.util.zip.ZipEntry("outputs");
zos.putNextEntry(outputEntry);
outputManager.exportConfiguration(zos);
zos.closeEntry();
java.util.zip.ZipEntry authEntry = new
java.util.zip.ZipEntry("authorities");
zos.putNextEntry(authEntry);
authManager.exportConfiguration(zos);
zos.closeEntry();
java.util.zip.ZipEntry connEntry = new
java.util.zip.ZipEntry("connections");
zos.putNextEntry(connEntry);
connManager.exportConfiguration(zos);
zos.closeEntry();
java.util.zip.ZipEntry jobsEntry = new
java.util.zip.ZipEntry("jobs");
zos.putNextEntry(jobsEntry);
jobManager.exportConfiguration(zos);
zos.closeEntry();
... so there's no *current* problem with this. If you've messed with
the order though and still want to use the file, try this:
- unpack
- pack each individual entry in its own zip
- import them, one by one, in the order above (outputs, authorities,
connections, jobs)
I bet that will work for you...
Karl
On Mon, Apr 23, 2012 at 1:33 PM, Karl Wright <[email protected]> wrote:
> The problem is not with the current version; it's with the ManifoldCF
> 0.3 version, which did not output things in the correct order.
> Therefore the outputs do not get processed before the jobs do, leading
> to this problem.
>
> There are two possible solutions. The first is to hack up your copy
> of 0.5 to do TWO passes over the data file, looking for "outputs" on
> the first pass, and looking for everything on the second. The second
> is create the "Solr" output connection by hand; running the script
> after that should populate everything.
>
> Thanks,
> Karl
>
>
> On Mon, Apr 23, 2012 at 9:25 AM, Karl Wright <[email protected]> wrote:
>> Hmm. Your output shows that the output connector is indeed registered:
>>
>> ./processes/script/executecommand.sh
>> org.apache.manifoldcf.agents.RegisterOutput
>> org.apache.manifoldcf.agents.output.solr.SolrConnector SolrConnector
>>
>> If you have no passwords in the config, you can send me the file and
>> I'll debug it to see what's going on.
>>
>> Karl
>>
>> On Mon, Apr 23, 2012 at 9:04 AM, Karl Wright <[email protected]> wrote:
>>> Hmm. There have been no changes to the format or meaning of the
>>> configuration file AFAIK for quite a while.
>>>
>>> This is complaining about there not being an output connection called
>>> "Solr" in place at the time it tries to create a job. In theory, on
>>> import, the creation of connections takes place before the creation of
>>> jobs. Is there any possibility that the connectors have not been
>>> registered? I think they would need to be before you attempt the
>>> import.
>>>
>>> Karl
>>>
>>> On Mon, Apr 23, 2012 at 8:57 AM, Erlend Garåsen <[email protected]>
>>> wrote:
>>>>
>>>> I'm trying to import my old configuration files into my freshly installed
>>>> MCF version, but I'm afraid that these configuration data is too old (from
>>>> version 0.3, I think). I get the following error: "Database exception:
>>>> Exception doing query: ERROR: insert or update on table "jobs" violates
>>>> foreign key constraint "jobs_outputname_fkey"
>>>> Detail: Key (outputname)=(Solr) is not present in table
>>>> "outputconnections"".
>>>>
>>>> Here's what I tried to do. The last step fails:
>>>> ./processes/script/executecommand.sh org.apache.manifoldcf.agents.Install
>>>> ./processes/script/executecommand.sh
>>>> org.apache.manifoldcf.agents.RegisterOutput
>>>> org.apache.manifoldcf.agents.output.solr.SolrConnector SolrConnector
>>>> ./processes/script/executecommand.sh org.apache.manifoldcf.agents.Register
>>>> org.apache.manifoldcf.crawler.system.CrawlerAgent
>>>> ./processes/script/executecommand.sh org.apache.manifoldcf.crawler.Register
>>>> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnector
>>>> WebCrawler
>>>> ./processes/script/executecommand.sh
>>>> org.apache.manifoldcf.crawler.ImportConfiguration export_mcf.zip
>>>>
>>>> Complete stack trace:
>>>>
>>>> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Database
>>>> exception: Exception doing query: ERROR: insert or update on table "jobs"
>>>> violates foreign key constraint "jobs_outputname_fkey"
>>>> Detail: Key (outputname)=(Solr) is not present in table
>>>> "outputconnections".
>>>> at
>>>> org.apache.manifoldcf.core.database.Database.executeViaThread(Database.java:680)
>>>> at
>>>> org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:708)
>>>> at
>>>> org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1393)
>>>> at
>>>> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:144)
>>>> at
>>>> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:185)
>>>> at
>>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performModification(DBInterfacePostgreSQL.java:631)
>>>> at
>>>> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performInsert(DBInterfacePostgreSQL.java:182)
>>>> at
>>>> org.apache.manifoldcf.core.database.BaseTable.performInsert(BaseTable.java:68)
>>>> at org.apache.manifoldcf.crawler.jobs.Jobs.save(Jobs.java:756)
>>>> at
>>>> org.apache.manifoldcf.crawler.jobs.JobManager.save(JobManager.java:576)
>>>> at
>>>> org.apache.manifoldcf.crawler.jobs.JobManager.importConfiguration(JobManager.java:230)
>>>> at
>>>> org.apache.manifoldcf.crawler.system.ManifoldCF.importConfiguration(ManifoldCF.java:1100)
>>>> at
>>>> org.apache.manifoldcf.crawler.ImportConfiguration.doExecute(ImportConfiguration.java:40)
>>>> at
>>>> org.apache.manifoldcf.crawler.BaseCrawlerInitializationCommand.execute(BaseCrawlerInitializationCommand.java:37)
>>>> at
>>>> org.apache.manifoldcf.crawler.ImportConfiguration.main(ImportConfiguration.java:57)
>>>> Caused by: org.postgresql.util.PSQLException: ERROR: insert or update on
>>>> table "jobs" violates foreign key constraint "jobs_outputname_fkey"
>>>> Detail: Key (outputname)=(Solr) is not present in table
>>>> "outputconnections".
>>>> at
>>>> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103)
>>>> at
>>>> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836)
>>>> at
>>>> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
>>>> at
>>>> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512)
>>>> at
>>>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388)
>>>> at
>>>> org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:334)
>>>> at
>>>> org.apache.manifoldcf.core.database.Database.execute(Database.java:839)
>>>> at
>>>> org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(Database.java:640)
>>>>
>>>>
>>>> --
>>>> Erlend Garåsen
>>>> Center for Information Technology Services
>>>> University of Oslo
>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050
