Re: Revisiting: Should Manifold include Pipelines
Hi Karl, Still pondering our last discussion. Wondering if I got things off track. As a start, what if I backtracked a bit, to this: What's the easiest way to do this: * A connector that tweaks metadata form a single source. * Sits between any existing MCF datasource connector and the main MCF engine Before: CMS/DB - Existing MCF connector - MCF core - output After: CMS/DB - Existing MCF connector - Metadata tweaker - MCF core - output Assume the matadata changes don't have any impact on security, or that no security is being used (public data)
Re: Revisiting: Should Manifold include Pipelines
Hi Mark, I think I'd describe this simplified proposal as pipeline (vs. Pipeline. Your original description was the latter.) This proposal is simpler but does not have the ability to amalgamate content from multiple connectors, correct? As long as it is just modifying the content and metadata (as described by RepositoryDocument), it's not hard to develop a generic idea of a content processing pipeline, e.g. Tika. There's a question in my mind as to where it belongs. If its purpose is to make up for missing code in particular search engines, then I'd argue it should be a service available to output connector coders, who can then choose how much configurability makes sense from the point of view of their target system. For instance, since Tika is already part of Solr, there would seem little benefit in adding a Tika pipeline upstream of Solr as well, but maybe a Google Appliance connector would want it and therefore expose it. If the pipeline's purpose is to include arbitrary business logic, on the other hand, then I think what you'd really need is a Pipeline and not a pipeline, if you see what I mean. So, my question to you is, what would the main use case(s) be for a pipeline in your view? Karl On Wed, Jan 11, 2012 at 6:31 AM, Mark Bennett mbenn...@ideaeng.com wrote: Hi Karl, Still pondering our last discussion. Wondering if I got things off track. As a start, what if I backtracked a bit, to this: What's the easiest way to do this: * A connector that tweaks metadata form a single source. * Sits between any existing MCF datasource connector and the main MCF engine Before: CMS/DB - Existing MCF connector - MCF core - output After: CMS/DB - Existing MCF connector - Metadata tweaker - MCF core - output Assume the matadata changes don't have any impact on security, or that no security is being used (public data)
[jira] [Updated] (CONNECTORS-373) Solr connector Japanese message properties file is not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitoshi Ozawa updated CONNECTORS-373: - Attachment: CONNECTORS-373.patch Solr connector Japanese message properties file is not fully translated --- Key: CONNECTORS-373 URL: https://issues.apache.org/jira/browse/CONNECTORS-373 Project: ManifoldCF Issue Type: Improvement Components: Solr-4.x-component Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-373.patch Solr connector's japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-374) SharePoint connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitoshi Ozawa updated CONNECTORS-374: - Attachment: CONNECTORS-374.patch SharePoint connector's Japanese messages are not fully translated - Key: CONNECTORS-374 URL: https://issues.apache.org/jira/browse/CONNECTORS-374 Project: ManifoldCF Issue Type: Improvement Components: SharePoint connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-374.patch SharePoint connector's Japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CONNECTORS-375) RSS connector's Japanese messages are not fully translated
RSS connector's Japanese messages are not fully translated -- Key: CONNECTORS-375 URL: https://issues.apache.org/jira/browse/CONNECTORS-375 Project: ManifoldCF Issue Type: Improvement Components: RSS connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Priority: Minor Fix For: ManifoldCF 0.5 Should translated RSS connector's Japanese message properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-376) Meridio connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitoshi Ozawa updated CONNECTORS-376: - Attachment: CONNECTORS-376.patch Meridio connector's Japanese messages are not fully translated -- Key: CONNECTORS-376 URL: https://issues.apache.org/jira/browse/CONNECTORS-376 Project: ManifoldCF Issue Type: Improvement Components: Meridio connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-376.patch Should translate Meridio connector's Japanese message properties -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CONNECTORS-376) Meridio connector's Japanese messages are not fully translated
Meridio connector's Japanese messages are not fully translated -- Key: CONNECTORS-376 URL: https://issues.apache.org/jira/browse/CONNECTORS-376 Project: ManifoldCF Issue Type: Improvement Components: Meridio connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Priority: Minor Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-376.patch Should translate Meridio connector's Japanese message properties -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CONNECTORS-373) Solr connector Japanese message properties file is not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-373: -- Assignee: Karl Wright Solr connector Japanese message properties file is not fully translated --- Key: CONNECTORS-373 URL: https://issues.apache.org/jira/browse/CONNECTORS-373 Project: ManifoldCF Issue Type: Improvement Components: Solr-4.x-component Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-373.patch Solr connector's japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CONNECTORS-374) SharePoint connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-374: -- Assignee: Karl Wright SharePoint connector's Japanese messages are not fully translated - Key: CONNECTORS-374 URL: https://issues.apache.org/jira/browse/CONNECTORS-374 Project: ManifoldCF Issue Type: Improvement Components: SharePoint connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-374.patch SharePoint connector's Japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CONNECTORS-373) Solr connector Japanese message properties file is not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright updated CONNECTORS-373: --- Component/s: (was: Solr-4.x-component) Lucene/SOLR connector Solr connector Japanese message properties file is not fully translated --- Key: CONNECTORS-373 URL: https://issues.apache.org/jira/browse/CONNECTORS-373 Project: ManifoldCF Issue Type: Improvement Components: Lucene/SOLR connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-373.patch Solr connector's japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CONNECTORS-375) RSS connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-375: -- Assignee: Karl Wright RSS connector's Japanese messages are not fully translated -- Key: CONNECTORS-375 URL: https://issues.apache.org/jira/browse/CONNECTORS-375 Project: ManifoldCF Issue Type: Improvement Components: RSS connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-375.patch Should translated RSS connector's Japanese message properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CONNECTORS-376) Meridio connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright reassigned CONNECTORS-376: -- Assignee: Karl Wright Meridio connector's Japanese messages are not fully translated -- Key: CONNECTORS-376 URL: https://issues.apache.org/jira/browse/CONNECTORS-376 Project: ManifoldCF Issue Type: Improvement Components: Meridio connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-376.patch Should translate Meridio connector's Japanese message properties -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-373) Solr connector Japanese message properties file is not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184087#comment-13184087 ] Karl Wright commented on CONNECTORS-373: r1230052 Solr connector Japanese message properties file is not fully translated --- Key: CONNECTORS-373 URL: https://issues.apache.org/jira/browse/CONNECTORS-373 Project: ManifoldCF Issue Type: Improvement Components: Lucene/SOLR connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-373.patch Solr connector's japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-371) LiveLink connector should have Japanese message properties file
[ https://issues.apache.org/jira/browse/CONNECTORS-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184090#comment-13184090 ] Karl Wright commented on CONNECTORS-371: r1230059 to remove quotes in messages. LiveLink connector should have Japanese message properties file --- Key: CONNECTORS-371 URL: https://issues.apache.org/jira/browse/CONNECTORS-371 Project: ManifoldCF Issue Type: Improvement Components: LiveLink connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-371.patch, CONNECTORS-371.patch LiveLink connector's Japanese message properties file is not fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-374) SharePoint connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-374. Resolution: Fixed r1230059. I removed the quotation marks from both files since these are no longer needed. SharePoint connector's Japanese messages are not fully translated - Key: CONNECTORS-374 URL: https://issues.apache.org/jira/browse/CONNECTORS-374 Project: ManifoldCF Issue Type: Improvement Components: SharePoint connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-374.patch SharePoint connector's Japanese message properties file should be fully translated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CONNECTORS-377) Should standardize on src/main/resources in directories, Maven build, and ant build
Should standardize on src/main/resources in directories, Maven build, and ant build --- Key: CONNECTORS-377 URL: https://issues.apache.org/jira/browse/CONNECTORS-377 Project: ManifoldCF Issue Type: Bug Components: Build Reporter: Karl Wright We use src/main/resource sometimes and that is going to cause nothing but trouble. We need to fix this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-375) RSS connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-375. Resolution: Fixed r1230068. I removed the quotes as they are no longer needed, and restored another fix which was required to prevent the run-rss-UI-tests-derby ant target from failing. RSS connector's Japanese messages are not fully translated -- Key: CONNECTORS-375 URL: https://issues.apache.org/jira/browse/CONNECTORS-375 Project: ManifoldCF Issue Type: Improvement Components: RSS connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-375.patch Should translated RSS connector's Japanese message properties. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-376) Meridio connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184106#comment-13184106 ] Karl Wright commented on CONNECTORS-376: I'm still missing an internationalized version of the MeridioConnector.java and MeridioAuthority.java. Do you have those by any chance? Meridio connector's Japanese messages are not fully translated -- Key: CONNECTORS-376 URL: https://issues.apache.org/jira/browse/CONNECTORS-376 Project: ManifoldCF Issue Type: Improvement Components: Meridio connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-376.patch Should translate Meridio connector's Japanese message properties -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-333) Solr 3.x and 4.x plugins should use best practices in setting up http connections
[ https://issues.apache.org/jira/browse/CONNECTORS-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-333. Resolution: Fixed I was able to get the CloseHook API to be used for both Plugins. This does not kill the thread, which is bound statically to the multithreaded connection manager class. The only way to kill it is to call shutdownAll(), which could blow up other things in the same JVM. Solr 3.x and 4.x plugins should use best practices in setting up http connections - Key: CONNECTORS-333 URL: https://issues.apache.org/jira/browse/CONNECTORS-333 Project: ManifoldCF Issue Type: Bug Components: Solr-3.x-component, Solr-4.x-component Affects Versions: ManifoldCF 0.4 Reporter: Karl Wright Assignee: Karl Wright Priority: Critical Fix For: ManifoldCF 0.5 The Solr components need to use keep-alive in order to not accumulate handles in CLOSE_WAIT. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-377) Should standardize on src/main/resources in directories, Maven build, and ant build
[ https://issues.apache.org/jira/browse/CONNECTORS-377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-377. Resolution: Fixed Fix Version/s: ManifoldCF 0.5 r1230101 Should standardize on src/main/resources in directories, Maven build, and ant build --- Key: CONNECTORS-377 URL: https://issues.apache.org/jira/browse/CONNECTORS-377 Project: ManifoldCF Issue Type: Bug Components: Build Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 We use src/main/resource sometimes and that is going to cause nothing but trouble. We need to fix this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CONNECTORS-369) i18n/localization work for connectors often includes javascript quotes in the translated text
[ https://issues.apache.org/jira/browse/CONNECTORS-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wright resolved CONNECTORS-369. Resolution: Fixed I think I've got all of these cleaned up now. As long as new contributions don't reintroduce the problem, we're all set. i18n/localization work for connectors often includes javascript quotes in the translated text - Key: CONNECTORS-369 URL: https://issues.apache.org/jira/browse/CONNECTORS-369 Project: ManifoldCF Issue Type: Task Components: File system connector, FileNet connector, GTS connector, JCIFS connector, JDBC connector, LiveLink connector, Lucene/SOLR connector, RSS connector, Web connector, Wiki connector Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The i18n for many of the connectors includes quotation marks that should not be in the translation, but should instead be in the code. See CONNECTORS-356 for a detailed description of the problem and the solution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CONNECTORS-378) Need to modify site to include pointer(s) to ManifoldCF 0.4-incubating and plugins
Need to modify site to include pointer(s) to ManifoldCF 0.4-incubating and plugins -- Key: CONNECTORS-378 URL: https://issues.apache.org/jira/browse/CONNECTORS-378 Project: ManifoldCF Issue Type: Task Components: Documentation Affects Versions: ManifoldCF 0.5 Reporter: Karl Wright Assignee: Karl Wright Fix For: ManifoldCF 0.5 The site documentation needs to be updated to include a reference to the latest (0.4-incubating) release. Links to the plugin source distributions are also desirable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CONNECTORS-376) Meridio connector's Japanese messages are not fully translated
[ https://issues.apache.org/jira/browse/CONNECTORS-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13184673#comment-13184673 ] Karl Wright commented on CONNECTORS-376: r1230375 updates the message files, but without the quotations, since by convention we won't be putting them there. Meridio connector's Japanese messages are not fully translated -- Key: CONNECTORS-376 URL: https://issues.apache.org/jira/browse/CONNECTORS-376 Project: ManifoldCF Issue Type: Improvement Components: Meridio connector Affects Versions: ManifoldCF 0.5 Reporter: Hitoshi Ozawa Assignee: Karl Wright Priority: Minor Labels: I18N Fix For: ManifoldCF 0.5 Attachments: CONNECTORS-376.patch Should translate Meridio connector's Japanese message properties -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
