[
https://issues.apache.org/jira/browse/CONNECTORS-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883923#action_12883923
]
Karl Wright commented on CONNECTORS-50:
---------------------------------------
I don't think much of "umbrella tickets". Each ticket should describe a
reasonably isolated feature or fix, not a wish list. Can you break this up
into more specific work items, being careful to check first whether there are
existing tickets covering the feature/service you are looking for?
I'm also still looking for much greater specificity as to the use cases. One
cannot design useful features without use cases. For example, the word "API"
is so unspecific as to be essentially meaningless. If you describe in detail
what your hoped-for interaction with this hypothetical "API" is, that would go
a long way towards clarifying the need. I'm not just interested in the API
format; I'm interested in how you intend to interact with it. This is crucial
because, as I've pointed out in various posts, one key design goal of LCF is to
make the connector developer provide the UI for their connector, and your
proposal may well force a violation of that principle, unless you have
something clever up your sleeve.
There are still a number of points in your document we have discussed in the
past which remain but whose controversy goes unacknowledged. It would be good,
if you create tickets or add to tickets already created, to mention the
associated issues and why you think they are unimportant or immaterial. For
example, I've discussed the limitations of using Derby as the prime database
for LCF - that should be captured somewhere.
> Proposal for initial two releases of LCF, including packaged product and full
> API
> ---------------------------------------------------------------------------------
>
> Key: CONNECTORS-50
> URL: https://issues.apache.org/jira/browse/CONNECTORS-50
> Project: Lucene Connector Framework
> Issue Type: New Feature
> Components: Framework core
> Reporter: Jack Krupansky
> Original Estimate: 3360h
> Remaining Estimate: 3360h
>
> Currently, LCF has a relatively high-bar for evaluation and use, requiring
> developer expertise. Also, although LCF has a comprehensive UI, it is not
> currently packaged for use as a crawling engine for advanced applications.
> A small set of individual feature requests are needed to address these
> issues. They are summarized briefly to show how they fit together for two
> initial releases of LCF, but will be broken out into individual LCF Jira
> issues.
> Goals:
> 1. LCF as a standalone, downloadable, usable-out-of-the-box product (much as
> Solr is today)
> 2. LCF as a toolkit for developers needing customized crawling and repository
> access
> 3. An API-based crawling engine that can be integrated with applications (as
> Aperture is today)
> Larger goals:
> 1. Make it very easy for users to evaluate LCF.
> 2. Make it very easy for developers to customize LCF.
> 3. Make it very easy for appplications to fully manage and control LCF in
> operation.
> Two phases:
> 1) Standalone, packaged app that is super-easy to evaluate and deploy. Call
> it LCF 0.5.
> 2) API-based crawling engine for applications for which the UI might not be
> appropriate. Call it LCF 1.0.
> Phase 1
> -------
> LCF 0.5 right out of the box would interface loosely with Solr 1.4 or later.
> It would contain roughly the features that are currently in place or
> currently underway, plus a little more.
> Specifically, LCF 0.5 would contain these additional capabilities:
> 1. Plug-in architecture for connectors (already underway)
> 2. Packaged app ready to run with embedded Jetty app server (I think this has
> been agreed to)
> 3. Bundled with database - PostgreSQL or derby - ready to run without
> additional manual setup
> 4. Mini-API to initially configure default connections and "example" jobs for
> file system and web crawl
> 5. Agent process started automatically (platform-specific startup required)
> 6. Solr output connector option to commit at end of job, by default
> Installation and basic evaluation of LCF would be essentially as simple as
> Solr is today. The example
> connections and jobs would permit the user to initiate example crawls of a
> file system example
> directory and an example web on the LCF web site with just a couple of clicks
> (as opposed to the
> detailed manual setup required today to create repository and output
> connections and jobs.
> It is worth considering whether the SharePoint connector could also be
> included as part of the default package.
> Users could then add additional connectors and repositories and jobs as
> desired.
> Timeframe for release? Level of effort?
> Phase 2
> -------
> The essence of Phase 2 is that LCF would be split to allow direct, full API
> access to LCF as a
> crawling "engine", in additional to the full LCF UI. Call this LCF 1.0.
> Specifically, LCF 1.0 would contain these additional capabilities:
> 1. Full API for LCF as a crawling engine
> 2. LCF can be bundled within an app (such as the default LCF package itself
> with its UI)
> 3. LCF event and activity notification for full control by an application
> (already a Jira request)
> Overall, LCF will offer roughly the same crawling capabilities as with LCF
> 0.5, plus whatever bug
> fixes and minor enhancements might also be added.
> Timeframe for release? Level of effort?
> -------------------------
> Issues:
> - Can we package PostgreSQL with LCF so LCF can set it up?
> - Or do we need Derby for that purpose?
> - Managing multiple processes (UI, database, agent, app processes)
> - What exactly would the API look like? (URL, XML, JSON, YAML?)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
