Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Ryan McKinley
On 1/16/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : >I left out "micro-plugins" because i don't quite have a good answer : >yet :) This may a place where a custom dispatcher servlet/filter : >defined in web.xml is the most appropriate solution. : : If the issue is munging HTTPServletReques

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Ryan McKinley
kind of like a binary stream equivilent to the way analyzers can be customized -- is thta kind of what you had in mind? exactly. interface SolrDocumentParser { public init(NamedList args); Document parse(SolrParams p, ContentStream content); } yes

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Chris Hostetter
: > - Revise the XML-based update code (broken out of SolrCore into a : > RequestHandler) to use all the above. : : +++1, that's been needed forever. yeah ... once we have a RequestHandler doing that work, and populating a SolrQueryResponse with it's result info, it would probably be pretty trivi

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Erik Hatcher
On Jan 17, 2007, at 1:41 AM, Chris Hostetter wrote: : The number of people writing update plugins will be small compared to : the number of users using the external HTTP API (the URL + query : parameters, and the relationship URL-wise between different update : formats). My main concern is ma

[jira] Commented: (SOLR-112) Hierarchical Handler Config

2007-01-16 Thread Ryan McKinley (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465380 ] Ryan McKinley commented on SOLR-112: I attached a simple implementaion. This includes the NamedList changes from

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Chris Hostetter
: >I left out "micro-plugins" because i don't quite have a good answer : >yet :) This may a place where a custom dispatcher servlet/filter : >defined in web.xml is the most appropriate solution. : : If the issue is munging HTTPServletRequest information, then a proper : separation of concerns sug

[jira] Updated: (SOLR-112) Hierarchical Handler Config

2007-01-16 Thread Ryan McKinley (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-112: --- Attachment: SOLR-112.patch > Hierarchical Handler Config > --- > >

[jira] Created: (SOLR-112) Hierarchical Handler Config

2007-01-16 Thread Ryan McKinley (JIRA)
Hierarchical Handler Config --- Key: SOLR-112 URL: https://issues.apache.org/jira/browse/SOLR-112 Project: Solr Issue Type: Improvement Components: update Affects Versions: 1.2 Reporter: Ryan

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Chris Hostetter
: > : In addition to RequestProcessors, maybe there should be a general : > : DocumentProcessor : > : interface SolrDocumentParser : > : { : > : Document parse(ContentStream content); : > : } : > what else would the RequestProcessor do if it was delegating all of the : > parsing to something e

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Chris Hostetter
: > So to understand better: : > : > user request -> micro-plugin -> RequestHandler -> ResponseHandler : or: : : HttpServletRequest -> SolrRequestParser -> SolrRequestProcessor -> : SolrResponse -> SolrResponseWriter specifically what i had in mind was something like this... class SolrUberSe

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Chris Hostetter
: The number of people writing update plugins will be small compared to : the number of users using the external HTTP API (the URL + query : parameters, and the relationship URL-wise between different update : formats). My main concern is making *that* as nice and utilitarian as : possible, and a

Re: Java version for solr development (was Re: Update Plugins)

2007-01-16 Thread Bertrand Delacretaz
On 1/17/07, Thorsten Scherler <[EMAIL PROTECTED]> wrote: ...Should I use 1.6 for a patch or above mentioned libs?... IMHO moving to 1.6 is way too soon, and if it's only to save two jars it's not worth it. -Bertrand

Re: Java version for solr development (was Re: Update Plugins)

2007-01-16 Thread Walter Underwood
On 1/16/07 8:03 PM, "Yonik Seeley" <[EMAIL PROTECTED]> wrote: > I think it's a bit soon to move to 1.6 - I don't know how many > platforms it's available for yet. It is still in "early release" from IBM for their PowerPC servers, so requiring 1.6 would be a serious problem for us. wunder -- Wal

[jira] Commented: (SOLR-107) Iterable NamedList with java5 generics

2007-01-16 Thread Ryan McKinley (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465357 ] Ryan McKinley commented on SOLR-107: updated patch for 1,2, and 3 > Iterable NamedList with java5 generics >

[jira] Updated: (SOLR-107) Iterable NamedList with java5 generics

2007-01-16 Thread Ryan McKinley (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley updated SOLR-107: --- Attachment: IterableNamedList.patch > Iterable NamedList with java5 generics >

Re: Java version for solr development (was Re: Update Plugins)

2007-01-16 Thread Yonik Seeley
On 1/16/07, Thorsten Scherler <[EMAIL PROTECTED]> wrote: I am on 1.5 ATM and using |-- stax-1.2.0-dev.jar `-- stax-utils.jar I don't know where those jars are from, but I guess one would need the stax API jar, and the implementation (woodstox I would think) jar. That's two jars instead of one,

Re: results while using SolrClient..

2007-01-16 Thread Chris Hostetter
First off, this question is probably best directed to the solr-user list ... solr-dev is for discussions about the development of Solr internals, solr-user is for questions baout using Solr. Second: the SolrClient code is still in active development, it may be buggy, and the API may changed. Thi

[jira] Commented: (SOLR-86) [PATCH] standalone updater cli based on httpClient

2007-01-16 Thread Thorsten Scherler (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465327 ] Thorsten Scherler commented on SOLR-86: --- Yeah, I know what you mean (had a similar problem today). if (!file.isD

Java version for solr development (was Re: Update Plugins)

2007-01-16 Thread Thorsten Scherler
On Tue, 2007-01-16 at 15:49 -0500, Yonik Seeley wrote: > On 1/16/07, J.J. Larrea <[EMAIL PROTECTED]> wrote: > > - Revise the XML-based update code (broken out of SolrCore into a > > RequestHandler) to use all the above. > > +++1, that's been needed forever. > If one has the time, I'd also advocat

[jira] Resolved: (SOLR-111) new repsonse classes and connection enhancements

2007-01-16 Thread Erik Hatcher (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Hatcher resolved SOLR-111. --- Resolution: Fixed Assignee: Erik Hatcher Applied, except tweaked autocommit to off by default. G

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Yonik Seeley
On 1/16/07, J.J. Larrea <[EMAIL PROTECTED]> wrote: - Revise the XML-based update code (broken out of SolrCore into a RequestHandler) to use all the above. +++1, that's been needed forever. If one has the time, I'd also advocate moving to StAX (via woodstox for Java5, but it's built into Java6)

Re: Can this be achieved? (Was: document support for file system crawling)

2007-01-16 Thread Eivind Hasle Amundsen
(...) http://andaluciajunta.es/aj-sea-.html This search engine will be based on nutch in the next version. The special character is that this main portal search engine has to search against the solr BOJA based indexed. Meaning Nutch will have to search the solr index and not vice versa. Looks in

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Yonik Seeley
On 1/16/07, J.J. Larrea <[EMAIL PROTECTED]> wrote: >POST: > if( multipart ) { > read all form fields into parameter map. This should use the same req.getParameterMap as for GET, which Servlet 2.4 says is suppose to be automatically by the servlet container if the payload is application/x-www-

Re: svn patches and directories...

2007-01-16 Thread Ryan McKinley
On 1/16/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : Date: Sat, 13 Jan 2007 19:12:27 -0800 (PST) : Subject: [jira] Commented: (SOLR-104) SQL Upload Plugin : 2) download HandlerRefactoring.DRAFT.zip and extract the contents to: : \solr\src\java\org\apache\solr\handler : : (svn patches do

[jira] Updated: (SOLR-111) new repsonse classes and connection enhancements

2007-01-16 Thread Ed Summers (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-111: Attachment: response_connection_changes.diff > new repsonse classes and connection enhancements > ---

[jira] Created: (SOLR-111) new repsonse classes and connection enhancements

2007-01-16 Thread Ed Summers (JIRA)
new repsonse classes and connection enhancements Key: SOLR-111 URL: https://issues.apache.org/jira/browse/SOLR-111 Project: Solr Issue Type: Improvement Components: clients - ruby -

[jira] Updated: (SOLR-111) new repsonse classes and connection enhancements

2007-01-16 Thread Ed Summers (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-111: Attachment: response_connection_changes.diff > new repsonse classes and connection enhancements > ---

[jira] Commented: (SOLR-86) [PATCH] standalone updater cli based on httpClient

2007-01-16 Thread Hoss Man (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465276 ] Hoss Man commented on SOLR-86: -- regarding Bertrand's comment, i'm not sure if there is any benefit in having this code and

Re: Merging Results from Multiple Solr Instances

2007-01-16 Thread Chris Hostetter
1) please don't reply to another thread with a new subject that is unrelated ... it makes following threads in mail readers and mailing list archives difficult. 2) the mailing list archives have some recent discussion on this you should look at... http://www.nabble.com/forum/Search.jtp?forum=144

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Yonik Seeley
On 1/15/07, Chris Hostetter <[EMAIL PROTECTED]> wrote: : The most important issue is to nail down the external HTTP interface. I'm not sure if i agree with that statement .. i would think that figuring out the "model" or how updates should be handled in a generic way, what all of the "Plugin" ty

svn patches and directories...

2007-01-16 Thread Chris Hostetter
: Date: Sat, 13 Jan 2007 19:12:27 -0800 (PST) : Subject: [jira] Commented: (SOLR-104) SQL Upload Plugin : 2) download HandlerRefactoring.DRAFT.zip and extract the contents to: : \solr\src\java\org\apache\solr\handler : : (svn patches don' t let you add new directories!) that's shouldn't be t

Re: [jira] Updated: (SOLR-20) A simple Java client for updating and searching

2007-01-16 Thread Chris Hostetter
: Do you think the package name should change? : : org.apache.solr.client.SolrClient : vs : org.apache.solr.client.solrj.SolrClient; yeah ... that way if down the road someone comes up with a radically new Java API (that we can't even fathom yet) they can call it "solrad" and put it in clients/ja

[jira] Commented: (SOLR-106) new facet params: facet.sort, facet.mincount, facet.offset

2007-01-16 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465254 ] Yonik Seeley commented on SOLR-106: --- Thanks for the info JJ... didn't see your update untill after I committed this

Re: solr.solr.home- what I have to do?

2007-01-16 Thread Chris Hostetter
: "Solr now looks in ./solr/conf for config, ./solr/data for data : configurable via solr.solr.home system property..." : ?? : : Is "system property" is really tag in web.xml file? : Or I have to define sime environment var with name "solr.solr.home"? it's a system property that can be

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread J.J. Larrea
I'm in frantic deadline mode so I'm just going to throw in some (hopefully) short comments... At 11:02 PM -0800 1/15/07, Ryan McKinley wrote: >>the one thing that still seems missing is those "micro-plugins" i was >> [SNIP] >> >> interface SolrRequestParser { >> SolrRequest process( HttpServ

[jira] Resolved: (SOLR-106) new facet params: facet.sort, facet.mincount, facet.offset

2007-01-16 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley resolved SOLR-106. --- Resolution: Fixed committed, with tests and other changes. I'll work on updating docs shortly. > new f

Re: Can this be achieved? (Was: document support for file system crawling)

2007-01-16 Thread Thorsten Scherler
On Tue, 2007-01-16 at 16:28 +0100, Eivind Hasle Amundsen wrote: > First: Please pardon the cross-post to solr-user for reference. I hope > to continue this thread in solr-dev. Please answer to solr-dev. > > > 1) more documentation (and posisbly some locking configuration options) on > > how you c

Re: To Spring or not to Spring? (was: Update Plugins)

2007-01-16 Thread Bertrand Delacretaz
On 1/16/07, Alan Burlison <[EMAIL PROTECTED]> wrote: ...I've had *bad* experiences with apps where people pulled in just about every framework, component and widget you can think of... That's what you previous message seemed to imply ;-) I agree that, if we start using Spring (or another) IoC

Re: To Spring or not to Spring? (was: Update Plugins)

2007-01-16 Thread Alan Burlison
Bertrand Delacretaz wrote: Using just the IoC container? I'm not talking about full-blown Spring magic, *just* IoC to assemble plugins. Spring's IoC is not complicated, and logging statements and debuggers are here to find out exactly what's happening if needed. I don't think it'd be more comp

To Spring or not to Spring? (was: Update Plugins)

2007-01-16 Thread Bertrand Delacretaz
On 1/16/07, Alan Burlison <[EMAIL PROTECTED]> wrote: Bertrand Delacretaz wrote: .../me can't help > thinking that this would be a good time to introduce the Spring IoC > container to manage this stuff... Please, no. I work on a big webapp that uses spring - it's a complete nightmare to figure

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Alan Burlison
Bertrand Delacretaz wrote: With all this talk about plugins, registries etc., /me can't help thinking that this would be a good time to introduce the Spring IoC container to manage this stuff. More info at http://www.springframework.org/docs/reference/beans.html for people who are not familiar

Merging Results from Multiple Solr Instances

2007-01-16 Thread Sangraal Aiken
I have three instances of Solr on a single machine that I would like to query as if they were a single instance. I was wondering if there's a facility, or if anyone has any recommendations, for searching across multiple instances with a single query, or merging the results of multiple insta

Re: Connectors, Parsers, Plugin architecture

2007-01-16 Thread Eivind Hasle Amundsen
: Solr aims at being an answer to "enterprise needs", by indexing : structured data for different applications. However I think that many : enterprises would like to be able to structure information themselves. thta's exactly what Solr is about: letting a schema creator define what the structure

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Alan Burlison
Yonik Seeley wrote: Brainstorming: - for errors, use HTTP error codes instead of putting it in the XML as now. That doesn't work so well if there are multiple documents to be indexed in a single request. -- Alan Burlison --

solr.solr.home- what I have to do?

2007-01-16 Thread Yury A. Buharkin
Hello! I'm a novice in Lucene technologies, and now only trying to install Solr. The main problem is appeared because I have to use Sun Java(bla-bla) web server as servlet contaner. So: Who can explain me what means phrase in solr's docs- "Solr now looks in ./solr/conf for config, ./solr/

Can this be achieved? (Was: document support for file system crawling)

2007-01-16 Thread Eivind Hasle Amundsen
First: Please pardon the cross-post to solr-user for reference. I hope to continue this thread in solr-dev. Please answer to solr-dev. 1) more documentation (and posisbly some locking configuration options) on how you can use Solr to access an index generated by the nutch crawler (i think Thors

Re: SOLR-67 query interface

2007-01-16 Thread Bertrand Delacretaz
On 1/16/07, rlawson <[EMAIL PROTECTED]> wrote: ...I'm new to SOLR and would like to contribute. I think my skills would best lend themselves to helping with a nice query interface. I'm a java web dev by profession... If you mean graphic design of the admin webpages, there are two issues about

SOLR-67 query interface

2007-01-16 Thread rlawson
I'm new to SOLR and would like to contribute. I think my skills would best lend themselves to helping with a nice query interface. I'm a java web dev by profession (couple of the sites/companies I have worked with are below) www.ptplace.com www.colinx.com www.getlocalbiz.com www.kemperinvestors.c

Re: client/ruby/flare/vender and svn:externals

2007-01-16 Thread Erik Hatcher
Hoss, Sorry 'bout that. You're right, Rails is too large to inject in like this. I've removed the svn:externals setting, and added an ignore for vendor/rails. If folks want to work on the Rails edge as I do, do this: rake rails:freeze:edge It's already a built-in Rakefile feat

Re: query to pull all document ids in an index

2007-01-16 Thread Bertrand Delacretaz
On 1/16/07, Edward Summers <[EMAIL PROTECTED]> wrote: I was thinking I could do a query that pulls back all the document ids in the index, and then delete each one... The "delete by query" feature will do this without requiring an iteration on the client side, see http://incubator.apache.o

query to pull all document ids in an index

2007-01-16 Thread Edward Summers
I'm new to solr (working on solrb with Erik). We have some functional tests that run against a live solr instance, and I'd like the tests to periodically remove all the documents from the index. This way tests will have a predictable outcome that is independent on the state of the index bef

[jira] Commented: (SOLR-69) PATCH:MoreLikeThis support

2007-01-16 Thread Bertrand Delacretaz (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465114 ] Bertrand Delacretaz commented on SOLR-69: - The method used to compute includeScore in MoreLikeThisHelper was inc

[jira] Updated: (SOLR-69) PATCH:MoreLikeThis support

2007-01-16 Thread Bertrand Delacretaz (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertrand Delacretaz updated SOLR-69: Attachment: SOLR-69.patch > PATCH:MoreLikeThis support > -- > >

[jira] Created: (SOLR-110) Factor out common code in our SolrRequestHandler classes

2007-01-16 Thread Bertrand Delacretaz (JIRA)
Factor out common code in our SolrRequestHandler classes Key: SOLR-110 URL: https://issues.apache.org/jira/browse/SOLR-110 Project: Solr Issue Type: Improvement Components: s

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Erik Hatcher
On Jan 16, 2007, at 3:20 AM, Bertrand Delacretaz wrote: On 1/16/07, Ryan McKinley <[EMAIL PROTECTED]> wrote: ...I think a DocumentParser registry is a good way to isolate this top level task... With all this talk about plugins, registries etc., /me can't help thinking that this would be a g

[jira] Commented: (SOLR-69) PATCH:MoreLikeThis support

2007-01-16 Thread Bertrand Delacretaz (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465092 ] Bertrand Delacretaz commented on SOLR-69: - SOLR-69.patch updated > PATCH:MoreLikeThis support > ---

[jira] Commented: (SOLR-69) PATCH:MoreLikeThis support

2007-01-16 Thread Bertrand Delacretaz (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465090 ] Bertrand Delacretaz commented on SOLR-69: - Thanks Ryan for spotting the fl param problem, I'll attach a revised

[jira] Updated: (SOLR-69) PATCH:MoreLikeThis support

2007-01-16 Thread Bertrand Delacretaz (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-69?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bertrand Delacretaz updated SOLR-69: Attachment: SOLR-69.patch > PATCH:MoreLikeThis support > -- > >

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Ryan McKinley
So to understand better: user request -> micro-plugin -> RequestHandler -> ResponseHandler Right? or: HttpServletRequest -> SolrRequestParser -> SolrRequestProcessor -> SolrResponse -> SolrResponseWriter

[jira] Commented: (SOLR-69) PATCH:MoreLikeThis support

2007-01-16 Thread Bertrand Delacretaz (JIRA)
[ https://issues.apache.org/jira/browse/SOLR-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465087 ] Bertrand Delacretaz commented on SOLR-69: - Yonik, you're right about the mindf parameter duplication, here's the

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Thorsten Scherler
On Mon, 2007-01-15 at 12:23 -0800, Chris Hostetter wrote: > : > Right, you're getting at issues of why I haven't committed my CSV handler > yet. > : > It currently handles reading a local file (this is more like an SQL > : > update handler... only a reference to the data is passed). But I also >

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Bertrand Delacretaz
On 1/16/07, Ryan McKinley <[EMAIL PROTECTED]> wrote: ...I think a DocumentParser registry is a good way to isolate this top level task... With all this talk about plugins, registries etc., /me can't help thinking that this would be a good time to introduce the Spring IoC container to manage t

Re: Update Plugins (was Re: Handling disparate data sources in Solr)

2007-01-16 Thread Ryan McKinley
: In addition to RequestProcessors, maybe there should be a general : DocumentProcessor : : interface SolrDocumentParser : { : Document parse(ContentStream content); : } : : solrconfig could register "text/html" -> HtmlDocumentParser, and : RequestProcessors could share the same parser. what e