Re: GData GSoC application

2006-05-18 Thread Ian Holsman
google.com/soc/mentor_step1.html > and am awaiting approval. > > -Yonik > > > On 5/18/06, Yoav Shapira <[EMAIL PROTECTED]> wrote: > > Guys, > > There's an application active for the Solr GData server idea as part > > of the Google Summer of Code, but no one

Re: GData GSoC application

2006-05-18 Thread Yoav Shapira
st signed up to be a Mentor here > > http://code.google.com/soc/mentor_step1.html > > and am awaiting approval. > > > > -Yonik > > > > > > On 5/18/06, Yoav Shapira <[EMAIL PROTECTED]> wrote: > > > Guys, > > > There's an applicati

Re: GData GSoC application

2006-05-18 Thread Yonik Seeley
k, I just signed up to be a Mentor here > http://code.google.com/soc/mentor_step1.html > and am awaiting approval. > > -Yonik > > > On 5/18/06, Yoav Shapira <[EMAIL PROTECTED]> wrote: > > Guys, > > There's an application active for the Solr GData ser

Re: GData GSoC application

2006-05-18 Thread Yoav Shapira
sted: http://wiki.apache.org/general/SummerOfCode2006 Oh, ok, I just signed up to be a Mentor here http://code.google.com/soc/mentor_step1.html and am awaiting approval. -Yonik On 5/18/06, Yoav Shapira <[EMAIL PROTECTED]> wrote: > Guys, > There's an application active for the Sol

Re: GData GSoC application

2006-05-18 Thread Yonik Seeley
> wrote: Guys, There's an application active for the Solr GData server idea as part of the Google Summer of Code, but no one has volunteered to mentor it. Other than me, is anyone here signed up as a GSoC mentor, and if so are you willing to mentor this project? -- Yoav Shapira Nimalex LLC

Re: GData GSoC application

2006-05-18 Thread Erik Hatcher
I don't have the bandwidth available to mentor it, but its an effort I support strongly and would very likely jump on whatever got developed. Erik On May 18, 2006, at 10:14 AM, Yoav Shapira wrote: Guys, There's an application active for the Solr GData server idea as p

GData GSoC application

2006-05-18 Thread Yoav Shapira
Guys, There's an application active for the Solr GData server idea as part of the Google Summer of Code, but no one has volunteered to mentor it. Other than me, is anyone here signed up as a GSoC mentor, and if so are you willing to mentor this project? -- Yoav Shapira Nimalex LLC 1 Mi

Re: GData

2006-05-02 Thread Erik Hatcher
On Apr 26, 2006, at 9:41 AM, Grant Ingersoll wrote: Erik Hatcher <[EMAIL PROTECTED]> wrote: I've been mulling over the idea of having a single Solr instance morph into system that can handle multiple client-defined schemas (why not? Lucene itself can handle it) rather than a static XML file and

Re: GData

2006-04-27 Thread Grant Ingersoll
I may already have existing indexes that I just want to move into SOLR. I may, for biz reasons, want separate indexes b/c my app is managing access, but I don't want the overhead of multiple SOLR war installations. I can see having one SOLR install that I am sharing with others who want to ind

Re: GData, updateable IndexSearcher

2006-04-26 Thread Yonik Seeley
On 4/26/06, jason rutherglen <[EMAIL PROTECTED]> wrote: > As this would be used with Solr I wonder if there would be a good way to also > update the Solr caches. Other than re-executing queries that generated the results? Probably not. The nice thing about knowing exactly when the view of an inde

Re: GData, updateable IndexSearcher

2006-04-26 Thread jason rutherglen
ginal Message From: Doug Cutting <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org Sent: Wednesday, April 26, 2006 11:27:44 AM Subject: Re: GData, updateable IndexSearcher jason rutherglen wrote: > Interesting, does this mean there is a plan for incrementally updateable > IndexSearchers

Re: GData, updateable IndexSearcher

2006-04-26 Thread Doug Cutting
jason rutherglen wrote: Interesting, does this mean there is a plan for incrementally updateable IndexSearchers to become part of Lucene? In general, there is no plan for Lucene. If someone implements a generally useful, efficient, feature in a back-compatible, easy to use, manner, and subm

Re: GData, updateable IndexSearcher

2006-04-26 Thread jason rutherglen
there any negatives to updateable IndexSearchers? Thanks, Jason - Original Message From: Doug Cutting <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org Sent: Tuesday, April 25, 2006 9:04:47 PM Subject: Re: GData jason rutherglen wrote: > Ah ok

Re: GData

2006-04-26 Thread Yonik Seeley
> I think SOLR needs to be able to support multiple Lucene indexes per WAR > deployment as well Is this because single requests need to query across multiple indexes? Or do you have different document types that you don't want to stick in the same physical Lucene index? > With this idea, you coul

Re: GData

2006-04-26 Thread Grant Ingersoll
Erik Hatcher <[EMAIL PROTECTED]> wrote: I've been mulling over the idea of having a single Solr instance morph into system that can handle multiple client-defined schemas (why not? Lucene itself can handle it) rather than a static XML file and allow the schemas themselves to be retrievable

Re: GData

2006-04-25 Thread Doug Cutting
jason rutherglen wrote: Ah ok, think I found it: org.apache.nutch.indexer.FsDirectory no? Couldn't this be used in Solr and distribute all the data rather than master/slave it? It's possible to search a Lucene index that lives in Hadoop's DFS, but not recommended. It's very slow. It's much

Re: GData

2006-04-25 Thread jason rutherglen
4:10:36 PM Subject: Re: GData jason rutherglen wrote: > Is a faster method of loading or updating the IndexSearcher something that > makes sense for Lucene? Yes. Folks have developed incrementally updateable IndexSearchers before, but none is yet part of Lucene. > Or just assume the Go

Re: GData

2006-04-25 Thread jason rutherglen
essage From: Doug Cutting <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org Sent: Tuesday, April 25, 2006 4:10:36 PM Subject: Re: GData jason rutherglen wrote: > Is a faster method of loading or updating the IndexSearcher something that > makes sense for Lucene? Yes. Folks

Re: GData

2006-04-25 Thread Doug Cutting
jason rutherglen wrote: Is a faster method of loading or updating the IndexSearcher something that makes sense for Lucene? Yes. Folks have developed incrementally updateable IndexSearchers before, but none is yet part of Lucene. Or just assume the Google architecture is a lot more comple

Re: GData

2006-04-25 Thread jason rutherglen
makes sense for Lucene? Or just assume the Google architecture is a lot more complex. - Original Message From: Yonik Seeley <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org; jason rutherglen <[EMAIL PROTECTED]> Sent: Tuesday, April 25, 2006 3:21:07 PM Subject: Re: GData

Re: GData

2006-04-25 Thread Doug Cutting
Ian Holsman wrote: I noticed you guys have created a 'gdata-lucene' server in the SoC project. are you planning on doing this via SoLR? or is it something brand new? We decided that doing this via Solr would probably make it more complicated. A simple, standalone GData server

Re: GData

2006-04-25 Thread Yonik Seeley
On 4/25/06, jason rutherglen <[EMAIL PROTECTED]> wrote: > Ok, if Google is using the GData architecture to store the GCalendar data, > assuming they are, how long do you think a write takes to show up on the > GCalendar web site? I think in this case something other than

Re: GData

2006-04-25 Thread Ian Holsman
I noticed you guys have created a 'gdata-lucene' server in the SoC project. are you planning on doing this via SoLR? or is it something brand new? --i On 4/26/06, jason rutherglen <[EMAIL PROTECTED]> wrote: > Ok, if Google is using the GData architecture to store the GCalenda

Re: GData

2006-04-25 Thread jason rutherglen
Ok, if Google is using the GData architecture to store the GCalendar data, assuming they are, how long do you think a write takes to show up on the GCalendar web site? I think in this case something other than rsync may be a better option. - Original Message From: Yonik Seeley

Re: GData

2006-04-25 Thread Chris Hostetter
: I've been mulling over the idea of having a single Solr instance : morph into system that can handle multiple client-defined schemas : (why not? Lucene itself can handle it) rather than a static XML file : and allow the schemas themselves to be retrievable (yes, I know it : already is). I'm st

Re: GData

2006-04-25 Thread Erik Hatcher
tatic XML file and allow the schemas themselves to be retrievable (yes, I know it already is). I'm still talking about a single Lucene index, but with each Document given a "soup" name field and filters automatically available to single out a specific soup. Make sense? I think t

Re: GData

2006-04-25 Thread jason rutherglen
TED]> To: solr-dev@lucene.apache.org; jason rutherglen <[EMAIL PROTECTED]> Sent: Tuesday, April 25, 2006 12:42:58 PM Subject: Re: GData On 4/25/06, jason rutherglen <[EMAIL PROTECTED]> wrote: > Here is a good blog entry with a talk on GData from someone who worked on it. > The

Re: GData

2006-04-25 Thread Yonik Seeley
On 4/25/06, jason rutherglen <[EMAIL PROTECTED]> wrote: > Here is a good blog entry with a talk on GData from someone who worked on it. > The only thing I think Solr needs is faster replication, which perhaps can > be done faster using a direct replication model, preferably o

GData

2006-04-25 Thread jason rutherglen
http://jeremy.zawodny.com/blog/archives/006687.html Here is a good blog entry with a talk on GData from someone who worked on it. The only thing I think Solr needs is faster replication, which perhaps can be done faster using a direct replication model, preferably over HTTP of the segments

Re: GData

2006-04-20 Thread Yoav Shapira
Good. I'll be available on a time-permitting basis as always, but I don't want to commit as a mentor for this, so having you two makes me feel at ease ;) Yoav On 4/20/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: > On 4/20/06, Doug Cutting <[EMAIL PROTECTED]> wrote: > > Would you (or someone else)

Re: GData

2006-04-20 Thread Yonik Seeley
On 4/20/06, Doug Cutting <[EMAIL PROTECTED]> wrote: > Would you (or someone else) be willing to co-mentor this one with me? > I'm travelling the month of July, so I'm hesitant to be the sole mentor. > (I'll be online, but at reduced capacity.) > > If I have a co-mentor, then I'd be happy to write

Re: GData

2006-04-20 Thread Yonik Seeley
te, delete+add, or query. At first blush, that's the approach I would take with Solr too (a gdata specific Servlet that interfaced to Solr). So I don't see a big difference in difficulty level. It shouldn't be hard to take a straight lucene-servlet version and adapt it to Solr later, so

Re: GData

2006-04-20 Thread Doug Cutting
Yoav Shapira wrote: Yeah, and a cool one at that, +1. Would you (or someone else) be willing to co-mentor this one with me? I'm travelling the month of July, so I'm hesitant to be the sole mentor. (I'll be online, but at reduced capacity.) If I have a co-mentor, then I'd be happy to write

Re: GData

2006-04-20 Thread Yoav Shapira
Hola, > It might actually be a simpler project if it were standalone: not built > into Solr, but rather a Lucene contrib project. One only has to write a > few servlets that translate each requests into Lucene events: add, > delete, delete+add, or query. It wouldn't have lots of Solr's fancy > f

Re: GData

2006-04-20 Thread Doug Cutting
Yoav Shapira wrote: Getting back to Doug's original point about this as a possible SoC project: it seems a little too big from the technical discussion so far. It might actually be a simpler project if it were standalone: not built into Solr, but rather a Lucene contrib project. One only has

Re: GData

2006-04-20 Thread Yoav Shapira
> There is probably a lot one could do to tailor the scope of many > projects to fit (making many parts of the spec optional, etc). > > > Part of the SoC's goal is to motivate a student into being able > > to say they created something, and getting them to enjoy the > > open-source development proc

Re: GData

2006-04-20 Thread Yonik Seeley
On 4/20/06, Yoav Shapira <[EMAIL PROTECTED]> wrote: > Getting back to Doug's original point about this as a possible SoC > project: it seems a little too big from the technical discussion so > far. There is probably a lot one could do to tailor the scope of many projects to fit (making many parts

Re: GData

2006-04-20 Thread Yoav Shapira
One might think the Google GData system is agnostic to and works quite well with a distributed filesystem like Google's (http://labs.google.com/papers/gfs-sosp2003.pdf) Getting back to Doug's original point about this as a possible SoC project: it seems a little too big from the

Re: GData

2006-04-20 Thread jason rutherglen
Does this mean that the Google system does some sort of realtime replication? - Original Message From: Doug Cutting <[EMAIL PROTECTED]> To: solr-dev@lucene.apache.org Sent: Thursday, April 20, 2006 8:59:01 AM Subject: GData How hard would it be to build a GData server using Sol

Re: GData

2006-04-20 Thread Erik Hatcher
On Apr 20, 2006, at 1:22 PM, Chris Hostetter wrote: : The easiest way I can think of to get that effect is to store all the : fields so you can re-create the Document and change the field being : updated. My brief reading of hte GData URL Doug sent suggestes that the overall theme is

Re: GData

2006-04-20 Thread Erik Hatcher
This was mainly very wishful thinking on my part :) Sadly, I'm still far from an expert on the low-level Lucene internals, so I'm only waving my hands at a high-level here. Storing all fields is not a practical solution, at least in many situations. So the GData update is quit

Re: GData

2006-04-20 Thread Chris Hostetter
: The easiest way I can think of to get that effect is to store all the : fields so you can re-create the Document and change the field being : updated. My brief reading of hte GData URL Doug sent suggestes that the overall theme is content storage -- if that's the goal, mandating that &q

Re: GData

2006-04-20 Thread Yonik Seeley
On 4/20/06, Erik Hatcher <[EMAIL PROTECTED]> wrote: > So to turn it around to ask you a question, what would it take to > allow a Lucene document to be "updatable" at the field granularity, > such that no other fields need to be specified again? That sounds like quite a job in Lucene... one thing

Re: GData

2006-04-20 Thread Erik Hatcher
It would require some work to add this to Solr, but not a huge effort. One of the most crucial missing pieces that I'm beginning to feel a strong need for is being able to update a single field in a Lucene index. I notice the GData protocol supports this: <http://code.go

GData

2006-04-20 Thread Doug Cutting
How hard would it be to build a GData server using Solr? An open-source, Lucene-based GData server would be a good thing to have. Does this fit in Solr, or should it be a separate project? http://code.google.com/apis/gdata/overview.html Another summer of code project? Doug