ng - the distributed ranking scheme very rarely
differed from that of the single large index.
- Original Message
From: Otis Gospodnetic <[EMAIL PROTECTED]>
To: java-user@lucene.apache.org
Sent: Friday, 2 May, 2008 1:35:04 AM
Subject: Re: Does Lucene Supports Billions of data
Right.
not be comparable.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
> From: Toke Eskildsen <[EMAIL PROTECTED]>
> To: java-user@lucene.apache.org
> Sent: Friday, May 2, 2008 12:13:04 AM
> Subject: Re: Does Lucene Supports Billions of d
From: John Wang <[EMAIL PROTECTED]>
[...]
> sub index 1: 1 billion docs
> sub index 2: 1 billion docs
> sub index 3: 1 billion docs
>
> federating search to these subindexes, you represent an index of 3
> billiondocs, and all internal doc ids are of type int.
That falls under Daniel's "...unless
I am not sure why this is the case, docid is internal to the sub index. As
long as the sub index size is below 2 bil, there is no need for docid to be
long. With multiple indexes, I was thinking having an aggregater which
merges maybe only a page of search result.
Example:
sub index 1: 1 billion
> Even if they're in multiple indexes, the doc IDs being ints
> will still prevent
> it going past 2Gi unless you wrap your own framework around it.
Hm. Does this mean that a MultiReader has the int-limit too?
I thought that this limit applies to a single index only...
On Wed, Apr 30, 2008 at 7:10 PM, Daniel Noll <[EMAIL PROTECTED]> wrote:
> On Thursday 01 May 2008 00:01:48 John Wang wrote:
> > I am not sure how well lucene would perform with > 2 Billion docs in a
> > single index anyway.
>
> Even if they're in multiple indexes, the doc IDs being ints will sti
On Thursday 01 May 2008 00:01:48 John Wang wrote:
> I am not sure how well lucene would perform with > 2 Billion docs in a
> single index anyway.
Even if they're in multiple indexes, the doc IDs being ints will still prevent
it going past 2Gi unless you wrap your own framework around it.
Daniel
; >
> > -Glen
> >
> > 2008/4/30 John Wang <[EMAIL PROTECTED]>:
> > > lucene docids are represented in a java int, so max signed int would be
> > the
> > > limit, a little over 2 billion.
> > >
> > > -John
> > >
&
> > limit, a little over 2 billion.
> >
> > -John
> >
> >
> >
> > On Wed, Apr 30, 2008 at 11:54 AM, Sebastin <[EMAIL PROTECTED]>
> wrote:
> >
> > >
> > > Hi All,
> > > Does Lucene supports Billions of data in
; limit, a little over 2 billion.
>
> -John
>
>
>
> On Wed, Apr 30, 2008 at 11:54 AM, Sebastin <[EMAIL PROTECTED]> wrote:
>
> >
> > Hi All,
> > Does Lucene supports Billions of data in a single index store of size 14
> > GB
> > for eve
lucene docids are represented in a java int, so max signed int would be the
limit, a little over 2 billion.
-John
On Wed, Apr 30, 2008 at 11:54 AM, Sebastin <[EMAIL PROTECTED]> wrote:
>
> Hi All,
> Does Lucene supports Billions of data in a single index store of size 14
&
Hi All,
Does Lucene supports Billions of data in a single index store of size 14 GB
for every search.I have 3 Index Store of size 14 GB per index i need to
search these index store and retreive the result.it throws out of memory
problem while searching this index stores.
--
View this message in
12 matches
Mail list logo
