Is this required by your application? Is there any way to reduce the number of terms?
A work around is to use shards. If your terms follow Zipf's Law each shard will have fewer than the complete number of terms. For N shards, each shard will have ~1/N of the singleton terms. For 2-count terms, 1/N or 2/N will have that term. Now I'm interested but not mathematically capable: what is the general probabilistic formula for splitting Zipf's Law across shards? On Mon, Aug 20, 2012 at 3:51 PM, Jack Krupansky <j...@basetechnology.com> wrote: > It appears that there is a hard limit of 24-bits or 16M for the number of > bytes to reference the terms in a single field of a single document. It > takes 1, 2, 3, 4, or 5 bytes to reference a term. If it took 4 bytes, that > would allow 16/4 or 4 million unique terms - per document. Do you have such > large documents? This appears to be a hard limit based of 24-bytes in a Java > int. > > You can try facet.method=enum, but that may be too slow. > > What release of Solr are you running? > > -- Jack Krupansky > > -----Original Message----- From: Fuad Efendi > Sent: Monday, August 20, 2012 4:34 PM > To: Solr-User@lucene.apache.org > Subject: UnInvertedField limitations > > > Hi All, > > > I have a problemŠ (Yonik, please!) help me, what is Term count limits? I > possibly have 256,000,000 different terms in a fieldŠ or 16,000,000? > > Thanks! > > > 2012-08-20 16:20:19,262 ERROR [solr.core.SolrCore] - [pool-1-thread-1] - : > org.apache.solr.common.SolrException: Too many values for UnInvertedField > faceting on field enrich_keywords_string_mv > at > org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:179) > at > org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.j > ava:668) > at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:326) > at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:4 > 23) > at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:206) > at > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java > :85) > at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand > ler.java:204) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase. > java:129) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1561) > > > > > -- > Fuad Efendi > http://www.tokenizer.ca > > > -- Lance Norskog goks...@gmail.com
