Re: multiple cache for same field
It does not seem to be memory footprint also ? looks too high for my index. ./zahoor On 20-May-2013, at 10:55 PM, Jason Hellman jhell...@innoventsolutions.com wrote: Most definitely not the number of unique elements in each segment. My 32 document sample index (built from the default example docs data) has the following: entry#0: 'StandardDirectoryReader(segments_b:29 _8(4.2.1):C32)'='manu_exact',class org.apache.lucene.index.SortedDocValues,0.5=org.apache.lucene.search.FieldCacheImpl$SortedDocValuesImpl#1778857102 There is no chance for there to be 1.8 billion unique elements in that index. On May 20, 2013, at 1:20 PM, Erick Erickson erickerick...@gmail.com wrote: Not sure, never had to worry about what they are.. On Mon, May 20, 2013 at 12:28 PM, J Mohamed Zahoor zah...@indix.com wrote: What is the number at the end? is it the no of unique elements in each segment? ./zahoor On 20-May-2013, at 7:37 PM, Erick Erickson erickerick...@gmail.com wrote: Because the same field is split amongst a number of segments. If you look in the index directory, you should see files like _3fgm.* and _3ffm.*. Each such group represents one segment. The number of segments changes with merging etc. Best Erick On Mon, May 20, 2013 at 6:43 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi Why is that lucene field cache has multiple entries for the same field S_24. It is a dynamic field. 'SegmentCoreReader(owner=_3fgm(4.2.1):C7681)'='S_24',double,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_DOUBLE_PARSER=org.apache.lucene.search.FieldCacheImpl$DoublesFromArray#1174240382 'SegmentCoreReader(owner=_3ffm(4.2.1):C1596758)'='S_24',double,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_DOUBLE_PARSER=org.apache.lucene.search.FieldCacheImpl$DoublesFromArray#83384344 'SegmentCoreReader(owner=_3fgh(4.2.1):C2301)'='S_24',double,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_DOUBLE_PARSER=org.apache.lucene.search.FieldCacheImpl$DoublesFromArray#1281331764 Also, the number at the end.. does it specified the no of entries in that cache bucket? ./zahoor
HyperLogLog for Solr
Hi Has anyone tried using HLL for using finding unique values of a field in solr? I am planning to use them to facet count on certain fields to reduce memory footprint. ./Zahoor
Avoid loading Lucene's field cache for certain fields
Hi I am trying to avoid loading some fields in Lucene's FieldCache. Is there a way to avoid loading certain failed in Lucene's FieldCache. One way is to declare them multivalued.. Is there any other way? ./zahoor
Re: Avoid loading Lucene's field cache for certain fields
I am using Solr 4.2.1 ./zahoor On 20-May-2013, at 11:48 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi I am trying to avoid loading some fields in Lucene's FieldCache. Is there a way to avoid loading certain failed in Lucene's FieldCache. One way is to declare them multivalued.. Is there any other way? ./zahoor
cache disable through solrJ
Hi
How do i disable cache (Solr FieldValueCache) for certain queries...
using HTTP it can be done using {!cache=false}...
how can i do it from solrj?
./zahoor
Re: multiple cache for same field
What is the number at the end? is it the no of unique elements in each segment? ./zahoor On 20-May-2013, at 7:37 PM, Erick Erickson erickerick...@gmail.com wrote: Because the same field is split amongst a number of segments. If you look in the index directory, you should see files like _3fgm.* and _3ffm.*. Each such group represents one segment. The number of segments changes with merging etc. Best Erick On Mon, May 20, 2013 at 6:43 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi Why is that lucene field cache has multiple entries for the same field S_24. It is a dynamic field. 'SegmentCoreReader(owner=_3fgm(4.2.1):C7681)'='S_24',double,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_DOUBLE_PARSER=org.apache.lucene.search.FieldCacheImpl$DoublesFromArray#1174240382 'SegmentCoreReader(owner=_3ffm(4.2.1):C1596758)'='S_24',double,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_DOUBLE_PARSER=org.apache.lucene.search.FieldCacheImpl$DoublesFromArray#83384344 'SegmentCoreReader(owner=_3fgh(4.2.1):C2301)'='S_24',double,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_DOUBLE_PARSER=org.apache.lucene.search.FieldCacheImpl$DoublesFromArray#1281331764 Also, the number at the end.. does it specified the no of entries in that cache bucket? ./zahoor
Best query method
Hi I am using solr 4.2.1. My index has products from different stores with different attributes. If i want to get the count of all products which belongs to store X which is coloured red and is in-stock… My question is : Which way of querying is better in-terms of performance and cache usage. 1) q=*.*fq=(store:X) AND (colour:red) AND (in-stock:true) 2) q=store:Xfq=(colour:red) AND (in-stock:true) 3) q=store:Xfq=colour:redfq:in-stock:true f there is any other option better than these three.. please add let me know.. i am assuming that which ever filter eliminates more products… should come first (q, then list of fq's) ./zahoor
Re: Java heap space exception in 4.2.1
aah… was doing a facet on a double field which was having 6 decimal places… No surprise that the lucene cache got full… .z/ahoor On 17-May-2013, at 11:56 PM, J Mohamed Zahoor zah...@indix.com wrote: Memory increase a lot with queries which have facets… ./Zahoor On 17-May-2013, at 10:00 PM, Shawn Heisey s...@elyograg.org wrote: On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote: I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query.. Now i am getting this error frequently that solr does not run even for 2 minutes continuously. All 5GB is getting used instantaneously in few queries... Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It wasn't OOM errors, they were just using a lot more heap than they were before and running into constant full garbage collections. There is another message on this list about someone who upgraded from 3.5 to 4.2 and is having memory troubles. The person on IRC made most of their fields unstored and reindexed, which fixed the problem for them. They only needed a few fields stored. Because the IRC user was on 4.0, I originally thought it had something to do with compressed stored fields, but on this thread, they started with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version, then they had compressed stored fields before the upgrade. The user on IRC was not using termvectors or docvalues, which would be potential pain points unique to 4.2. I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's considerably larger than I need. There are no apparent memory leaks - it's been running for over a month with updates once a minute. I've finally switched over from the 3.5.0 index to the new one, so for the last few days, it has been also taking our full query load. What could have changed between 4.1 and 4.2 to cause dramatically increased memory usage? From my /admin/system: date name=startTime2013-04-05T15:52:55.751Z/date Thanks, Shawn
Java heap space exception in 4.2.1
Hi I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query.. Now i am getting this error frequently that solr does not run even for 2 minutes continuously. All 5GB is getting used instantaneously in few queries... SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:365) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:722) one thing i did was when we moved from 4.1 to 4.2.1 i changed only the solr.war and left other jars and config as it is... WIll that create a problem...?? ./zahoor
Re: Solr 4 memory usage increase
I get the same issue in 1.7.0_09-b05 also. ./zahoor On 17-May-2013, at 12:07 PM, Walter Underwood wun...@wunderwood.org wrote: It is past time to get off of Java 6. That is dead. End of life. No more updates, not even for security bugs. What version of Java 6? Some earlier versions had bad bugs that Solr would run into. We hit them in prod until we upgraded. wunder On May 16, 2013, at 11:28 PM, Wei Zhao wrote: No, exactly the same JVM of Java6 -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-4-memory-usage-increase-tp4064066p4064108.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Java heap space exception in 4.2.1
Hprof introspection shows that huge Double Array are using up 75% of heap space... which belongs to Lucen's FieldCache.. ./zahoor On 17-May-2013, at 12:47 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query.. Now i am getting this error frequently that solr does not run even for 2 minutes continuously. All 5GB is getting used instantaneously in few queries... SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:653) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:366) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:365) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:856) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:722) one thing i did was when we moved from 4.1 to 4.2.1 i changed only the solr.war and left other jars and config as it is... WIll that create a problem...?? ./zahoor
Re: Java heap space exception in 4.2.1
Memory increase a lot with queries which have facets… ./Zahoor On 17-May-2013, at 10:00 PM, Shawn Heisey s...@elyograg.org wrote: On 5/17/2013 1:17 AM, J Mohamed Zahoor wrote: I moved to 4.2.1 from 4.1 recently.. everything was working fine until i added few more stats query.. Now i am getting this error frequently that solr does not run even for 2 minutes continuously. All 5GB is getting used instantaneously in few queries... Someone on IRC ran into memory problems upgrading from 4.0 to 4.2. It wasn't OOM errors, they were just using a lot more heap than they were before and running into constant full garbage collections. There is another message on this list about someone who upgraded from 3.5 to 4.2 and is having memory troubles. The person on IRC made most of their fields unstored and reindexed, which fixed the problem for them. They only needed a few fields stored. Because the IRC user was on 4.0, I originally thought it had something to do with compressed stored fields, but on this thread, they started with 4.1. If that was the released 4.1.0 and not a SNAPSHOT version, then they had compressed stored fields before the upgrade. The user on IRC was not using termvectors or docvalues, which would be potential pain points unique to 4.2. I'm using 4.2.1 with no trouble in my setup, but I do have a heap that's considerably larger than I need. There are no apparent memory leaks - it's been running for over a month with updates once a minute. I've finally switched over from the 3.5.0 index to the new one, so for the last few days, it has been also taking our full query load. What could have changed between 4.1 and 4.2 to cause dramatically increased memory usage? From my /admin/system: date name=startTime2013-04-05T15:52:55.751Z/date Thanks, Shawn
Re: stats cache
Thanks.. i am caching in HTTP now.. ./zahoor On 08-May-2013, at 3:58 AM, Yonik Seeley yo...@lucidworks.com wrote: On Tue, May 7, 2013 at 12:48 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I am computing lots of stats as part of a query… looks like the solr caching is not helping here… Does solr caches stats of a query? No. Neither facet counts or stats part of a request are cached. The query cache only caches top N docs (plus scores if applicable) for a given query + filters. If the whole request is identical, then you can use an HTTP caching mechanism though. -Yonik http://lucidworks.com
Re: stats cache
I am using a simple LRU cache in my client where i store req and response for now. Later might move to something like varnish. ./zahoor On 08-May-2013, at 3:26 PM, Dmitry Kan solrexp...@gmail.com wrote: Mohamed, (out of curiosity) What kind of tool are you using for that? On Wed, May 8, 2013 at 10:13 AM, J Mohamed Zahoor zah...@indix.com wrote: Thanks.. i am caching in HTTP now.. ./zahoor On 08-May-2013, at 3:58 AM, Yonik Seeley yo...@lucidworks.com wrote: On Tue, May 7, 2013 at 12:48 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I am computing lots of stats as part of a query… looks like the solr caching is not helping here… Does solr caches stats of a query? No. Neither facet counts or stats part of a request are cached. The query cache only caches top N docs (plus scores if applicable) for a given query + filters. If the whole request is identical, then you can use an HTTP caching mechanism though. -Yonik http://lucidworks.com
stats cache
Hi I am computing lots of stats as part of a query… looks like the solr caching is not helping here… Does solr caches stats of a query? ./zahoor
facet.pivot limit
Hi is there a limit for facet pivot like we have in facet.limit? ./zahoor
Re: Max http connections in CloudSolrServer
On 18-Apr-2013, at 9:43 PM, Shawn Heisey s...@elyograg.org wrote: Are you using the Jetty included with Solr, or a Jetty installed separately? I am using the Jetty that comes with Solr. The Jetty included with Solr has a maxThreads value of 1 in its config. The default would be closer to 200, and a single request from a Cloud client likely uses multiple Jetty threads. The default maxThreads is 1 and minThreads is 10. ./zahoor
Re: stats.facet not working for timestamp field
It is a date field. field name=timestamp type=date indexed=true stored=true default=NOW multiValued=false/ ./zahoor On 19-Apr-2013, at 5:02 PM, Erick Erickson erickerick...@gmail.com wrote: I'm guessing that your timestamp is a tdate, which stores extra information in the index for fast range searches. What happens if you try to facet on just a date field? Best Erick On Thu, Apr 18, 2013 at 8:37 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi I am using SOlr 4.1 with 6 shards. i want to find out some price stats for all the days in my index. I ended up using stats component like stats=truestats.field=pricestats.facet=timestamp. but it throws up error like str name=msgInvalid Date String:' #1;#0;#0;#0;'[my(#0;'/str My Question is : is timestamp supported as stats.facet ? ./zahoor
Re: Max http connections in CloudSolrServer
Thanks for this. The reason i asked this was.. when i fire 30 queries simultaneously from 30 threads using the same CloudSolrServer instance, some queries gets fired after a delay.. sometime the delay is 30-50 seconds... In solr logs i can see.. 20+ queries get fired almost immediately... but some of them gets fired late.. i increased the connections per host from 32 to 200.. still no respite... ./zahoor On 18-Apr-2013, at 12:20 AM, Shawn Heisey s...@elyograg.org wrote: ModifiableSolrParams params = new ModifiableSolrParams(); params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000); params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 200); HttpClient client = HttpClientUtil.createClient(params); LBHttpSolrServer lbServer = new LBHttpSolrServer (client, http://localhost/solr;); lbServer.removeSolrServer(http://localhost/solr;); SolrServer server = new CloudSolrServer(zkHost, lbServer);
stats.facet not working for timestamp field
Hi I am using SOlr 4.1 with 6 shards. i want to find out some price stats for all the days in my index. I ended up using stats component like stats=truestats.field=pricestats.facet=timestamp. but it throws up error like str name=msgInvalid Date String:' #1;#0;#0;#0;'[my(#0;'/str My Question is : is timestamp supported as stats.facet ? ./zahoor
Re: Max http connections in CloudSolrServer
I dont yet know if this is the reason... I am looking if jetty has some limit on accepting connections.. ./zahoor On 18-Apr-2013, at 12:52 PM, J Mohamed Zahoor zah...@indix.com wrote: Thanks for this. The reason i asked this was.. when i fire 30 queries simultaneously from 30 threads using the same CloudSolrServer instance, some queries gets fired after a delay.. sometime the delay is 30-50 seconds... In solr logs i can see.. 20+ queries get fired almost immediately... but some of them gets fired late.. i increased the connections per host from 32 to 200.. still no respite... ./zahoor On 18-Apr-2013, at 12:20 AM, Shawn Heisey s...@elyograg.org wrote: ModifiableSolrParams params = new ModifiableSolrParams(); params.set(HttpClientUtil.PROP_MAX_CONNECTIONS, 1000); params.set(HttpClientUtil.PROP_MAX_CONNECTIONS_PER_HOST, 200); HttpClient client = HttpClientUtil.createClient(params); LBHttpSolrServer lbServer = new LBHttpSolrServer (client, http://localhost/solr;); lbServer.removeSolrServer(http://localhost/solr;); SolrServer server = new CloudSolrServer(zkHost, lbServer);
Max http connections in CloudSolrServer
Hi I am pumping parallel select queries using CloudSolrServer. It looks like it can handle only certain no of max connections... my Question is, How many concurrent queries does a CloudSolrServer can handle? An old thread tries to answer this by asking to give our own instance of LBHttpSolrServer... But it looks like there is no way from LBHttpSolrServer to up the maxConnection of the httpClient it has... Can someone let me know how to bump up the maxConnections and maxConnectionsPerHost parameter for the httpCLient used by cloudSolrServer? ./zahoor
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
It sure increased the performance . Thanks for the input. ./zahoor On 14-Apr-2013, at 10:13 PM, J Mohamed Zahoor zah...@indix.com wrote: Thanks.. Will try multithreading with CloudSolrServer. ./zahoor On 13-Apr-2013, at 9:11 PM, Mark Miller markrmil...@gmail.com wrote: On Apr 13, 2013, at 11:07 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi This question has come up many times in the list with lots of variations (which confuses me a lot). Iam using Solr 4.1. one collection , 6 shards, 6 machines. I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance. Question is: 1) is CloudSolrServer multiThreaded? No. The proper fast way to use it is to start many threads that all add docs to the same CloudSolrServer instance. In other words, currently, you must do the multi threading yourself. CloudSolrServer is thread safe. 2) Will using ConcurrentUpdateSolr server increase indexing performance? Yes, but at the cost of having to specify a server to talk to - if it goes down, so does your indexing. It's also not very great at reporting errors. Finally, using multiple threads and CloudSolrServer, you can approach the performance of ConcurrentUpdateSolr server. - Mark ./Zahoor
zkState changes too often
Hi I am using SolrCloud (4.1) with 6 nodes. When i index the documents from the mapper and as the load increases.. i see these messages in my mapper logs… WHich looks like it is slowing down my indexing speed. 2013-04-16 06:04:18,013 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (5) 2013-04-16 06:04:18,186 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (6) 2013-04-16 06:04:18,186 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (6) 2013-04-16 06:04:19,485 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6) 2013-04-16 06:04:19,487 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6) 2013-04-16 06:08:30,006 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6) 2013-04-16 06:08:30,010 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (5) 2013-04-16 06:08:30,010 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5) 2013-04-16 06:08:30,019 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (5) 2013-04-16 06:08:35,443 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5) 2013-04-16 06:08:35,446 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (6) 2013-04-16 06:08:35,446 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6) 2013-04-16 06:08:35,459 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (6) 2013-04-16 06:08:48,929 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6) 2013-04-16 06:08:48,931 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6) 2013-04-16 06:09:12,005 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6) 2013-04-16 06:09:12,010 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (5) 2013-04-16 06:09:12,011 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5) 2013-04-16 06:09:12,014 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (5) 2013-04-16 06:09:15,438 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5) 2013-04-16 06:09:15,441 INFO org.apache.solr.common.cloud.ZkStateReader: Updating live nodes... (6) 2013-04-16 06:09:15,441 INFO org.apache.solr.common.cloud.ZkStateReader: A cluster state change: WatchedEvent stat I tried increasing the Zk timeout from 15 to 20 sec… but i still see this message… anything i might try to avoid this? ./Zahoor
Re: CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Thanks.. Will try multithreading with CloudSolrServer. ./zahoor On 13-Apr-2013, at 9:11 PM, Mark Miller markrmil...@gmail.com wrote: On Apr 13, 2013, at 11:07 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi This question has come up many times in the list with lots of variations (which confuses me a lot). Iam using Solr 4.1. one collection , 6 shards, 6 machines. I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance. Question is: 1) is CloudSolrServer multiThreaded? No. The proper fast way to use it is to start many threads that all add docs to the same CloudSolrServer instance. In other words, currently, you must do the multi threading yourself. CloudSolrServer is thread safe. 2) Will using ConcurrentUpdateSolr server increase indexing performance? Yes, but at the cost of having to specify a server to talk to - if it goes down, so does your indexing. It's also not very great at reporting errors. Finally, using multiple threads and CloudSolrServer, you can approach the performance of ConcurrentUpdateSolr server. - Mark ./Zahoor
CloudSolrServer vs ConcurrentUpdateSolrServer for indexing
Hi This question has come up many times in the list with lots of variations (which confuses me a lot). Iam using Solr 4.1. one collection , 6 shards, 6 machines. I am using CloudSolrServer inside each mapper to index my documents…. While it is working fine , iam trying to improve the indexing performance. Question is: 1) is CloudSolrServer multiThreaded? 2) Will using ConcurrentUpdateSolr server increase indexing performance? ./Zahoor
Query using function query result
Hi i want to query documents which match a certain dynamic criteria. like, How do i get all documents, where sub(field1,field2) 0 ? i tried _val_: sub(field1,field2) and used fq:[_val_:[0 TO *] But it doesnt work. ./Zahoor
Accessing multicore setup using solrj
Hi I am having a multi core setup with 2 core core0 and core1. How do i insert doc in core 1? I am using as below. searchServer = new CloudSolrServer(zooQourumUrl); searchServer.setDefaultCollection(core1); searchServer.connect(); and i get No live solr servers exception. But i could see both the cores in UI up and running. am i missing something.? ./zahoor
multicore vs multi collection
Hi I am kind of confuzed between multi core and multi collection. Docs dont seem to clarify this.. can someone enlighten me what is ther difference between a core and a collection? Are they same? ./zahoor
Re: multicore vs multi collection
Thanks. This make it clear than the wiki. How do you create multiple collection which can have different schema? ./zahoor On 26-Mar-2013, at 3:52 PM, Furkan KAMACI furkankam...@gmail.com wrote: Did you check that document: http://wiki.apache.org/solr/SolrCloud#A_little_about_SolrCores_and_CollectionsIt says: On a single instance, Solr has something called a SolrCorehttp://wiki.apache.org/solr/SolrCorethat is essentially a single index. If you want multiple indexes, you create multiple SolrCores http://wiki.apache.org/solr/SolrCores. With SolrCloud, a single index can span multiple Solr instances. This means that a single index can be made up of multiple SolrCorehttp://wiki.apache.org/solr/SolrCore's on different machines. We call all of these SolrCoreshttp://wiki.apache.org/solr/SolrCoresthat make up one logical index a collection. A collection is a essentially a single index that spans many SolrCorehttp://wiki.apache.org/solr/SolrCore's, both for index scaling as well as redundancy. If you wanted to move your 2 SolrCore http://wiki.apache.org/solr/SolrCore Solr setup to SolrCloud, you would have 2 collections, each made up of multiple individual SolrCoreshttp://wiki.apache.org/solr/SolrCores. 2013/3/26 J Mohamed Zahoor zah...@indix.com Hi I am kind of confuzed between multi core and multi collection. Docs dont seem to clarify this.. can someone enlighten me what is ther difference between a core and a collection? Are they same? ./zahoor
NPE when adding docs in 4.2
Hi I am using Solr 4.2. I created 2 collections by using http://localhost:8983/solr/admin/collections?action=CREATEname=collection1numShards=1replicationFactor=0maxShardsPerNode=1; http://localhost:8983/solr/admin/collections?action=CREATEname=collection2numShards=1replicationFactor=0maxShardsPerNode=1; Now. When i try to insert the document in collection2, i get the following NPE. java.lang.NullPointerException at org.apache.solr.client.solrj.util.ClientUtils.addSlices(ClientUtils.java:273) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:214) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54) Anything obvious i am missing? ./zahoor
Re: NPE when adding docs in 4.2
aahha… i used a replication factor of 0. I thought 0 means no replication of original.. Should that be 1 if i want no replication? ./zahoor On 16-Mar-2013, at 8:46 PM, Mark Miller markrmil...@gmail.com wrote: You mean replication factor of 1 not 0. I'm cleaning up the collection api responses for 4.3 so that you would get a clear error for this type of thing (https://issues.apache.org/jira/browse/SOLR-4494). There should be info about it in the overseer log currently. I think I've also already fix that cloud server call to return an message about not finding the collection rather than an NPE (for 4.3). - Mark On Mar 16, 2013, at 9:47 AM, J Mohamed Zahoor zah...@indix.com wrote: Hi I am using Solr 4.2. I created 2 collections by using http://localhost:8983/solr/admin/collections?action=CREATEname=collection1numShards=1replicationFactor=0maxShardsPerNode=1; http://localhost:8983/solr/admin/collections?action=CREATEname=collection2numShards=1replicationFactor=0maxShardsPerNode=1; Now. When i try to insert the document in collection2, i get the following NPE. java.lang.NullPointerException at org.apache.solr.client.solrj.util.ClientUtils.addSlices(ClientUtils.java:273) at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:214) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:68) at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:54) Anything obvious i am missing? ./zahoor
Re: NPE when adding docs in 4.2
Got it.. Thanks. ./Zahoor On 16-Mar-2013, at 9:13 PM, Yonik Seeley yo...@lucidworks.com wrote: On Sat, Mar 16, 2013 at 11:36 AM, J Mohamed Zahoor jmo...@gmail.com wrote: aahha… i used a replication factor of 0. I thought 0 means no replication of original.. Should that be 1 if i want no replication? Think of it as the number of copies of a book at a library. replicationFactor is the number of copies of each document. If a library says they have 3 copies of a book, they mean they actually have 3 books (not 4). -Yonik http://lucidworks.com
Exception while using CloudServer
Hi I was trying to connect to solr cloud using CloudServer, I get the following exception. I tried clearing the zookeeper state and then restarting the solr instances, still i get the same exception. am i missing something? org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper... Exception in thread main java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:297) at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:270) at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:274) at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:138) ./Zahoor
Re: Exception while using CloudServer
I am using Solr 4.0. ./zahoor On 13-Feb-2013, at 3:56 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I was trying to connect to solr cloud using CloudServer, I get the following exception. I tried clearing the zookeeper state and then restarting the solr instances, still i get the same exception. am i missing something? org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper... Exception in thread main java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:297) at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:270) at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:274) at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:138) ./Zahoor
Re: Exception while using CloudServer
Hi I think the router:compositeId value inside the cluster state is creating this problem. ./Zahoor On 13-Feb-2013, at 4:06 PM, J Mohamed Zahoor zah...@indix.com wrote: I am using Solr 4.0. ./zahoor On 13-Feb-2013, at 3:56 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I was trying to connect to solr cloud using CloudServer, I get the following exception. I tried clearing the zookeeper state and then restarting the solr instances, still i get the same exception. am i missing something? org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper... Exception in thread main java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:297) at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:270) at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:274) at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:138) ./Zahoor
Re: Exception while using CloudServer
Apologies... I was using 4.1 in solr server and 4.0 in solrj client which caused this problem. ./zahoor On 13-Feb-2013, at 4:08 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I think the router:compositeId value inside the cluster state is creating this problem. ./Zahoor On 13-Feb-2013, at 4:06 PM, J Mohamed Zahoor zah...@indix.com wrote: I am using Solr 4.0. ./zahoor On 13-Feb-2013, at 3:56 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi I was trying to connect to solr cloud using CloudServer, I get the following exception. I tried clearing the zookeeper state and then restarting the solr instances, still i get the same exception. am i missing something? org.apache.solr.common.cloud.ZkStateReader: Updating cluster state from ZooKeeper... Exception in thread main java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Map at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:297) at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:270) at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:274) at org.apache.solr.client.solrj.impl.CloudSolrServer.connect(CloudSolrServer.java:138) ./Zahoor
LoadBalancing while adding documents
Hi I have multi shard replicated index spread across two machines. Once a week, i delete the entire index and create it from scratch. Today i am using ConcurrentUpdateSolrServer in solrj to add documents to the index. I want to add documents through both the servers.. to utilise the resources... i read in wiki (i think) that LBHttpSolrServer should not be used for indexing documents. Is there any other way to send request to both the servers without using any external load balancers? I am using Solr 4.1. ./zahoor
Re: LoadBalancing while adding documents
On 13-Feb-2013, at 8:11 AM, Erick Erickson erickerick...@gmail.com wrote: Hold on here. LBHttpSolrServer should not be used for indexing in a Master/Slave setup, but in SolrCloud you may use it. Indeed, CloudSolrServer uses LBHttpSolrServer under the covers. In SolrCloud mode, ConcurrentUpdateSolrServer will already do the LoadBalacing while adding and querying documents from Solr. is my understanding right? Now, why would you want to send requests to both servers? I just wanted to send some docs to machine1 and some docs to machine2 to load balance. Not the same doc to both the machines. If you're in master/slave mode (i.e. not running Zookeeper), you _must_ send the update to the right master. If you're in SolrCloud mode, you don't care. You have to send each document to Solr only once. In Master/Slave mode, you must send it to the correct master. In SolrCloud mode you don't care where you send it, it'll be routed to the right place. I am in SolrCloud mode. I always send it to one of the server. And if i get you right, they will automatically loadBalance is what i take. ./Zahoor
Re: LoadBalancing while adding documents
Ooh.. I dint know that there is CloudSolrServer. Thanks for the pointer. Will explore that. ./zahoor On 13-Feb-2013, at 11:49 AM, J Mohamed Zahoor zah...@indix.com wrote: On 13-Feb-2013, at 8:11 AM, Erick Erickson erickerick...@gmail.com wrote: Hold on here. LBHttpSolrServer should not be used for indexing in a Master/Slave setup, but in SolrCloud you may use it. Indeed, CloudSolrServer uses LBHttpSolrServer under the covers. In SolrCloud mode, ConcurrentUpdateSolrServer will already do the LoadBalacing while adding and querying documents from Solr. is my understanding right? Now, why would you want to send requests to both servers? I just wanted to send some docs to machine1 and some docs to machine2 to load balance. Not the same doc to both the machines. If you're in master/slave mode (i.e. not running Zookeeper), you _must_ send the update to the right master. If you're in SolrCloud mode, you don't care. You have to send each document to Solr only once. In Master/Slave mode, you must send it to the correct master. In SolrCloud mode you don't care where you send it, it'll be routed to the right place. I am in SolrCloud mode. I always send it to one of the server. And if i get you right, they will automatically loadBalance is what i take. ./Zahoor
distinct count of facet field values
Hi Is it possible to get the distinct count of a given facet field in Solr? A query like this q=*:*facet=truefacet.field=cat display the counts of all the unique categories present like electronics: 100 applicances:200 etc.. But if the list is big.. i dont want to get the entire list and take a count by looping... Instead if i get a count of the no of items in the list.. i am okay.. SOLR-2242 was doing just that... But it is not giving a distinct count if i have multiple shards... Is there any other way to get this? ./Zahoor
Re: Solr 4.1 UI fail to display result
I'm not sure why .. but this sounds like the JSON Parser was called with an HTML- or XML-String? After you hit the Execute Button on the Website, on the top of the right content-area, there is a link - which is what the UI will request .. if you open that in another browser-tab or with curl/wget .. what is the response you get? Is that really JSON? Or perhaps some kind of Error Message? The link itself does not seem to be okay. It shows only this for q=*:* http://localhost:8983/solr/collection1/select? But if i add a wt=json in another tab.. i get a json response. ./zahoor
Re: Solr 4.1 UI fail to display result
It works In chrome though... ./Zahoor@iPhone On 02-Feb-2013, at 4:34 PM, J Mohamed Zahoor zah...@indix.com wrote: I'm not sure why .. but this sounds like the JSON Parser was called with an HTML- or XML-String? After you hit the Execute Button on the Website, on the top of the right content-area, there is a link - which is what the UI will request .. if you open that in another browser-tab or with curl/wget .. what is the response you get? Is that really JSON? Or perhaps some kind of Error Message? The link itself does not seem to be okay. It shows only this for q=*:* http://localhost:8983/solr/collection1/select? But if i add a wt=json in another tab.. i get a json response. ./zahoor
Image retrieval
Hi Iam trying to integrate a image retrieval system (Lire - http://www.semanticmetadata.net/lire/) in to Solr. Images are stored as its descriptors or features. But when searching for a image match, I need to pick images whose features are close (by calculating the distance) to the given image. Lire does this by going through all the docs in the Lucene index in a tight loop. What is the best way to implement this in Solr? Storing the feature is straight forward text or Binary field. Retrieving is a problem…. as we need to pick images below a certain distance threshold. Is this possible using a custom filter placed in the query path? ./zahoor
Solr 4.1 UI fail to display result
Hi Iam using 4.1 release and i see a problem when i set the response type as JSON in the UI. I am using Safari 6.0.2 and i see a SyntaxError: JSON Parse error: Unrecognized token ''. app.js line 465. When i debug more.. i see the response is still coming in XML format. Is anyone else facing this problem? ./Zahoor
Re: Solr 4.1 UI fail to display result
The stack is format_json -- app.js (465) json -- query.js (59) complete - query.js (77) fire -- require.js (3099) fireWith -- require.js (3217) done -- require.js (9469) callback -- require.js (10235) ./zahoor On 30-Jan-2013, at 6:43 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi Iam using 4.1 release and i see a problem when i set the response type as JSON in the UI. I am using Safari 6.0.2 and i see a SyntaxError: JSON Parse error: Unrecognized token ''. app.js line 465. When i debug more.. i see the response is still coming in XML format. Is anyone else facing this problem? ./Zahoor
Re: Solr 4.1 UI fail to display result
Hi Alex, Cleared Cache - Problem persists. Disabled Cache - problem Persists. This was in Safari though. ./zahoor On 30-Jan-2013, at 6:55 PM, Alexandre Rafalovitch arafa...@gmail.com wrote: Before worrying about anything else, try doing a full cache clean. My (Chrome) browser was caching Solr 4.0 resources for unreasonably long period of time until I completely disable its cache (in dev tools) and tried the full reload. Or try a browser you did not use before. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Wed, Jan 30, 2013 at 8:17 AM, J Mohamed Zahoor zah...@indix.com wrote: The stack is format_json -- app.js (465) json -- query.js (59) complete - query.js (77) fire -- require.js (3099) fireWith -- require.js (3217) done -- require.js (9469) callback -- require.js (10235) ./zahoor On 30-Jan-2013, at 6:43 PM, J Mohamed Zahoor zah...@indix.com wrote: Hi Iam using 4.1 release and i see a problem when i set the response type as JSON in the UI. I am using Safari 6.0.2 and i see a SyntaxError: JSON Parse error: Unrecognized token ''. app.js line 465. When i debug more.. i see the response is still coming in XML format. Is anyone else facing this problem? ./Zahoor
Re: zookeeper config
Cool. Thanks. On 24-Jan-2013, at 1:28 PM, Per Steffensen st...@designware.dk wrote: This is supported. You just need to ajust your ZK connection-string: host1:port1/solr,host2:port2/solr,...,hostN:portN/solr Regards, Per Steffensen On 1/24/13 7:57 AM, J Mohamed Zahoor wrote: Hi I am using Solr 4.0. I see the Solr data in zookeeper is placed on the root znode itself. This becomes a pain if the zookeeper instance is used for multiple projects like HBase and like. I am thinking of raising a Jira for putting them under a znode /solr or something like that? ./Zahoor
zookeeper config
Hi I am using Solr 4.0. I see the Solr data in zookeeper is placed on the root znode itself. This becomes a pain if the zookeeper instance is used for multiple projects like HBase and like. I am thinking of raising a Jira for putting them under a znode /solr or something like that? ./Zahoor
Re: retrieving latest document **only**
group field is timestamp… it is not multivalued. ./zahoor On 15-Jan-2013, at 7:14 PM, Upayavira u...@odoko.co.uk wrote: Is your group field multivalued? Could docs appear in more than one group? Upayavira On Tue, Jan 15, 2013, at 01:22 PM, J Mohamed Zahoor wrote: The sum of all the count in the groups… does not match the total no of docs found. ./zahoor On 12-Jan-2013, at 1:27 PM, Upayavira u...@odoko.co.uk wrote: Not sure exactly what you mean, can you give an example? Upayavira On Sat, Jan 12, 2013, at 06:32 AM, J Mohamed Zahoor wrote: Cool… it worked… But the count of all the groups and the count inside stats component does not match… Is that a bug? ./zahoor On 11-Jan-2013, at 6:48 PM, Upayavira u...@odoko.co.uk wrote: could you use field collapsing? Boost by date and only show one value per group, and you'll have the most recent document only. Upayavira On Fri, Jan 11, 2013, at 01:10 PM, jmozah wrote: one crude way is first query and pick the latest date from the result then issue a query with q=timestamp[latestDate TO latestDate] But i dont want to execute two queries... ./zahoor On 11-Jan-2013, at 6:37 PM, jmozah jmo...@gmail.com wrote: What do you want? 'the most recent ones' or '**only** the latest' ? Perhaps a range query q=timestamp:[refdate TO NOW] will match your needs. Uwe I need **only** the latest documents... in the above query , refdate can vary based on the query. ./zahoor
Re: retrieving latest document **only**
The sum of all the count in the groups… does not match the total no of docs found. ./zahoor On 12-Jan-2013, at 1:27 PM, Upayavira u...@odoko.co.uk wrote: Not sure exactly what you mean, can you give an example? Upayavira On Sat, Jan 12, 2013, at 06:32 AM, J Mohamed Zahoor wrote: Cool… it worked… But the count of all the groups and the count inside stats component does not match… Is that a bug? ./zahoor On 11-Jan-2013, at 6:48 PM, Upayavira u...@odoko.co.uk wrote: could you use field collapsing? Boost by date and only show one value per group, and you'll have the most recent document only. Upayavira On Fri, Jan 11, 2013, at 01:10 PM, jmozah wrote: one crude way is first query and pick the latest date from the result then issue a query with q=timestamp[latestDate TO latestDate] But i dont want to execute two queries... ./zahoor On 11-Jan-2013, at 6:37 PM, jmozah jmo...@gmail.com wrote: What do you want? 'the most recent ones' or '**only** the latest' ? Perhaps a range query q=timestamp:[refdate TO NOW] will match your needs. Uwe I need **only** the latest documents... in the above query , refdate can vary based on the query. ./zahoor
Re: retrieving latest document **only**
Cool… it worked… But the count of all the groups and the count inside stats component does not match… Is that a bug? ./zahoor On 11-Jan-2013, at 6:48 PM, Upayavira u...@odoko.co.uk wrote: could you use field collapsing? Boost by date and only show one value per group, and you'll have the most recent document only. Upayavira On Fri, Jan 11, 2013, at 01:10 PM, jmozah wrote: one crude way is first query and pick the latest date from the result then issue a query with q=timestamp[latestDate TO latestDate] But i dont want to execute two queries... ./zahoor On 11-Jan-2013, at 6:37 PM, jmozah jmo...@gmail.com wrote: What do you want? 'the most recent ones' or '**only** the latest' ? Perhaps a range query q=timestamp:[refdate TO NOW] will match your needs. Uwe I need **only** the latest documents... in the above query , refdate can vary based on the query. ./zahoor
4.1 release date
Hi When can we expect 4.1? ./zahoor
Re: multi field query with selective results
I don't think I hijacked any thread. it is a new thread. Can you please enlighten me? On Sunday, December 23, 2012, Lance Norskog wrote: Please start a new thread. Thanks! On 12/22/2012 11:03 AM, J Mohamed Zahoor wrote: Hi I have a word completion requirement where i need to pick result from two indexed fields. The trick is i need to pick top 5 results from each field and display as suggestions. If i set fq as field1:XXX AND field2:XXX, the top result comes entirely from field1 matches. Is there any other way to get top 5 from field 1 matches and top 5 from field 2 matched results? ./Zahoor
multi field query with selective results
Hi I have a word completion requirement where i need to pick result from two indexed fields. The trick is i need to pick top 5 results from each field and display as suggestions. If i set fq as field1:XXX AND field2:XXX, the top result comes entirely from field1 matches. Is there any other way to get top 5 from field 1 matches and top 5 from field 2 matched results? ./Zahoor
