Unique Key error on trunk
Hello, We use solr.UUIDField to generate unique ids, using the latest trunk (change list 1163767) seems to throw an error Document is missing mandatory uniqueKey field: id. The schema is setup to generate a id field on updates field name=id type=uuid indexed=true stored=true default=NEW / Thanks Viswa SEVERE: org.apache.solr.common.SolrException: Document is missing mandatory uniqueKey field: id at org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:80) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:145) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:127) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1406) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Re: Unique Key error on trunk
You can replicate it with the example app by replacing the id definition in schema.xml with field name=id type=uuid indexed=true stored=true default=NEW / Removing the id fields in the one of the example doc.xml and posting it to solr. Thanks Viswa On Sep 26, 2011, at 12:15 AM, Viswa S wrote: Hello, We use solr.UUIDField to generate unique ids, using the latest trunk (change list 1163767) seems to throw an error Document is missing mandatory uniqueKey field: id. The schema is setup to generate a id field on updates field name=id type=uuid indexed=true stored=true default=NEW / Thanks Viswa SEVERE: org.apache.solr.common.SolrException: Document is missing mandatory uniqueKey field: id at org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:80) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:145) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:127) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:77) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:67) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1406) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:945) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
RE: Solr Terms and Date field issues
Thanks Erick Ahmet, that helps. Date: Fri, 6 May 2011 09:25:11 -0400 Subject: Re: Solr Terms and Date field issues From: erickerick...@gmail.com To: solr-user@lucene.apache.org OK, I'm reaching a little here, but I think it's got a pretty good chance of being the issue you're seeing. Sure hope somebody jumps in and corrects me if I'm wrong (hint hint)... I haven't delved into the actual Trie code, this is just from looking with TermsComponent and Luke. Using Solr 1.4.1 BTW. What you're seeing it a consequence of the trie field type with a precision step other than 0. Trie fields with precisionstep 0 add extra stuff to the index to allow more efficient range queries. A hint about this is that your 5 documents with the tdate type produce 16 tokens rather than just 5. If you try your experiment with the date type (which is a trie type with precisionstep=0) you'll see exactly what you expect. So the long and short of it is that Solr's working as expected, and you can use your index without worrying. But, if you're trying to do some lower-level term walking, you'll either have to filter stuff out, copy your dates to something with precisionstep=0 and use that field or Best Erick On Thu, May 5, 2011 at 9:08 PM, Ahmet Arslan iori...@yahoo.com wrote: It is okey to see weird things in admin/schema.jsp or terms component with trie based types. Please see http://search-lucene.com/m/WEfSI1Yi4562/ If you really need terms component, consider using copyField (tdate to string type) Please find attached the schema and some test data (test.xml). Thanks for looking this. Viswa Date: Thu, 5 May 2011 19:08:31 -0400 Subject: Re: Solr Terms and Date field issues From: erickerick...@gmail.com To: solr-user@lucene.apache.org H, this is puzzling. If you could come up with a couple of xml files and a schema that illustrate this, I'll see what I can see... Thanks, Erick On Wed, May 4, 2011 at 7:05 PM, Viswa S svis...@hotmail.com wrote: Erik, I suspected the same, and setup a test instance to reproduce this. The date field I used is setup to capture indexing time, in other words the schema has a default value of NOW. However, I have reproduced this issue with fields which do no have defaults too. On the second one, I did a delete-commit (with expungeDeletes=true) and then a optimize. All other fields show updated terms except the date fields. I have also double checked to see if the Luke handler has any different terms, and it did not. Thanks Viswa Date: Wed, 4 May 2011 08:17:39 -0400 Subject: Re: Solr Terms and Date field issues From: erickerick...@gmail.com To: solr-user@lucene.apache.org Hmmm, this *looks* like you've changed your schema without re-indexing all your data so you're getting old (string?) values in that field, but that's just a guess. If this is really happening on a clean index it's a problem. I'm also going to guess that you're not really deleting the documents you think. Are you committing after the deletes? Best Erick On Wed, May 4, 2011 at 2:18 AM, Viswa S svis...@hotmail.com wrote: Hello, The terms query for a date field seems to get populated with some weird dates, many of these dates (1970,2009,2011-04-23) are not present in the indexed data. Please see sample data below I also notice that a delete and optimize does not remove the relevant terms for date fields, the string fields seems work fine. Thanks Viswa Results from Terms component: int name=2011-05-04T02:01:32.928Z3479/int int name=2011-05-04T02:00:19.2Z3479/int int name=2011-05-03T22:34:58.432Z3479/int int name=2011-04-23T01:36:14.336Z3479/int int name=2009-03-13T13:23:01.248Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=2011-05-04T02:01:34.592Z265/int Result from facet component, rounded by seconds.: lst name=InsertTime int name=2011-05-04T02:01:32Z1/int int name=2011-05-04T02:01:33Z1148/int int name=2011-05-04T02:01:34Z2333/int str name=gap+1SECOND/str date name=start2011-05-03T06:14:14Z/date date name=end2011-05-04T06:14:14Z/date/lst
RE: Solr Terms and Date field issues
Please find attached the schema and some test data (test.xml). Thanks for looking this. Viswa Date: Thu, 5 May 2011 19:08:31 -0400 Subject: Re: Solr Terms and Date field issues From: erickerick...@gmail.com To: solr-user@lucene.apache.org H, this is puzzling. If you could come up with a couple of xml files and a schema that illustrate this, I'll see what I can see... Thanks, Erick On Wed, May 4, 2011 at 7:05 PM, Viswa S svis...@hotmail.com wrote: Erik, I suspected the same, and setup a test instance to reproduce this. The date field I used is setup to capture indexing time, in other words the schema has a default value of NOW. However, I have reproduced this issue with fields which do no have defaults too. On the second one, I did a delete-commit (with expungeDeletes=true) and then a optimize. All other fields show updated terms except the date fields. I have also double checked to see if the Luke handler has any different terms, and it did not. Thanks Viswa Date: Wed, 4 May 2011 08:17:39 -0400 Subject: Re: Solr Terms and Date field issues From: erickerick...@gmail.com To: solr-user@lucene.apache.org Hmmm, this *looks* like you've changed your schema without re-indexing all your data so you're getting old (string?) values in that field, but that's just a guess. If this is really happening on a clean index it's a problem. I'm also going to guess that you're not really deleting the documents you think. Are you committing after the deletes? Best Erick On Wed, May 4, 2011 at 2:18 AM, Viswa S svis...@hotmail.com wrote: Hello, The terms query for a date field seems to get populated with some weird dates, many of these dates (1970,2009,2011-04-23) are not present in the indexed data. Please see sample data below I also notice that a delete and optimize does not remove the relevant terms for date fields, the string fields seems work fine. Thanks Viswa Results from Terms component: int name=2011-05-04T02:01:32.928Z3479/int int name=2011-05-04T02:00:19.2Z3479/int int name=2011-05-03T22:34:58.432Z3479/int int name=2011-04-23T01:36:14.336Z3479/int int name=2009-03-13T13:23:01.248Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=2011-05-04T02:01:34.592Z265/int Result from facet component, rounded by seconds.: lst name=InsertTime int name=2011-05-04T02:01:32Z1/int int name=2011-05-04T02:01:33Z1148/int int name=2011-05-04T02:01:34Z2333/int str name=gap+1SECOND/str date name=start2011-05-03T06:14:14Z/date date name=end2011-05-04T06:14:14Z/date/lst add doc field name=fullTextLogI suspected the same, and setup a test instance to reproduce this/field /doc doc field name=fullTextLogThe date field I used is setup to capture indexing time, in other words the schema has a default value of NOW/field /doc doc field name=fullTextLogHowever, I have reproduced this issue with fields which do not have defaults too./field /doc doc field name=fullTextLog Lorem Ipsum is simply dummy text of the printing and typesetting industry/field /doc doc field name=fullTextLogContrary to popular belief, Lorem Ipsum is not simply random text. It has roots in a piece of classical Latin literature from 45 BC, making it over 2000 years old./field /doc /add ?xml version=1.0 encoding=UTF-8 ? !-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the License); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -- !-- This is the Solr schema file. This file should be named schema.xml and should be in the conf directory under the solr home (i.e. ./solr/conf/schema.xml by default) or located where the classloader for the Solr webapp can find it. This example schema is the recommended starting point for users. It should be kept correct and concise, usable out-of-the-box. For more information, on how to customize this file, please see http://wiki.apache.org/solr/SchemaXml PERFORMANCE NOTE: this schema includes many optional features and should not be used for benchmarking. To improve
Solr Terms and Date field issues
Hello, The terms query for a date field seems to get populated with some weird dates, many of these dates (1970,2009,2011-04-23) are not present in the indexed data. Please see sample data below I also notice that a delete and optimize does not remove the relevant terms for date fields, the string fields seems work fine. Thanks Viswa Results from Terms component: int name=2011-05-04T02:01:32.928Z3479/int int name=2011-05-04T02:00:19.2Z3479/int int name=2011-05-03T22:34:58.432Z3479/int int name=2011-04-23T01:36:14.336Z3479/int int name=2009-03-13T13:23:01.248Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=2011-05-04T02:01:34.592Z265/int Result from facet component, rounded by seconds.: lst name=InsertTime int name=2011-05-04T02:01:32Z1/int int name=2011-05-04T02:01:33Z1148/int int name=2011-05-04T02:01:34Z2333/int str name=gap+1SECOND/str date name=start2011-05-03T06:14:14Z/date date name=end2011-05-04T06:14:14Z/date/lst
RE: Solr Terms and Date field issues
Erik, I suspected the same, and setup a test instance to reproduce this. The date field I used is setup to capture indexing time, in other words the schema has a default value of NOW. However, I have reproduced this issue with fields which do no have defaults too. On the second one, I did a delete-commit (with expungeDeletes=true) and then a optimize. All other fields show updated terms except the date fields. I have also double checked to see if the Luke handler has any different terms, and it did not. Thanks Viswa Date: Wed, 4 May 2011 08:17:39 -0400 Subject: Re: Solr Terms and Date field issues From: erickerick...@gmail.com To: solr-user@lucene.apache.org Hmmm, this *looks* like you've changed your schema without re-indexing all your data so you're getting old (string?) values in that field, but that's just a guess. If this is really happening on a clean index it's a problem. I'm also going to guess that you're not really deleting the documents you think. Are you committing after the deletes? Best Erick On Wed, May 4, 2011 at 2:18 AM, Viswa S svis...@hotmail.com wrote: Hello, The terms query for a date field seems to get populated with some weird dates, many of these dates (1970,2009,2011-04-23) are not present in the indexed data. Please see sample data below I also notice that a delete and optimize does not remove the relevant terms for date fields, the string fields seems work fine. Thanks Viswa Results from Terms component: int name=2011-05-04T02:01:32.928Z3479/int int name=2011-05-04T02:00:19.2Z3479/int int name=2011-05-03T22:34:58.432Z3479/int int name=2011-04-23T01:36:14.336Z3479/int int name=2009-03-13T13:23:01.248Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=1970-01-01T00:00:00Z3479/int int name=2011-05-04T02:01:34.592Z265/int Result from facet component, rounded by seconds.: lst name=InsertTime int name=2011-05-04T02:01:32Z1/int int name=2011-05-04T02:01:33Z1148/int int name=2011-05-04T02:01:34Z2333/int str name=gap+1SECOND/str date name=start2011-05-03T06:14:14Z/date date name=end2011-05-04T06:14:14Z/date/lst
Exception on distributed date facet SOLR-1709
Folks,
We are trying to do some date faceting on our distributed environment, applied
solr-1709 on the trunk. A date facet query throws the below exception, I have
attached the patched source for reference. Any help would be appreciated.
Other Info:
Java ver: 1_6_0_24
Trung change list: 1022216
SEVERE: java.lang.ClassCastException: java.util.Date cannot be cast to
java.lang.Integer
at
org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponent.java:294)
at
org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:232)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:326)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547)
at
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
/**
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the License); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an AS IS BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.handler.component;
import java.io.IOException;
import java.net.URL;
import java.util.*;
import org.apache.solr.common.params.CommonParams;
import org.apache.solr.common.params.FacetParams;
import org.apache.solr.common.params.SolrParams;
import org.apache.solr.common.params.ModifiableSolrParams;
import org.apache.solr.common.util.NamedList;
import org.apache.solr.common.util.SimpleOrderedMap;
import org.apache.solr.common.util.StrUtils;
import org.apache.solr.common.SolrException;
import org.apache.solr.request.SimpleFacets;
import org.apache.lucene.util.OpenBitSet;
import org.apache.solr.search.QueryParsing;
import org.apache.solr.schema.FieldType;
import org.apache.lucene.queryParser.ParseException;
/**
* TODO!
*
* @version $Id: FacetComponent.java 1004082 2010-10-04 01:54:23Z rmuir $
* @since solr 1.3
*/
public class FacetComponent extends SearchComponent
{
public static final String COMPONENT_NAME = facet;
@Override
public void prepare(ResponseBuilder rb) throws IOException
{
if (rb.req.getParams().getBool(FacetParams.FACET,false)) {
rb.setNeedDocSet( true );
rb.doFacets = true;
}
}
/**
* Actually run the query
* @param rb
*/
@Override
public void process(ResponseBuilder rb) throws IOException
{
if (rb.doFacets) {
SolrParams params = rb.req.getParams();
SimpleFacets f = new SimpleFacets(rb.req,
rb.getResults().docSet,
RE: Exception on distributed date facet SOLR-1709
Peter I haven't, our implementation needs ZK integration.Would be great if you can give any additional pointers on porting this to trunk. Thanks Viswa Date: Fri, 18 Mar 2011 13:52:39 + Subject: Re: Exception on distributed date facet SOLR-1709 From: peter.stu...@gmail.com To: solr-user@lucene.apache.org Hi Viswa, This patch was orignally built for the 3x branch, and I don't see any ported patch revision or testing for trunk. A lot has changed in faceting from 3x to trunk, so it will likely need a bit of adjusting to cater for these changes (e.g. deprecation of date range in favour of range). Have you tried this patch on 3x branch? Thanks, Peter On Fri, Mar 18, 2011 at 7:09 AM, Viswa S svis...@hotmail.com wrote: Folks, We are trying to do some date faceting on our distributed environment, applied solr-1709 on the trunk. A date facet query throws the below exception, I have attached the patched source for reference. Any help would be appreciated. Other Info: Java ver: 1_6_0_24 Trung change list: 1022216 SEVERE: java.lang.ClassCastException: java.util.Date cannot be cast to java.lang.Integer at org.apache.solr.handler.component.FacetComponent.countFacets(FacetComponent.java:294) at org.apache.solr.handler.component.FacetComponent.handleResponses(FacetComponent.java:232) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:326) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1325) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:547) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Trunk Compile failure/ hang
Hello, I am trying to build source out of trunk (to apply a patch) and ran into an issue were the build process hangs ( below output) during build lucene at sanity-load-lib. Just when build sanity-load-lib starts, I see an dialog box asking for applet access permission The applet is attempting to invoke the java/lang/System.loadLibrary() operation on db_java-4.7 . I click on Allow and the build process never resumes (waited for more than an hour) Any thoughts would be very helpful. -Viswa ...build-lucene: contrib-build.init: get-db-jar: check-and-get-db-jar: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: compile-core: compile-test-framework: common.compile-test: sanity-load-lib:
RE: Empty value/string matching
Anyone know why this would not be working in solr?. Just to recap, we are trying to exclude document which have fields missing values in the search results. I have tried and none of it seems to be working: 1. *:* -field:[* TO *]2. -field:[* TO *]3. field: The fields are either typed string or custom and the query parser used is the,LuceneQParser. The below suggested solutions of using some default values do not work for our use case. ThanksViswa From: bob.sandif...@sirsidynix.com To: solr-user@lucene.apache.org Date: Mon, 22 Nov 2010 08:35:22 -0700 Subject: RE: Empty value/string matching One possibility to consider - if you really need documents with specifically empty or non-defined values (if that's not an oxymoron :)), and you have control over the values you send into the indexing, you could set a special value that means 'no value'. We've done that in a similar vein, using something like '@@EMPTY@@' for a given field, meaning that the original document didn't actually have a value for that field. I.E. it is something very unlikely to be a 'real' value - and then we can easily select on documents by querying for the field:@@EMPTY@@ instead of the negated form of the select... However, we haven't considered things like what it does to index size. It's relatively rare for us (that there not be a value), so our 'gut feel' is that it's not impacting the indexes very much size-wise or performance-wise. Bob Sandiford | Lead Software Engineer | SirsiDynix P: 800.288.8020 X6943 | bob.sandif...@sirsidynix.com www.sirsidynix.com -Original Message- From: Viswa S [mailto:svis...@hotmail.com] Sent: Saturday, November 20, 2010 5:38 PM To: solr-user@lucene.apache.org Subject: RE: Empty value/string matching Erick, Thanks for the quick response. The output i showed is on a test instance i created to simulate this issue. I intentionally tried to create documents with no values by creating xml nodes with field name=fieldName/field, but having values in the other fields in a document. Are you saying that there is no way have a field with no value?, with text fields they seem to make sense than for string?. You are right on fieldName:[* TO *] results, which basically returned all the documents which included the couple of documents in question. -Viswa Date: Sat, 20 Nov 2010 17:20:53 -0500 Subject: Re: Empty value/string matching From: erickerick...@gmail.com To: solr-user@lucene.apache.org I don't think that's correct. The documents wouldn't be showing up in the facets if they had no value for the field. So I think you're being mislead by the printout from the faceting. Perhaps you have unprintable characters in there or some such. Certainly the name: is actually a value, admittedly just a space. As for the other, I suspect something similar. What results do you get back when you just search for FieldName:[* TO *]? I'm betting you get all the docs back, but I've been very wrong before. Best Erick On Sat, Nov 20, 2010 at 5:02 PM, Viswa S svis...@hotmail.com wrote: Yes I do have a couple of documents with no values and one with an empty string. Find below the output of a facet on the fieldName. ThanksViswa int name=2/intint name=CASTIGO.4302/intint name=GDOGPRODY.4242/intint name=QMAGIC.4122/intint name= 1/int Date: Sat, 20 Nov 2010 15:29:06 -0500 Subject: Re: Empty value/string matching From: erickerick...@gmail.com To: solr-user@lucene.apache.org Are you absolutely sure your documents really don't have any values for FieldName? Because your results are perfectly correct if every doc has a value for FieldName. Or are you saying there no such field as FieldName? Best Erick On Sat, Nov 20, 2010 at 3:12 PM, Viswa S svis...@hotmail.com wrote: Folks,Am trying to query documents which have no values present, I have used the following constructs and it doesn't seem to work on the solr dev tip (as of 09/22) or the 1.4 builds.1. (*:* AND -FieldName[* TO *]) - returns no documents, parsedquery was +MatchAllDocsQuery(*:*) -FieldName:[* TO *]2. -FieldName:[* TO *] - returns no documents, parsedquery was -FieldName:[* TO *]3. FieldName: - returns no documents, parsedquery was empty (str name=parsedquery/)The field is type string, using the LuceneQParser, I have also tried to see if FieldName:[* TO *] if the documents with no terms are ignored and didn't seem to be the case, the result set was everything.Any help would be appreciated.-Viswa
RE: Userdefined Field type - Faceting
This worked, thanks Yonik. -Viswa Date: Mon, 13 Dec 2010 22:54:35 -0500 Subject: Re: Userdefined Field type - Faceting From: yo...@lucidimagination.com To: solr-user@lucene.apache.org Perhaps try overriding indexedToReadable() also? -Yonik http://www.lucidimagination.com On Mon, Dec 13, 2010 at 10:00 PM, Viswa S svis...@hotmail.com wrote: Hello, We implemented an IP-Addr field type which internally stored the ips as hex-ed string (e.g. 192.2.103.29 will be stored as c002671d). My toExternal and toInternal methods for appropriate conversion seems to be working well for query results, but however when faceting on this field it returns the raw strings. in other words the query response would have 192.2.103.29, but facet on the field would return int name=c002671d1/int Why are these methods not used by the faceting component to convert the resulting values? Thanks Viswa
Userdefined Field type - Faceting
Hello, We implemented an IP-Addr field type which internally stored the ips as hex-ed string (e.g. 192.2.103.29 will be stored as c002671d). My toExternal and toInternal methods for appropriate conversion seems to be working well for query results, but however when faceting on this field it returns the raw strings. in other words the query response would have 192.2.103.29, but facet on the field would return int name=c002671d1/int Why are these methods not used by the faceting component to convert the resulting values? Thanks Viswa
Empty value/string matching
Folks,Am trying to query documents which have no values present, I have used the following constructs and it doesn't seem to work on the solr dev tip (as of 09/22) or the 1.4 builds.1. (*:* AND -FieldName[* TO *]) - returns no documents, parsedquery was +MatchAllDocsQuery(*:*) -FieldName:[* TO *]2. -FieldName:[* TO *] - returns no documents, parsedquery was -FieldName:[* TO *]3. FieldName: - returns no documents, parsedquery was empty (str name=parsedquery/)The field is type string, using the LuceneQParser, I have also tried to see if FieldName:[* TO *] if the documents with no terms are ignored and didn't seem to be the case, the result set was everything.Any help would be appreciated.-Viswa
RE: Empty value/string matching
Yes I do have a couple of documents with no values and one with an empty string. Find below the output of a facet on the fieldName. ThanksViswa int name=2/intint name=CASTIGO.4302/intint name=GDOGPRODY.4242/intint name=QMAGIC.4122/intint name= 1/int Date: Sat, 20 Nov 2010 15:29:06 -0500 Subject: Re: Empty value/string matching From: erickerick...@gmail.com To: solr-user@lucene.apache.org Are you absolutely sure your documents really don't have any values for FieldName? Because your results are perfectly correct if every doc has a value for FieldName. Or are you saying there no such field as FieldName? Best Erick On Sat, Nov 20, 2010 at 3:12 PM, Viswa S svis...@hotmail.com wrote: Folks,Am trying to query documents which have no values present, I have used the following constructs and it doesn't seem to work on the solr dev tip (as of 09/22) or the 1.4 builds.1. (*:* AND -FieldName[* TO *]) - returns no documents, parsedquery was +MatchAllDocsQuery(*:*) -FieldName:[* TO *]2. -FieldName:[* TO *] - returns no documents, parsedquery was -FieldName:[* TO *]3. FieldName: - returns no documents, parsedquery was empty (str name=parsedquery/)The field is type string, using the LuceneQParser, I have also tried to see if FieldName:[* TO *] if the documents with no terms are ignored and didn't seem to be the case, the result set was everything.Any help would be appreciated.-Viswa
RE: Empty value/string matching
Erick, Thanks for the quick response. The output i showed is on a test instance i created to simulate this issue. I intentionally tried to create documents with no values by creating xml nodes with field name=fieldName/field, but having values in the other fields in a document. Are you saying that there is no way have a field with no value?, with text fields they seem to make sense than for string?. You are right on fieldName:[* TO *] results, which basically returned all the documents which included the couple of documents in question. -Viswa Date: Sat, 20 Nov 2010 17:20:53 -0500 Subject: Re: Empty value/string matching From: erickerick...@gmail.com To: solr-user@lucene.apache.org I don't think that's correct. The documents wouldn't be showing up in the facets if they had no value for the field. So I think you're being mislead by the printout from the faceting. Perhaps you have unprintable characters in there or some such. Certainly the name: is actually a value, admittedly just a space. As for the other, I suspect something similar. What results do you get back when you just search for FieldName:[* TO *]? I'm betting you get all the docs back, but I've been very wrong before. Best Erick On Sat, Nov 20, 2010 at 5:02 PM, Viswa S svis...@hotmail.com wrote: Yes I do have a couple of documents with no values and one with an empty string. Find below the output of a facet on the fieldName. ThanksViswa int name=2/intint name=CASTIGO.4302/intint name=GDOGPRODY.4242/intint name=QMAGIC.4122/intint name= 1/int Date: Sat, 20 Nov 2010 15:29:06 -0500 Subject: Re: Empty value/string matching From: erickerick...@gmail.com To: solr-user@lucene.apache.org Are you absolutely sure your documents really don't have any values for FieldName? Because your results are perfectly correct if every doc has a value for FieldName. Or are you saying there no such field as FieldName? Best Erick On Sat, Nov 20, 2010 at 3:12 PM, Viswa S svis...@hotmail.com wrote: Folks,Am trying to query documents which have no values present, I have used the following constructs and it doesn't seem to work on the solr dev tip (as of 09/22) or the 1.4 builds.1. (*:* AND -FieldName[* TO *]) - returns no documents, parsedquery was +MatchAllDocsQuery(*:*) -FieldName:[* TO *]2. -FieldName:[* TO *] - returns no documents, parsedquery was -FieldName:[* TO *]3. FieldName: - returns no documents, parsedquery was empty (str name=parsedquery/)The field is type string, using the LuceneQParser, I have also tried to see if FieldName:[* TO *] if the documents with no terms are ignored and didn't seem to be the case, the result set was everything.Any help would be appreciated.-Viswa
Solr Negative query
Dear Solr/Lucene gurus, I have run into a weird issue trying use a negative condition in my query. Parser:StandardQueryParserMy Query: Field1:Val1 NOT Field2:Val2Resolved as: Field1:Val1 -Field2:Val2 The above query never returns any document, no matter how we use a paranthesis. I did see some suggestions on LuceneQParser to use something like:*:* Field1:Val1 -Field2:Val2 This seems to return some documents, however it seems to ignore the first condition (Field1:Val1), Please help. ThanksVis
RE: Solr Negative query
Apologies for starting a new thread again, my mailing list subscription didn't finalize till later than Yonik's response. Using Field1:Val1 AND (*:* NOT Field2:Val2) works, thanks. Does my original query Field1:Value1 AND (NOT Field2:Val2) fall into need the *:* trick if all of the clauses of a boolean query are negative case?. It does seem to have a positive and negative match attributes? can you please elaborate. Thanks for quick responses, you guys are awesome. From:Yonik Seeley (yon...@lucidimagination.com)Date:Nov 14, 2010 9:08:32 am wrote: On Sun, Nov 14, 2010 at 4:17 AM, Leonardo Menezes leon...@googlemail.com wrote:try Field1:Val1 AND (*:* NOT Field2:Val2), that shoud work ok That should be equivalent to Field1:Val1 -Field2:Val2 You only need the *:* trick if all of the clauses of a boolean query are negative. -Yonik http://www.lucidimagination.com To: solr-user@lucene.apache.org Subject: Solr Negative query Date: Sun, 14 Nov 2010 13:32:51 +0530 Dear Solr/Lucene gurus, I have run into a weird issue trying use a negative condition in my query. Parser:StandardQueryParserMy Query: Field1:Val1 NOT Field2:Val2Resolved as: Field1:Val1 -Field2:Val2 The above query never returns any document, no matter how we use a paranthesis. I did see some suggestions on LuceneQParser to use something like:*:* Field1:Val1 -Field2:Val2 This seems to return some documents, however it seems to ignore the first condition (Field1:Val1), Please help. ThanksVis
