Re: Unparseable date
I indexed my docs with field : field name=order_dt1995-12-31T23:59:59.000Z/field But when i try to search on that field : order_dt:1995-12-31T23:59:59.000Z , I get an exception : Mar 11, 2008 4:13:55 PM org.apache.solr.core.SolrException log SEVERE: org.apache.solr.core.SolrException: Invalid Date String:'1995-12-31T23' at org.apache.solr.schema.DateField.toInternal(DateField.java:108) at org.apache.solr.schema.FieldType$DefaultAnalyzer$1.next(FieldType.java:298) at org.apache.lucene.queryParser.QueryParser.getFieldQuery(QueryParser.java:437) at org.apache.solr.search.SolrQueryParser.getFieldQuery(SolrQueryParser.java:78) at org.apache.lucene.queryParser.QueryParser.Term(QueryParser.java:1092) at org.apache.lucene.queryParser.QueryParser.Clause(QueryParser.java:979) at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:907) at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:896) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:146) Am I missing anything ? Thanks, Monica. Daniel Andersson-5 wrote: On Mar 5, 2008, at 11:08 PM, Chris Hostetter wrote: It's .000 not :00 ... 2008-02-12T15:02:06.000Z but like i said: that stack trace is odd, the time doesn't seem like it actually comes from any query params, it looks like it's coming from a previously indexed doc. To work arround this you may need to reindex all of your docs with those optional milliseconds. Ah, re-indexing now. Thanks for your help! / d -- View this message in context: http://www.nabble.com/Unparseable-date-tp15854401p15994506.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unparseable date
: I indexed my docs with field : field : name=order_dt1995-12-31T23:59:59.000Z/field : But when i try to search on that field : order_dt:1995-12-31T23:59:59.000Z , : I get an exception : : Mar 11, 2008 4:13:55 PM org.apache.solr.core.SolrException log : SEVERE: org.apache.solr.core.SolrException: Invalid Date : String:'1995-12-31T23' : is a special character for the query parser, so it either needs to be escaped or the date needs to be quoted... order_dt:1995-12-31T23:59:59.000Z this isn't something most people typically need to worry about, because dates are typically only queried using ranges... order_dt:[1995-12-31T23:59:59.000Z TO *] -Hoss
Unparseable date
Hi people I've got a date(time] indexed with every document, defined as: field name=datetime_found type=date indexed=true multiValued=false / According to the schema.xml-file The format for this date field is of the form 1995-12-31T23:59:59Z. Yet I'm getting the following error on SOME queries: Mar 5, 2008 10:32:53 AM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at org.apache.solr.schema.DateField.toObject(DateField.java: 173) at org.apache.solr.schema.DateField.toObject(DateField.java:83) at org.apache.solr.update.DocumentBuilder.loadStoredFields (DocumentBuilder.java:285) at com.pjaol.search.solr.component.LocalSolrQueryComponent.luceneDocToSolrD oc(LocalSolrQueryComponent.java:403) at com.pjaol.search.solr.component.LocalSolrQueryComponent.mergeResultsDist ances(LocalSolrQueryComponent.java:363) at com.pjaol.search.solr.component.LocalSolrQueryComponent.process (LocalSolrQueryComponent.java:305) at org.apache.solr.handler.component.SearchHandler.handleRequestBody (SearchHandler.java:158) at org.apache.solr.handler.RequestHandlerBase.handleRequest (RequestHandlerBase.java:118) at org.apache.solr.core.SolrCore.execute(SolrCore.java:944) at org.apache.solr.servlet.SolrDispatchFilter.execute (SolrDispatchFilter.java:326) at org.apache.solr.servlet.SolrDispatchFilter.doFilter (SolrDispatchFilter.java:278) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle (ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle (SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle (SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle (ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle (WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle (ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle (HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle (HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest (HttpConnection.java:502) at org.mortbay.jetty.HttpConnection $RequestHandler.headerComplete(HttpConnection.java:821) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) at org.mortbay.jetty.HttpParser.parseAvailable (HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle (HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run (SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run (BoundedThreadPool.java:442) Caused by: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at java.text.DateFormat.parse(DateFormat.java:335) at org.apache.solr.schema.DateField.toObject(DateField.java: 170) ... 27 more Could this be because we're using 24h instead of 12h? (the example seems to imply that 24h is what should be used though) Thanks in advance! Kind regards, Daniel
Re: Unparseable date
Solr does use 24 hour dates. Are you positive there are no extraneous characters at the end of your date string such as carriage returns, spaces, or tabs? I have the same format in the code I've written and have never had a date parsing problem (yet). Ryan Grange, IT Manager DollarDays International, LLC [EMAIL PROTECTED] 480-922-8155 x106 Daniel Andersson wrote: Hi people I've got a date(time] indexed with every document, defined as: field name=datetime_found type=date indexed=true multiValued=false / According to the schema.xml-file The format for this date field is of the form 1995-12-31T23:59:59Z. Yet I'm getting the following error on SOME queries: Mar 5, 2008 10:32:53 AM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at org.apache.solr.schema.DateField.toObject(DateField.java:173) at org.apache.solr.schema.DateField.toObject(DateField.java:83) at org.apache.solr.update.DocumentBuilder.loadStoredFields(DocumentBuilder.java:285) at com.pjaol.search.solr.component.LocalSolrQueryComponent.luceneDocToSolrDoc(LocalSolrQueryComponent.java:403) at com.pjaol.search.solr.component.LocalSolrQueryComponent.mergeResultsDistances(LocalSolrQueryComponent.java:363) at com.pjaol.search.solr.component.LocalSolrQueryComponent.process(LocalSolrQueryComponent.java:305) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:158) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:118) at org.apache.solr.core.SolrCore.execute(SolrCore.java:944) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:326) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:278) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:821) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) Caused by: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at java.text.DateFormat.parse(DateFormat.java:335) at org.apache.solr.schema.DateField.toObject(DateField.java:170) ... 27 more Could this be because we're using 24h instead of 12h? (the example seems to imply that 24h is what should be used though) Thanks in advance! Kind regards, Daniel
Re: Unparseable date
: According to the schema.xml-file The format for this date field is of the : form 1995-12-31T23:59:59Z. : : Yet I'm getting the following error on SOME queries: : : Mar 5, 2008 10:32:53 AM org.apache.solr.common.SolrException log : SEVERE: java.lang.RuntimeException: java.text.ParseException: Unparseable : date: 2008-02-12T15:02:06Z : at org.apache.solr.schema.DateField.toObject(DateField.java:173) : at org.apache.solr.schema.DateField.toObject(DateField.java:83) : at org.apache.solr.update.DocumentBuilder.loadStoredFields : (DocumentBuilder.java:285) : at : com.pjaol.search.solr.component.LocalSolrQueryComponent.luceneDocToSolrD : oc(LocalSolrQueryComponent.java:403) : at : com.pjaol.search.solr.component.LocalSolrQueryComponent.mergeResultsDist : ances(LocalSolrQueryComponent.java:363) Hmmm... this seems related to SOLR-470 in the sense that it has to do with reusing the same SimpleDateParser for more things then it was ment for ... looking at the current code for DateField.toObject(Fieldable) it seems inheriently broken, attempting to parse a string right after concating 'Z' on the end even though the parser expects the Z to already be gone -- i'm not sure how this could path could *ever* work, regardless of the input. ugh. just to clarify, this stack trace doesn't look like you are actually doing a query, it seems like it's happening during an update of some kind (using DocumentBuilder.loadStoredFields to populate a SolrDocument from a Document) .. can you elaborate on what you are doing here? -Hoss
Re: Unparseable date
It's stored in MySQL (datatype: datetime), then extracted and run through the following code: $date = substr($date, 0, 10) . T . substr($date, 11) . Z; If there was some odd chars at the end, I would have assumed it would have been included in the error message. SEVERE: java.lang.RuntimeException: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at org.apache.solr.schema.DateField.toObject(DateField.java: 173) Seems to imply that that's all it's getting.. Cheers, Daniel On Mar 5, 2008, at 5:40 PM, Ryan Grange wrote: Solr does use 24 hour dates. Are you positive there are no extraneous characters at the end of your date string such as carriage returns, spaces, or tabs? I have the same format in the code I've written and have never had a date parsing problem (yet). Ryan Grange, IT Manager DollarDays International, LLC [EMAIL PROTECTED] 480-922-8155 x106 Daniel Andersson wrote: Hi people I've got a date(time] indexed with every document, defined as: field name=datetime_found type=date indexed=true multiValued=false / According to the schema.xml-file The format for this date field is of the form 1995-12-31T23:59:59Z. Yet I'm getting the following error on SOME queries: Mar 5, 2008 10:32:53 AM org.apache.solr.common.SolrException log SEVERE: java.lang.RuntimeException: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at org.apache.solr.schema.DateField.toObject (DateField.java:173) at org.apache.solr.schema.DateField.toObject (DateField.java:83) at org.apache.solr.update.DocumentBuilder.loadStoredFields (DocumentBuilder.java:285) at com.pjaol.search.solr.component.LocalSolrQueryComponent.luceneDocToSo lrDoc(LocalSolrQueryComponent.java:403) at com.pjaol.search.solr.component.LocalSolrQueryComponent.mergeResultsD istances(LocalSolrQueryComponent.java:363) at com.pjaol.search.solr.component.LocalSolrQueryComponent.process (LocalSolrQueryComponent.java:305) at org.apache.solr.handler.component.SearchHandler.handleRequestBody (SearchHandler.java:158) at org.apache.solr.handler.RequestHandlerBase.handleRequest (RequestHandlerBase.java:118) at org.apache.solr.core.SolrCore.execute(SolrCore.java:944) at org.apache.solr.servlet.SolrDispatchFilter.execute (SolrDispatchFilter.java:326) at org.apache.solr.servlet.SolrDispatchFilter.doFilter (SolrDispatchFilter.java:278) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle (ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle (SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle (SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle (ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle (WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle (ContextHandlerCollection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle (HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle (HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest (HttpConnection.java:502) at org.mortbay.jetty.HttpConnection $RequestHandler.headerComplete(HttpConnection.java:821) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java: 513) at org.mortbay.jetty.HttpParser.parseAvailable (HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle (HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run (SocketConnector.java:226) at org.mortbay.thread.BoundedThreadPool$PoolThread.run (BoundedThreadPool.java:442) Caused by: java.text.ParseException: Unparseable date: 2008-02-12T15:02:06Z at java.text.DateFormat.parse(DateFormat.java:335) at org.apache.solr.schema.DateField.toObject (DateField.java:170) ... 27 more Could this be because we're using 24h instead of 12h? (the example seems to imply that 24h is what should be used though) Thanks in advance! Kind regards, Daniel
Re: Unparseable date
: looking at the current code for DateField.toObject(Fieldable) it seems : inheriently broken, attempting to parse a string right after concating 'Z' : on the end even though the parser expects the Z to already be gone -- i'm : not sure how this could path could *ever* work, regardless of the input. Duh, i forgot a key aspect of DateFormat.parse(String): The method may not use the entire text of the given string. ... which is why the code can work sometimes, the parser will happily ignore the 'Z' ... the exception is about the fact that there are no milliseconds (which are suppose to be optional) so this really is the exact same bug as SOLR-470 ... the problem is just more significant then i thought -- I assumed the only way to trigger this was a DateMath expression that didn't have milliseconds, but DateField.toObject(Fieldable) has the same bug so direct usage of it (by code like DocumentBuilder.loadStoredFields) will also cause this problem if the orriginal date didn't have any milliseconds. -Hoss
Re: Unparseable date
On Mar 5, 2008, at 10:46 PM, Chris Hostetter wrote: : According to the schema.xml-file The format for this date field is of the : form 1995-12-31T23:59:59Z. : : Yet I'm getting the following error on SOME queries: : : Mar 5, 2008 10:32:53 AM org.apache.solr.common.SolrException log : SEVERE: java.lang.RuntimeException: java.text.ParseException: Unparseable : date: 2008-02-12T15:02:06Z : at org.apache.solr.schema.DateField.toObject (DateField.java:173) : at org.apache.solr.schema.DateField.toObject (DateField.java:83) : at org.apache.solr.update.DocumentBuilder.loadStoredFields : (DocumentBuilder.java:285) : at : com.pjaol.search.solr.component.LocalSolrQueryComponent.luceneDocToSol rD : oc(LocalSolrQueryComponent.java:403) : at : com.pjaol.search.solr.component.LocalSolrQueryComponent.mergeResultsDi st : ances(LocalSolrQueryComponent.java:363) Hmmm... this seems related to SOLR-470 in the sense that it has to do with reusing the same SimpleDateParser for more things then it was ment for ... looking at the current code for DateField.toObject(Fieldable) it seems inheriently broken, attempting to parse a string right after concating 'Z' on the end even though the parser expects the Z to already be gone -- i'm not sure how this could path could *ever* work, regardless of the input. ugh. just to clarify, this stack trace doesn't look like you are actually doing a query, it seems like it's happening during an update of some kind (using DocumentBuilder.loadStoredFields to populate a SolrDocument from a Document) .. can you elaborate on what you are doing here? It is a query, which is run through LocalSolr/LocalLucene CC'ing them in, since it seems you're suggesting that they might be re-using something incorrectly.. ? Cheers, Daniel
Re: Unparseable date
On Mar 5, 2008, at 10:57 PM, Chris Hostetter wrote: : looking at the current code for DateField.toObject(Fieldable) it seems : inheriently broken, attempting to parse a string right after concating 'Z' : on the end even though the parser expects the Z to already be gone -- i'm : not sure how this could path could *ever* work, regardless of the input. Duh, i forgot a key aspect of DateFormat.parse(String): The method may not use the entire text of the given string. ... which is why the code can work sometimes, the parser will happily ignore the 'Z' ... the exception is about the fact that there are no milliseconds (which are suppose to be optional) so this really is the exact same bug as SOLR-470 ... the problem is just more significant then i thought -- I assumed the only way to trigger this was a DateMath expression that didn't have milliseconds, but DateField.toObject(Fieldable) has the same bug so direct usage of it (by code like DocumentBuilder.loadStoredFields) will also cause this problem if the orriginal date didn't have any milliseconds. So if I add :00 to every time, it should be fine? ie 2008-02-12T15:02:06:00Z instead of 2008-02-12T15:02:06Z Cheers, Daniel
Re: Unparseable date
: So if I add :00 to every time, it should be fine? : ie : 2008-02-12T15:02:06:00Z instead of 2008-02-12T15:02:06Z It's .000 not :00 ... 2008-02-12T15:02:06.000Z but like i said: that stack trace is odd, the time doesn't seem like it actually comes from any query params, it looks like it's coming from a previously indexed doc. To work arround this you may need to reindex all of your docs with those optional milliseconds. -Hoss
