RE: Invalid Date in Date Math String
Hi Grant,
Thank you for the response!
The Sitecore version is 9.0 update 2 and we already have applied the patch you
mentioned below (which I was referring earlier) but still the issue (random
times) would be observed sometimes.
Note: We have to restart the solr-zk ensemble + Sitecore VM to have a
successful rebuild without this error.
Thanks
Ritesh
-Original Message-
From: Grant Killian
Sent: Wednesday, August 21, 2019 6:12 AM
To: solr-user@lucene.apache.org
Subject: RE: Invalid Date in Date Math String
I happened to see this, Ritesh, so let me offer that this issue (#178247) is
addressed in the Sitecore 9 version:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev.sitecore.net%2FDownloads%2FSitecore%2520Experience%2520Platform%2F90%2FSitecore%2520Experience%2520Platform%252090%2520Update2%2FRelease%2520Notesdata=02%7C01%7Cv-kurite%40microsoft.com%7C63a7c4eb202c4debc1d908d726392e39%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C637019899347402934sdata=7CWVpIIIpketown%2BwomAo%2FMa15ceteUZIKwmeYdo4sw%3Dreserved=0.
That fix, however, resolves the issue ONLY for Sitecore specific DateTime
fields (__smallcreateddate, __smallupdateddate). Apparently, an additional
patch is necessary to address other DateTime fields with Solr. This patch
linked here may resolve the issue:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsitecore.box.com%2Fs%2Fy4oct5rpf1022hrez3n4gb5fdkc7obypdata=02%7C01%7Cv-kurite%40microsoft.com%7C63a7c4eb202c4debc1d908d726392e39%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C637019899347402934sdata=OT4RtWsZkJISG26Ow4Vaq11Y6yl45fs1b7c459E4Q8Q%3Dreserved=0.
Otherwise, open a support ticket with Sitecore and they'll get you squared
away.
Thanks,
Grant
-Original Message-
From: Ritesh Kumar (Avanade)
Sent: Tuesday, August 20, 2019 7:39 PM
To: solr-user@lucene.apache.org
Subject: RE: Invalid Date in Date Math String
Thanks Eric for the response!
Regards
Ritesh
-Original Message-
From: Erick Erickson
Sent: Tuesday, August 20, 2019 4:34 PM
To: solr-user@lucene.apache.org
Subject: Re: Invalid Date in Date Math String
You probably need to talk to the SiteCore people. The Apache project doesn’t
have much insight into Sitecore, nor does it maintain the .net version of a
Solr client…
Best,
Erick
> On Aug 20, 2019, at 7:16 PM, Ritesh Kumar (Avanade)
> wrote:
>
> Hello,
>
> We have been using solr 6.6.3 with Sitecore 9, however currently we seem to
> be observing the below issue while rebuilding the index "Invalid Date in Date
> Math String"
> We already applied the patch located at the below location but it did not fix
> the issue:
> https://gith
> ub.com%2FSitecoreSupport%2FSitecore.Support.178247%2Freleases%2Ftag%2F
> 8.2.2.0data=02%7C01%7Cv-kurite%40microsoft.com%7Cf8ba7751ef5b413c
> c3de08d725c6da9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63701940
> 8314524415sdata=Y6rsPNdcpjIvgPFAGVsLYFJqW01fjkcGsxUqKloyvYc%3D
> p;reserved=0
>
> Please let me know if anyone was able to fix this issue with any other
> method. (Note: I have updated my Sitecore support ticket as well with
> the same details)
>
> 5980 22:10:20 WARN [Index=xxx _index] Crawler : AddRecursive
> DoItemAdd failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
> Exception: SolrNet.Exceptions.SolrConnectionException
> Message:name="responseHeader">400 name="QTime">325 name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException
> name="root-error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException name="msg">7 Async exceptions during distributed update:
> Error from server at x.x.x.x:8983/solr/_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/x
RE: Invalid Date in Date Math String
I happened to see this, Ritesh, so let me offer that this issue (#178247) is
addressed in the Sitecore 9 version:
https://dev.sitecore.net/Downloads/Sitecore%20Experience%20Platform/90/Sitecore%20Experience%20Platform%2090%20Update2/Release%20Notes.
That fix, however, resolves the issue ONLY for Sitecore specific DateTime
fields (__smallcreateddate, __smallupdateddate). Apparently, an additional
patch is necessary to address other DateTime fields with Solr. This patch
linked here may resolve the issue:
https://sitecore.box.com/s/y4oct5rpf1022hrez3n4gb5fdkc7obyp. Otherwise, open a
support ticket with Sitecore and they'll get you squared away.
Thanks,
Grant
-Original Message-
From: Ritesh Kumar (Avanade)
Sent: Tuesday, August 20, 2019 7:39 PM
To: solr-user@lucene.apache.org
Subject: RE: Invalid Date in Date Math String
Thanks Eric for the response!
Regards
Ritesh
-Original Message-
From: Erick Erickson
Sent: Tuesday, August 20, 2019 4:34 PM
To: solr-user@lucene.apache.org
Subject: Re: Invalid Date in Date Math String
You probably need to talk to the SiteCore people. The Apache project doesn’t
have much insight into Sitecore, nor does it maintain the .net version of a
Solr client…
Best,
Erick
> On Aug 20, 2019, at 7:16 PM, Ritesh Kumar (Avanade)
> wrote:
>
> Hello,
>
> We have been using solr 6.6.3 with Sitecore 9, however currently we seem to
> be observing the below issue while rebuilding the index "Invalid Date in Date
> Math String"
> We already applied the patch located at the below location but it did not fix
> the issue:
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2FSitecoreSupport%2FSitecore.Support.178247%2Freleases%2Ftag%2F
> 8.2.2.0data=02%7C01%7Cv-kurite%40microsoft.com%7Cf8ba7751ef5b413c
> c3de08d725c6da9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63701940
> 8314524415sdata=Y6rsPNdcpjIvgPFAGVsLYFJqW01fjkcGsxUqKloyvYc%3D
> p;reserved=0
>
> Please let me know if anyone was able to fix this issue with any other
> method. (Note: I have updated my Sitecore support ticket as well with
> the same details)
>
> 5980 22:10:20 WARN [Index=xxx _index] Crawler : AddRecursive
> DoItemAdd failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
> Exception: SolrNet.Exceptions.SolrConnectionException
> Message:name="responseHeader">400 name="QTime">325 name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException
> name="root-error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException name="msg">7 Async exceptions during distributed update:
> Error from server at x.x.x.x:8983/solr/_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/_index_shard2_replica2/update?update.distrib
> =TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%
> _index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20180730T164926Z'
> Error from server at
> x.x.x.x:8983/solr/x_index_shard1_replica2: Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=
> TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20170901T183400Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=
> TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Ma
RE: Invalid Date in Date Math String
Thanks Eric for the response!
Regards
Ritesh
-Original Message-
From: Erick Erickson
Sent: Tuesday, August 20, 2019 4:34 PM
To: solr-user@lucene.apache.org
Subject: Re: Invalid Date in Date Math String
You probably need to talk to the SiteCore people. The Apache project doesn’t
have much insight into Sitecore, nor does it maintain the .net version of a
Solr client…
Best,
Erick
> On Aug 20, 2019, at 7:16 PM, Ritesh Kumar (Avanade)
> wrote:
>
> Hello,
>
> We have been using solr 6.6.3 with Sitecore 9, however currently we seem to
> be observing the below issue while rebuilding the index "Invalid Date in Date
> Math String"
> We already applied the patch located at the below location but it did not fix
> the issue:
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2FSitecoreSupport%2FSitecore.Support.178247%2Freleases%2Ftag%2F
> 8.2.2.0data=02%7C01%7Cv-kurite%40microsoft.com%7Cf8ba7751ef5b413c
> c3de08d725c6da9f%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C63701940
> 8314524415sdata=Y6rsPNdcpjIvgPFAGVsLYFJqW01fjkcGsxUqKloyvYc%3D
> p;reserved=0
>
> Please let me know if anyone was able to fix this issue with any other
> method. (Note: I have updated my Sitecore support ticket as well with
> the same details)
>
> 5980 22:10:20 WARN [Index=xxx _index] Crawler : AddRecursive
> DoItemAdd failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
> Exception: SolrNet.Exceptions.SolrConnectionException
> Message:name="responseHeader">400 name="QTime">325 name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException
> name="root-error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException name="msg">7 Async exceptions during distributed update:
> Error from server at x.x.x.x:8983/solr/_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/_index_shard2_replica2/update?update.distrib
> =TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%
> _index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20180730T164926Z'
> Error from server at
> x.x.x.x:8983/solr/x_index_shard1_replica2: Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=
> TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20170901T183400Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=
> TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20171026T181027Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=
> TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20171028T07Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard1_replica2:
> Bad Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=
> TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2
> Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20171004T07Z'
>
Re: Invalid Date in Date Math String
You probably need to talk to the SiteCore people. The Apache project doesn’t
have much insight into Sitecore, nor does it maintain the .net version of a
Solr client…
Best,
Erick
> On Aug 20, 2019, at 7:16 PM, Ritesh Kumar (Avanade)
> wrote:
>
> Hello,
>
> We have been using solr 6.6.3 with Sitecore 9, however currently we seem to
> be observing the below issue while rebuilding the index "Invalid Date in Date
> Math String"
> We already applied the patch located at the below location but it did not fix
> the issue:
> https://github.com/SitecoreSupport/Sitecore.Support.178247/releases/tag/8.2.2.0
>
> Please let me know if anyone was able to fix this issue with any other
> method. (Note: I have updated my Sitecore support ticket as well with the
> same details)
>
> 5980 22:10:20 WARN [Index=xxx _index] Crawler : AddRecursive DoItemAdd
> failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
> Exception: SolrNet.Exceptions.SolrConnectionException
> Message:
>
> 400 name="QTime">325 name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.common.SolrException name="root-error-class">java.time.format.DateTimeParseException name="error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException
> name="root-error-class">org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException name="msg">7 Async exceptions during distributed update:
> Error from server at x.x.x.x:8983/solr/_index_shard2_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20180730T164926Z'
> Error from server at x.x.x.x:8983/solr/x_index_shard1_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20170901T183400Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20171026T181027Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20171028T07Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard1_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20171004T07Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
> Remote error message: Invalid Date in Date Math String:'20170314T00Z'
> Error from server at x.x.x.x:8983/solr/xxx_index_shard1_replica2: Bad
> Request
>
> request:
> x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=TOLEADERdistrib.from=https
Invalid Date in Date Math String
Hello,
We have been using solr 6.6.3 with Sitecore 9, however currently we seem to be
observing the below issue while rebuilding the index "Invalid Date in Date Math
String"
We already applied the patch located at the below location but it did not fix
the issue:
https://github.com/SitecoreSupport/Sitecore.Support.178247/releases/tag/8.2.2.0
Please let me know if anyone was able to fix this issue with any other method.
(Note: I have updated my Sitecore support ticket as well with the same details)
5980 22:10:20 WARN [Index=xxx _index] Crawler : AddRecursive DoItemAdd
failed - {F8057A68-6A5A-43ED-8781-BFA36D4DF3EE}
Exception: SolrNet.Exceptions.SolrConnectionException
Message:
400325org.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.common.SolrExceptionjava.time.format.DateTimeParseExceptionorg.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncExceptionorg.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException7 Async exceptions during distributed update:
Error from server at x.x.x.x:8983/solr/_index_shard2_replica2: Bad
Request
request:
x.x.x.x:8983/solr/_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math String:'20180730T164926Z'
Error from server at x.x.x.x:8983/solr/x_index_shard1_replica2: Bad
Request
request:
x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math String:'20170901T183400Z'
Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2: Bad
Request
request:
x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math String:'20171026T181027Z'
Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2: Bad
Request
request:
x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math String:'20171028T07Z'
Error from server at x.x.x.x:8983/solr/xxx_index_shard1_replica2: Bad
Request
request:
x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math String:'20171004T07Z'
Error from server at x.x.x.x:8983/solr/xxx_index_shard2_replica2: Bad
Request
request:
x.x.x.x:8983/solr/xxx_index_shard2_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math String:'20170314T00Z'
Error from server at x.x.x.x:8983/solr/xxx_index_shard1_replica2: Bad
Request
request:
x.x.x.x:8983/solr/xxx_index_shard1_replica2/update?update.distrib=TOLEADERdistrib.from=https%3A%2F%2F40.113.227.147%3A8983%2Fsolr%2Fxxx_index_shard1_replica3%2Fwt=javabinversion=2
Remote error message: Invalid Date in Date Math
String:'20170626T00Z'400
Source: SolrNet
at SolrNet.Impl.SolrConnection.PostStream(String relativeUrl, String
contentType, Stream content, IEnumerable`1 parameters)
at SolrNet.Impl.SolrConnection.Post(String relativeUrl, String s)
at SolrNet.Impl.SolrBasicServer`1.SendAndParseHeader(ISolrCommand cmd)
at
Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddRange(IEnumerable`1
group, Int32 groupSize)
at
Sitecore.ContentSearch.SolrProvider.SolrBatchUpdateContext.AddDocument(Object
itemToAdd, IExecutionContext[] executionContexts)
at
Sitecore.ContentSearch.SolrProvider.SolrIndexOperations.ApplyPermissionsThenIndex(IProviderUpdateContext
context, IIndexable version)
at Sitecore.ContentSearch.SitecoreItemCrawler.DoAdd(IProviderUpdateContext
context, SitecoreIndexableItem indexable)
at Sitecore.ContentSearch.HierarchicalDataCrawler`1.CrawlItem(T indexable,
IProviderUpdateContext context, CrawlState`1 state)
Nested Exception
Exception: System.Net.WebException
Message: The remote server returned an e
Invalid Date Math Strings silently fail in q, not in fq
Hello,
I encountered an issue where invalid dates throw an exception when they
are in an fq parameter but are silently dropped when in q. For example:
{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":4,
"params":{
"q":"visible_date:[2019-01-01T00:00:00Z TO 2019-12-31T23:59:99Z]",
"defType":"edismax",
"df":"text",
"fl":"visible_date",
"wt":"json",
"debugQuery":"on",
"stopwords":"true"}},
"response":{"numFound":0,"start":0,"docs":[]
},
"debug":{
"rawquerystring":"visible_date:[2019-01-01T00:00:00Z TO
2019-12-31T23:59:99Z]",
"querystring":"visible_date:[2019-01-01T00:00:00Z TO
2019-12-31T23:59:99Z]",
"parsedquery":"+()",
"parsedquery_toString":"+()",
"explain":{},
"QParser":"ExtendedDismaxQParser",
"altquerystring":null,
"boost_queries":null,
"parsed_boost_queries":[],
"boostfuncs":null,
--snip--
Above the parsedquery is empty because the timestamp
2019-12-31T23:59:99Z is invalid.
However:
{
"responseHeader":{
"zkConnected":true,
"status":400,
"QTime":2,
"params":{
"q":"*:*",
"defType":"edismax",
"df":"text",
"fl":"visible_date",
"fq":"visible_date:[\"2019-01-01T00:00:00Z\" TO
\"2019-12-31T23:59:99Z\"]",
"debugQuery":"on",
"stopwords":"true",
"_":"1557734718206"}},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","java.time.format.DateTimeParseException"],
"msg":"Invalid Date in Date Math String:'2019-12-31T23:59:99Z'",
"code":400}}
Above the date range filter is in fq, and now a DateTimeParseException
is thrown.
Is it intended behaviour that in a q parameter date filters are silently
dropped on parsing errors while in an fq parameter an exception is
thrown? Or is this a bug?
I am running Solr version 7.7.1.
--
Ronja Koistinen
IT Specialist
University of Helsinki
signature.asc
Description: OpenPGP digital signature
Date Math, NOW and filter queries
hi!,
http://lucidworks.com/blog/2012/02/23/date-math-now-and-filter-queries/
for date range query i am following above article,in this article
I try to querying fq=date:[NOW/DAY-7DAYS TO NOW/DAY], it is working fine,
when i fire query fq=date:[NOW/DAY-7DAYS TO NOW/DAY+1DAY], it is giving
below error
"fq":"initial_release_date:[NOW/DAY-7DAYS TO NOW/DAY 1DAY]",
"rows":"32"}},
"error":{
"msg":"org.apache.solr.search.SyntaxError: Cannot parse
'initial_release_date:[NOW/DAY-7DAYS TO NOW/DAY 1DAY]': Encountered \"
\"1DAY \"\" at line 1, column 47.\nWas expecting one of:\n
\"]\" ...\n\"}\" ...\n",
"code":400}}
why it is giving error
Thanks
mugeesh
--
View this message in context:
http://lucene.472066.n3.nabble.com/Date-Math-NOW-and-filter-queries-tp4240561.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Date Math, NOW and filter queries
Congratulations, you are in "Url escaping hell" ;)
the '+' sign is a URL-escape for space, which you
see in the error message.
Escape it as %2B and you should be fine.
Best,
Erick
On Tue, Nov 17, 2015 at 6:07 AM, Mugeesh Husain <muge...@gmail.com> wrote:
> hi!,
>
> http://lucidworks.com/blog/2012/02/23/date-math-now-and-filter-queries/
>
> for date range query i am following above article,in this article
>
> I try to querying fq=date:[NOW/DAY-7DAYS TO NOW/DAY], it is working fine,
>
> when i fire query fq=date:[NOW/DAY-7DAYS TO NOW/DAY+1DAY], it is giving
> below error
>
> "fq":"initial_release_date:[NOW/DAY-7DAYS TO NOW/DAY 1DAY]",
> "rows":"32"}},
> "error":{
> "msg":"org.apache.solr.search.SyntaxError: Cannot parse
> 'initial_release_date:[NOW/DAY-7DAYS TO NOW/DAY 1DAY]': Encountered \"
> \"1DAY \"\" at line 1, column 47.\nWas expecting one of:\n
> \"]\" ...\n\"}\" ...\n",
> "code":400}}
>
>
> why it is giving error
>
> Thanks
> mugeesh
>
>
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Date-Math-NOW-and-filter-queries-tp4240561.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Date Math, NOW and filter queries
On 11/17/2015 7:07 AM, Mugeesh Husain wrote:
> when i fire query fq=date:[NOW/DAY-7DAYS TO NOW/DAY+1DAY], it is giving
> below error
>
> "fq":"initial_release_date:[NOW/DAY-7DAYS TO NOW/DAY 1DAY]",
> "rows":"32"}},
> "error":{
> "msg":"org.apache.solr.search.SyntaxError: Cannot parse
> 'initial_release_date:[NOW/DAY-7DAYS TO NOW/DAY 1DAY]': Encountered \"
> \"1DAY \"\" at line 1, column 47.\nWas expecting one of:\n
> \"]\" ...\n\"}\" ...\n",
> "code":400}}
How are you sending the queries to Solr? I'm betting that you are
constructing a URL manually in your own code and sending it to Solr
directly via HTTP, rather than using a Solr library like SolrJ,
Solarium, Sunspot, etc.
https://wiki.apache.org/solr/IntegratingSolr
What's happening here is that the query is not URL escaped, which means
the URL contains an actual plus sign for the "NOW/DAY+1DAY" part of your
query. A plus sign in a URL is interpreted by the webserver as a space,
in accordance with the standards that govern HTTP. This is not a bug.
A Solr library would have taken care of all the URL escaping for
characters that require it, and the user code is typically a lot easier
to write than URL construction code.
Whatever programming language you are using to construct your queries
very likely has a function for doing URL escaping on the parameters for
your URL. You would want to be careful to only run it on the values in
your parameters, not the entire URL, or it would escape everything and
the URL would not work.
The URL escaped version of a plus sign is %2B if you want to quickly fix
this before you look into URL escaping functions or a Solr library.
Thanks,
Shawn
Re: Date Math, NOW and filter queries
: the '+' sign is a URL-escape for space, which you : see in the error message. more specifically, the error indicates that something/somewhere/somehow when you are construction your request to Solr, your HTTP request params are not getting properly escaped -- so the '+' is being sent literlay over the wire, and Solr is URL un-escaping it as whitespace. : Escape it as %2B and you should be fine. Rather then you manually "escaping" just the "+" character in your code, you should instead figure out where/how to ensure that *all* the HTTP communication you have with Solr gets properly escaped -- so you don't have this problem over and over with other params/characters. if you can tell us a bit more about how you communicate with Solr, we can try to help you with the larger problem you are having. (any decent HTTP client library should automatically escape any request param keys/values you ask it to include in the request) -Hoss http://www.lucidworks.com/
Re: Date Math, NOW and filter queries
thanks all of you, actually the problem was '+' sign is a URL-escape for space, Using %2B instead of + sign, should be fine -- View this message in context: http://lucene.472066.n3.nabble.com/Date-Math-NOW-and-filter-queries-tp4240561p4240675.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Stored values and date math query
Thanks for the insights to the query Hoss. I am going to try out the
methods you highlighted.
Thanks,
Indika
On 3 March 2013 01:19, Chris Hostetter hossman_luc...@fucit.org wrote:
: sessionAvailableNowQuery = {!edismax}(start_time:[* TO
: 1970-01-01T12:37:030Z] AND end_time:[1970-01-01T12:37:030Z +
: (_val_:order_prep_time)MINUTES TO *] AND consumers:[1 TO *] AND
: session_time_range_available:true)
you can't embed the valuef of a field inside a query string like that (the
_val_ hook only lets you embed a function in a place where a query
clause would normally be expected)
: Is it possible to retrive an integer value from the index and pass it on
it
: a date math query ? Is there anything else that needs to be in the query
?
not using the date match syntax, but you can use the function syntax to
write a function that performs a math equation in which two indexed fields
are the input - and there is an ms() function which can be used to get
the milliseconds since epoch from a date value (either in an indexed field
or a date match expression), or the diff in milliseconds between two date
values.
so you should be able to do something like this (untested) in a
function which should return a positive value if end_time is at least
order_prep_time minutes past your input date...
sub(ms(end_time, 1970-01-01T12:37:030Z),
product(order_prep_time,6))
..and then you can use that function in the frange parser to only match
documents with positive values...
fq={frange
l=0}sub(ms(end_time,1970-01-01T12:37:030Z),product(order_prep_time,6))
-Hoss
Re: Stored values and date math query
: sessionAvailableNowQuery = {!edismax}(start_time:[* TO
: 1970-01-01T12:37:030Z] AND end_time:[1970-01-01T12:37:030Z +
: (_val_:order_prep_time)MINUTES TO *] AND consumers:[1 TO *] AND
: session_time_range_available:true)
you can't embed the valuef of a field inside a query string like that (the
_val_ hook only lets you embed a function in a place where a query
clause would normally be expected)
: Is it possible to retrive an integer value from the index and pass it on it
: a date math query ? Is there anything else that needs to be in the query ?
not using the date match syntax, but you can use the function syntax to
write a function that performs a math equation in which two indexed fields
are the input - and there is an ms() function which can be used to get
the milliseconds since epoch from a date value (either in an indexed field
or a date match expression), or the diff in milliseconds between two date
values.
so you should be able to do something like this (untested) in a
function which should return a positive value if end_time is at least
order_prep_time minutes past your input date...
sub(ms(end_time, 1970-01-01T12:37:030Z),
product(order_prep_time,6))
..and then you can use that function in the frange parser to only match
documents with positive values...
fq={frange
l=0}sub(ms(end_time,1970-01-01T12:37:030Z),product(order_prep_time,6))
-Hoss
Re: Stored values and date math query
Hi Erick,
Thanks for the response. The the terms are indexed. I will need to dig
deeper and see there the issue is, it might be a correct syntax but I may
be using it incorrectly. In the meantime I have added a new column that
is end_time - order_prep_time and validate if a session is available
against that. Basically move the order_prep_time to the other side of the
equation.
Thanks,
Indika
On 28 February 2013 18:45, Erick Erickson erickerick...@gmail.com wrote:
Just to check, your order_prep_time is _indexed_ too, right? It's
a bit confusing but anything you use in your function queries will
be from indexed terms, not stored ones
Best
Erick
On Tue, Feb 26, 2013 at 4:05 PM, Indika Tantrigoda indik...@gmail.com
wrote:
Hi All,
I am trying to use a stored value in the index and add it to a date
component as follows
sessionAvailableNowQuery = {!edismax}(start_time:[* TO
1970-01-01T12:37:030Z] AND end_time:[1970-01-01T12:37:030Z +
(_val_:order_prep_time)MINUTES TO *] AND consumers:[1 TO *] AND
session_time_range_available:true)
(order_prep_time is the stored value)
However, when doing so causes the query to return incorrect results. The
query is part of a if/else query present in the fields (fl) list
_session_available_now_:if(exists(query($sessionAvailableNowQuery)), 100,
0)
Is it possible to retrive an integer value from the index and pass it on
it
a date math query ? Is there anything else that needs to be in the query
?
Thanks in advance.
Re: Stored values and date math query
Just to check, your order_prep_time is _indexed_ too, right? It's
a bit confusing but anything you use in your function queries will
be from indexed terms, not stored ones
Best
Erick
On Tue, Feb 26, 2013 at 4:05 PM, Indika Tantrigoda indik...@gmail.comwrote:
Hi All,
I am trying to use a stored value in the index and add it to a date
component as follows
sessionAvailableNowQuery = {!edismax}(start_time:[* TO
1970-01-01T12:37:030Z] AND end_time:[1970-01-01T12:37:030Z +
(_val_:order_prep_time)MINUTES TO *] AND consumers:[1 TO *] AND
session_time_range_available:true)
(order_prep_time is the stored value)
However, when doing so causes the query to return incorrect results. The
query is part of a if/else query present in the fields (fl) list
_session_available_now_:if(exists(query($sessionAvailableNowQuery)), 100,
0)
Is it possible to retrive an integer value from the index and pass it on it
a date math query ? Is there anything else that needs to be in the query ?
Thanks in advance.
Re: Date math query syntax
You can do numeric math in function queries, but not string operations, and there is no provision for doing string operations in a query using the result of an embedded function query. You haven't shown us a full query. Sounds like we have another candidate for XY Problem. First you need to tell us what you are really trying to accomplish, THEN a solution can be pursued. Take a look at the ms function query for date calculations - which is separate from so-called date-math which is a constant offset from a base date literal value. -- Jack Krupansky -Original Message- From: Indika Tantrigoda Sent: Sunday, November 18, 2012 3:11 PM To: solr-user@lucene.apache.org Subject: Date math query syntax Hi All, I am trying to use date math along with a filed value. 1970-01-01T04:32:010Z+(_val_:prep_time)MINUTES is the date math query I am using. prep_time is of type int. I am having trouble properly formatting the string resulting in exceptions. I am guessing that I am not properly formatting the query with quotes and/or parenthesis. Thanks, Indika
Re: Date math query syntax
Hi Jack,
Thanks for the response. Let me explain my question better.
I have a filter query to limit the query results,
start_time:[* TO 1970-01-01T10:55:002Z] AND end_time:[1970-01-01T10:55:002Z
TO *] AND session_time_range_available:true
and I also use the same query as a function query in fl using the
if(exists(query), x, y) where query is,
{!edismax}(start_time:[* TO 1970-01-01T10:55:002Z] AND
end_time:[1970-01-01T10:55:002Z TO *] AND session_time_range_available:true)
I need to extend the end_time range query based on the logic, similar to
end_time:[1970-01-01T10:55:002Z+30MINUTUES TO *]
How much I need to offset the end_time range is defined as an int type in
the schema. I was trying to use the field value query (_val_:field) to
retrive the end_time offset, resulting in the query being
1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES
However the query seems to have syntax errors. Or is it allowed to use the
_val_:field in such instances ?
As an extension I'll also have two offsets resulting in something similar
to
1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES+(_val_:extend_time)MINUTES
Thanks,
Indika
On 19 November 2012 09:17, Jack Krupansky j...@basetechnology.com wrote:
You can do numeric math in function queries, but not string operations,
and there is no provision for doing string operations in a query using the
result of an embedded function query.
You haven't shown us a full query.
Sounds like we have another candidate for XY Problem. First you need to
tell us what you are really trying to accomplish, THEN a solution can be
pursued.
Take a look at the ms function query for date calculations - which is
separate from so-called date-math which is a constant offset from a base
date literal value.
-- Jack Krupansky
-Original Message- From: Indika Tantrigoda
Sent: Sunday, November 18, 2012 3:11 PM
To: solr-user@lucene.apache.org
Subject: Date math query syntax
Hi All,
I am trying to use date math along with a filed
value. 1970-01-01T04:32:010Z+(_val_:**prep_time)MINUTES is the date math
query I am using. prep_time is of type int. I am having trouble properly
formatting the string resulting in exceptions. I am guessing that I am not
properly formatting the query with quotes and/or parenthesis.
Thanks,
Indika
RE: solr Invalid Date in Date Math String/Invalid Date String
: Here is the error message: : : Fieldtype: tdate (I use the default one in solr schema.xml) : Field value(Index): 2006-12-22T13:52:13Z : Field value(query): [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] : with '[' and ']' : : And it generates the result below: i think the piece of info people were overlooking here is that you are describing input to the analysis.jsp page. you can't enter arbitrary query expressions on this page -- just *values* for hte analyzer of the specifeid field (or field type) DateField doesn't know abything about the [... TO ...] syntax -- that is syntax of the query parser. all the DateField knows is that what you have entered into the Field Value text box is not a date value, and it is not a date match value either. -Hoss
RE: solr Invalid Date in Date Math String/Invalid Date String
Hi Erick Here is the error message: Fieldtype: tdate (I use the default one in solr schema.xml) Field value(Index): 2006-12-22T13:52:13Z Field value(query): [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] with '[' and ']' And it generates the result below: ---Start--- HTTP ERROR: 500 org.apache.solr.common.SolrException: Invalid Date in Date Math String:'[2006-12-22T00:00:00Z TO 2006-12' org.apache.jasper.JasperException: org.apache.solr.common.SolrException: Invalid Date in Date Math String:'[2006-12-22T00:00:00Z TO 2006-12' at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:4 02) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:464) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:358) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:367) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:268) at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java: 264) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .java:1089) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerColl ection.java:211) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:11 4) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:285) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java: 835) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:641) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:22 6) at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:4 42) Caused by: org.apache.solr.common.SolrException: Invalid Date in Date Math String:'[2006-12-22T00:00:00Z TO 2006-12' at org.apache.solr.schema.DateField.parseMath(DateField.java:158) at org.apache.solr.analysis.TrieTokenizer.reset(TrieTokenizerFactory.java:101) at org.apache.solr.analysis.TrieTokenizer.init(TrieTokenizerFactory.java:73) at org.apache.solr.analysis.TrieTokenizerFactory.create(TrieTokenizerFactory.ja va:51) at org.apache.solr.analysis.TrieTokenizerFactory.create(TrieTokenizerFactory.ja va:41) at org.apache.solr.analysis.TokenizerChain.getStream(TokenizerChain. java:69) at org.apache.solr.analysis.SolrAnalyzer.reusableTokenStream(SolrAnalyzer.java: 74) at org.apache.jsp.admin.analysis_jsp._jspService(org.apache.jsp.admin.analysis_ jsp:685) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:80) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:3 73) ... 29 more Caused by: java.text.ParseException: Unparseable date: [2006-12-22T00:00:00Z at java.text.DateFormat.parse(Unknown Source) at org.apache.solr.schema.DateField.parseDate(DateField.java:254) at org.apache.solr.schema.DateField.parseMath(DateField.java:156) ... 39 more RequestURI=/solr/i-audience.com-contacts-test/admin/analysis.jsp Powered by Jetty:// --- End --- Can you tell me what is the problem? Thank you very much in advance. -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: 2011年5月31日 9:54 下午 To: solr-user@lucene.apache.org; elleryle...@be-o.com Subject: Re: solr Invalid Date in Date Math String/Invalid Date String Can we see the results of attaching debugQuery=on to the query? That often points out the issue. I'd expect this form to work
Re: solr Invalid Date in Date Math String/Invalid Date String
Can we see the results of attaching debugQuery=on to the query? That often points out the issue. I'd expect this form to work: [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] Best Erick 2011/5/27 Ellery Leung elleryle...@be-o.com: Thank you Mike. So I understand that now. But what about the other items that have values on both size? They don't work at all. -Original Message- From: Mike Sokolov [mailto:soko...@ifactory.com] Sent: 2011年5月27日 10:23 下午 To: solr-user@lucene.apache.org Cc: alucard001 Subject: Re: solr Invalid Date in Date Math String/Invalid Date String The * endpoint for range terms wasn't implemented yet in 1.4.1 As a workaround, we use very large and very small values. -Mike On 05/27/2011 12:55 AM, alucard001 wrote: Hi all I am using SOLR 1.4.1 (according to solr info), but no matter what date field I use (date or tdate) defined in default schema.xml, I cannot do a search in solr-admin analysis.jsp: fieldtype: date(or tdate) fieldvalue(index): 2006-12-22T13:52:13Z (I type it in manually, no trailing space) fieldvalue(query): The only success case: 2006-12-22T13:52:13Z All search below are failed: * TO NOW [* TO NOW] 2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z 2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] [2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z] 2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z 2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z [2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z] [2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z] 2006-12-22T00:00:00Z TO * 2006\-12\-22T00\:00\:00Z TO * [2006-12-22T00:00:00Z TO *] [2006\-12\-22T00\:00\:00Z TO *] 2006-12-22T00:00:00.000Z TO * 2006\-12\-22T00\:00\:00\.000Z TO * [2006-12-22T00:00:00.000Z TO *] [2006\-12\-22T00\:00\:00\.000Z TO *] (vice versa) I get either: Invalid Date in Date Math String or Invalid Date String error What's wrong with it? Can anyone please help me on that? Thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-Invalid-Date-in-Date-Math-String-Inv alid-Date-String-tp2991763p2991763.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr Invalid Date in Date Math String/Invalid Date String
The * endpoint for range terms wasn't implemented yet in 1.4.1 As a workaround, we use very large and very small values. -Mike On 05/27/2011 12:55 AM, alucard001 wrote: Hi all I am using SOLR 1.4.1 (according to solr info), but no matter what date field I use (date or tdate) defined in default schema.xml, I cannot do a search in solr-admin analysis.jsp: fieldtype: date(or tdate) fieldvalue(index): 2006-12-22T13:52:13Z (I type it in manually, no trailing space) fieldvalue(query): The only success case: 2006-12-22T13:52:13Z All search below are failed: * TO NOW [* TO NOW] 2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z 2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] [2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z] 2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z 2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z [2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z] [2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z] 2006-12-22T00:00:00Z TO * 2006\-12\-22T00\:00\:00Z TO * [2006-12-22T00:00:00Z TO *] [2006\-12\-22T00\:00\:00Z TO *] 2006-12-22T00:00:00.000Z TO * 2006\-12\-22T00\:00\:00\.000Z TO * [2006-12-22T00:00:00.000Z TO *] [2006\-12\-22T00\:00\:00\.000Z TO *] (vice versa) I get either: Invalid Date in Date Math String or Invalid Date String error What's wrong with it? Can anyone please help me on that? Thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-Invalid-Date-in-Date-Math-String-Invalid-Date-String-tp2991763p2991763.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr Invalid Date in Date Math String/Invalid Date String
Thank you Mike. So I understand that now. But what about the other items that have values on both size? They don't work at all. -Original Message- From: Mike Sokolov [mailto:soko...@ifactory.com] Sent: 2011年5月27日 10:23 下午 To: solr-user@lucene.apache.org Cc: alucard001 Subject: Re: solr Invalid Date in Date Math String/Invalid Date String The * endpoint for range terms wasn't implemented yet in 1.4.1 As a workaround, we use very large and very small values. -Mike On 05/27/2011 12:55 AM, alucard001 wrote: Hi all I am using SOLR 1.4.1 (according to solr info), but no matter what date field I use (date or tdate) defined in default schema.xml, I cannot do a search in solr-admin analysis.jsp: fieldtype: date(or tdate) fieldvalue(index): 2006-12-22T13:52:13Z (I type it in manually, no trailing space) fieldvalue(query): The only success case: 2006-12-22T13:52:13Z All search below are failed: * TO NOW [* TO NOW] 2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z 2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] [2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z] 2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z 2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z [2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z] [2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z] 2006-12-22T00:00:00Z TO * 2006\-12\-22T00\:00\:00Z TO * [2006-12-22T00:00:00Z TO *] [2006\-12\-22T00\:00\:00Z TO *] 2006-12-22T00:00:00.000Z TO * 2006\-12\-22T00\:00\:00\.000Z TO * [2006-12-22T00:00:00.000Z TO *] [2006\-12\-22T00\:00\:00\.000Z TO *] (vice versa) I get either: Invalid Date in Date Math String or Invalid Date String error What's wrong with it? Can anyone please help me on that? Thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-Invalid-Date-in-Date-Math-String-Inv alid-Date-String-tp2991763p2991763.html Sent from the Solr - User mailing list archive at Nabble.com.
solr Invalid Date in Date Math String/Invalid Date String
Hi all I am using SOLR 1.4.1 (according to solr info), but no matter what date field I use (date or tdate) defined in default schema.xml, I cannot do a search in solr-admin analysis.jsp: fieldtype: date(or tdate) fieldvalue(index): 2006-12-22T13:52:13Z (I type it in manually, no trailing space) fieldvalue(query): The only success case: 2006-12-22T13:52:13Z All search below are failed: * TO NOW [* TO NOW] 2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z 2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z [2006-12-22T00:00:00Z TO 2006-12-22T23:59:59Z] [2006\-12\-22T00\:00\:00Z TO 2006\-12\-22T23\:59\:59Z] 2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z 2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z [2006-12-22T00:00:00.000Z TO 2006-12-22T23:59:59.999Z] [2006\-12\-22T00\:00\:00\.000Z TO 2006\-12\-22T23\:59\:59\.999Z] 2006-12-22T00:00:00Z TO * 2006\-12\-22T00\:00\:00Z TO * [2006-12-22T00:00:00Z TO *] [2006\-12\-22T00\:00\:00Z TO *] 2006-12-22T00:00:00.000Z TO * 2006\-12\-22T00\:00\:00\.000Z TO * [2006-12-22T00:00:00.000Z TO *] [2006\-12\-22T00\:00\:00\.000Z TO *] (vice versa) I get either: Invalid Date in Date Math String or Invalid Date String error What's wrong with it? Can anyone please help me on that? Thank you. -- View this message in context: http://lucene.472066.n3.nabble.com/solr-Invalid-Date-in-Date-Math-String-Invalid-Date-String-tp2991763p2991763.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: More Date Math: NOW/WEEK
Thank you for the clarification.
Personally, I believe it is correct for a week to start in a different
month/year and it is certainly what I would expect. As you pointed out, these
time units don't form a strictly ordered set (...yearmonthday...,
weekday...).
Complications arise from the different notions of what the first day of the
week
is (Sunday - US and Canada, Monday - Europe and ISO 8601, Saturday - Middle
East). This is handled by the locale, I think.
Further complications are introduced by week numbering, but I don't think this
applies here (http://en.wikipedia.org/wiki/Seven-day_week#Week_numbering).
Both MySQL
(http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_yearweek)
and Postgres have the notion of weeks.
All this ignores complications of 5-day or 6-day weeks, which were used in
Russia during certain parts of the last century. There might be other
historical cases or even current ones, but as you, I believe a definition like
A week is a time unit equal to seven days. is commonly accepted.
But maybe you are correct and this is special logic and belongs in the client.
Regards,
Andreas
From: Chris Hostetter hossman_luc...@fucit.org
To: solr-user@lucene.apache.org
Sent: Tue, March 1, 2011 6:30:26 PM
Subject: Re: More Date Math: NOW/WEEK
: Digging into the source code of DateMathParser.java, i found the following
: comment:
:99 // NOTE: consciously choosing not to support WEEK at this time,
: 100 // because of complexity in rounding down to the nearest week 101
: // arround a month/year boundry. 102 // (Not to mention: it's not
clear
: what people would *expect*)
:
: I was able to implement a work-around in my ruby client using the following
: pseudo code:
: wd=NOW.wday; NOW-#{wd}DAY/DAY
the main issue that comment in DateMathParser.java is refering to is what
the ambiguity of what should happen when you try do something like
2009-01-02T00:00:00Z/WEEK
WEEK would be the only unit where rounding changed a unit
*larger* then the one you rounded on -- ie: rounding day only affects
hours, minutes, seconds, millis; rounding on month only affects days,
hours, minutes, seconds, millies; but in an example like the one above,
where Jan 2 2009 was a friday. rounding down a week (using logic similar
to what you have) would result in 2008-12-28T00:00:00Z -- changing the
month and year.
It's not really clear that that is what people would expect -- i'm
guessing at least a few people would expect it to stop at the 1st of the
month.
the ambiguity of what behavior makes the most sense is why never got
arround to implementing it -- it's certianly possible, but the
various options seemed too confusing to really be very generally useful
and easy to understand
as you point out: people who really want special logic like this (and know
how they want it to behave) have an easy workarround by evaluating NOW
in the client since every week has exactly seven days.
-Hoss
Re: More Date Math: NOW/WEEK
: Digging into the source code of DateMathParser.java, i found the following
: comment:
:99 // NOTE: consciously choosing not to support WEEK at this time,
: 100 // because of complexity in rounding down to the nearest week 101
: // arround a month/year boundry. 102 // (Not to mention: it's not
clear
: what people would *expect*)
:
: I was able to implement a work-around in my ruby client using the following
: pseudo code:
: wd=NOW.wday; NOW-#{wd}DAY/DAY
the main issue that comment in DateMathParser.java is refering to is what
the ambiguity of what should happen when you try do something like
2009-01-02T00:00:00Z/WEEK
WEEK would be the only unit where rounding changed a unit
*larger* then the one you rounded on -- ie: rounding day only affects
hours, minutes, seconds, millis; rounding on month only affects days,
hours, minutes, seconds, millies; but in an example like the one above,
where Jan 2 2009 was a friday. rounding down a week (using logic similar
to what you have) would result in 2008-12-28T00:00:00Z -- changing the
month and year.
It's not really clear that that is what people would expect -- i'm
guessing at least a few people would expect it to stop at the 1st of the
month.
the ambiguity of what behavior makes the most sense is why never got
arround to implementing it -- it's certianly possible, but the
various options seemed too confusing to really be very generally useful
and easy to understand
as you point out: people who really want special logic like this (and know
how they want it to behave) have an easy workarround by evaluating NOW
in the client since every week has exactly seven days.
-Hoss
Re: Date Math
Thank you, that clarifies it. Good catch on -DAY. I had noticed it after submitting but as -1DAY causes the same ParseException, I didn't amend the question. Andreas From: Chris Hostetter hossman_luc...@fucit.org To: solr-user@lucene.apache.org Sent: Tue, February 22, 2011 6:18:56 PM Subject: Re: Date Math : org.apache.lucene.queryParser.ParseException: Cannot parse 'last_modified:-DAY': ... : Are they not supported as a short-cut for NOW-1DAY? I'm using Solr 1.4. No, -1DAY is a valid DateMath string (to the DateMathParser) but as a field value you must specify a valid date string, which can *end* with a DateMath string. so NOW-1DAY is legal, as is 2011-02-22T12:34:56Z-1DAY Note also: you didn't do -1DAY you tried -DAY which isn't valid anywhere. -Hoss
More Date Math: NOW/WEEK
Date Math is great.
NOW/MONTH, NOW/DAY are all working and very useful, so naively I tried
NOW/WEEK,
which failed.
Digging into the source code of DateMathParser.java, i found the following
comment:
99 // NOTE: consciously choosing not to support WEEK at this time,
100 // because of complexity in rounding down to the nearest week 101
// arround a month/year boundry. 102 // (Not to mention: it's not clear
what people would *expect*)
I was able to implement a work-around in my ruby client using the following
pseudo code:
wd=NOW.wday; NOW-#{wd}DAY/DAY
This could be extended and integrated into the DateMathParser.java directly
using the something like the following mapping:
valWEEKS -- (val*7)DAYS
date/WEEK -- (date-(date.DAY_OF_WEEK)DAYS)/DAY
What other concerns are there to consider?
Andreas
Re: Date Math
: org.apache.lucene.queryParser.ParseException: Cannot parse 'last_modified:-DAY': ... : Are they not supported as a short-cut for NOW-1DAY? I'm using Solr 1.4. No, -1DAY is a valid DateMath string (to the DateMathParser) but as a field value you must specify a valid date string, which can *end* with a DateMath string. so NOW-1DAY is legal, as is 2011-02-22T12:34:56Z-1DAY Note also: you didn't do -1DAY you tried -DAY which isn't valid anywhere. -Hoss
Date Math
The SolrQuerySyntax Wiki page refers to DateMathParser for examples.
When I tried -1DAY, I got:
org.apache.lucene.queryParser.ParseException: Cannot parse
'last_modified:-DAY':
Encountered - - at line 1, column 14.
Was expecting one of: ( ... * ... QUOTED ... TERM ...
PREFIXTERM ... WILDTERM ... [ ... { ... NUMBER ...
Are they not supported as a short-cut for NOW-1DAY? I'm using Solr 1.4.
date math help
If I wanted to return documents faceted by LastMod and I wanted them grouped by week what would the correct syntax be? I would like to eventually format the results to display this: Last day (facet count) Last Week (facet count) Last 2 Weeks (facet count) Last Month (facet count) Last 2 Months (facet count) Last 3 Months (facet count) So I figured I can get a date facet results going back 3 months in 7 day increments then map the results to my desired format. facet=true facet.date=LastMod f.LastMod.facet.date.end=NOW/DAY+1DAY f.LastMod.facet.date.gap=+7DAYS f.LastMod.facet.date.start=NOW/DAY-84DAYS (84 days is 12 weeks) Am I going about this wrong or is there a better way to get my desired result? thanks! -- View this message in context: http://old.nabble.com/date-math-help-tp27315500p27315500.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: date math help
If I wanted to return documents faceted by LastMod and I wanted them grouped by week what would the correct syntax be? I would like to eventually format the results to display this: Last day (facet count) Last Week (facet count) Last 2 Weeks (facet count) Last Month (facet count) Last 2 Months (facet count) Last 3 Months (facet count) What not use separate facet query for each these? facet.query:[NOW-1DAY TO NOW]facet.query:[NOW-7DAYS TO NOW] etc.
Re: date math help
--- What not use separate facet query for each these? facet.query:[NOW-1DAY TO NOW]facet.query:[NOW-7DAYS TO NOW] etc. Sorry I forgot to add the field name: facet.query=LastMod:[NOW-1DAY TO NOW]facet.query=LastMod:[NOW-7DAYS TO NOW]
Re: date math in bf?
: Is it possible to do date math in a FunctionQuery? This doesn't work, but I'm : looking for something like: : : bf=recip((NOW-updated),1,200,10) when using DisMax to get the elapsed time : between NOW and when the document was updated (where updated is a Date field). Date Math (as implemented by the DateMathParser) only deals with rounding or adding intervals to dates -- not computing differneces between dates. The key distinction being that DateMath expressions all evaluate to real dates, what you are descrbing owuld evaluate to a date interval (ie: a number) I imagine what you are describing could be implmeneted as a new type of ValueSource that took in two date fields and a unit (ie: month, day, year, etc...) and returned the difference between those dates as a numeric valud of those units. An alternate form could take in a date math expression as a string in place of one date field so you could compute the difference beteween a field and a fixed value. ...but that doesn't exist yet. -Hoss
date math in bf?
Is it possible to do date math in a FunctionQuery? This doesn't work, but I'm looking for something like: bf=recip((NOW-updated),1,200,10) when using DisMax to get the elapsed time between NOW and when the document was updated (where updated is a Date field). I know one can do rord(updated) instead, but I find that difficult to think about and the ordering may not be linear with respect to time making it only a rough approximation of document age. -Sean
