Re: How to sort on dates?

[Georgios Petasis]

Στις 18/12/2017 15:09, ο Michael Kuhlmann έγραψε:

Am 16.12.2017 um 19:39 schrieb Georgios Petasis:

Even if the DateRangeField field can store a range of dates, doesn't
Solr understand that I have used single timestamps?

No. It could theoretically, but sorting just isn't implemented in
DateRangeField.


I have even stored the dates.
My problem is that I need to use the query formating stated in the
documentation:
https://lucene.apache.org/solr/guide/7_1/working-with-dates.html#date-range-formatting

For example, if "financialYear" is a date range, I can do
q=financialYear:2014 and it will return everything that has a date
within 2014. If the field is date point, will it work?

Yes, just query with the plain old range syntax:
q=financialYear:[2014-01-01T00:00:00.000Z TO 2015-01-01T00:00:00.000Z}

DateRangeField might be slightly faster for such queries, but that
doesn't really matter much. I only used normal date fields yet, usually
they're fast enough.

As a rule of thunb, only use DateRangeField if you really need to index
date ranges.

-Michael


Hi all,

I tried to create an index that uses DatePointField for dates. Now the 
query "financialYear:2014" raises an error:


Invalid Date String:'2014'

On the other hand, with DateRangeField it works. And since I cannot 
restrict users (to not send only years like 2014), I cannot use 
DatePointField.


I am starting to think that there is no alternative than using both, one 
for queries and the other for sorting.


Is there a reason Solr does not understand "[2014 TO 2016]" in 
DatePointField as it does for DatePointField?


Best,

George

Re: How to sort on dates?

[Georgios Petasis]

Dear Erick,

Thanks for the idea, it doesn't work. It raises an exception that the 
value is missing the double() method.


Best,
George

Στις 18/12/2017 21:19, ο Erick Erickson έγραψε:

You _might_ (and I haven't tried this) be able to use sorting by
funtion with min/max for a date range field, see: "Sort by Function"
here: https://lucene.apache.org/solr/guide/6_6/function-queries.html.


On Mon, Dec 18, 2017 at 5:09 AM, Michael Kuhlmann <k...@solr.info> wrote:

Am 16.12.2017 um 19:39 schrieb Georgios Petasis:

Even if the DateRangeField field can store a range of dates, doesn't
Solr understand that I have used single timestamps?

No. It could theoretically, but sorting just isn't implemented in
DateRangeField.


I have even stored the dates.
My problem is that I need to use the query formating stated in the
documentation:
https://lucene.apache.org/solr/guide/7_1/working-with-dates.html#date-range-formatting

For example, if "financialYear" is a date range, I can do
q=financialYear:2014 and it will return everything that has a date
within 2014. If the field is date point, will it work?

Yes, just query with the plain old range syntax:
q=financialYear:[2014-01-01T00:00:00.000Z TO 2015-01-01T00:00:00.000Z}

DateRangeField might be slightly faster for such queries, but that
doesn't really matter much. I only used normal date fields yet, usually
they're fast enough.

As a rule of thunb, only use DateRangeField if you really need to index
date ranges.

-Michael

Re: How to sort on dates?

[Georgios Petasis]

Στις 15/12/2017 20:05, ο Shawn Heisey έγραψε:

On 12/15/2017 2:53 AM, Georgios Petasis wrote:

I have a field of type "date_range" defined as:



The problem is that sorting on this field does not work (despite the
fact that I put dates in there). Instead I get an error prompting to
perform sorting through a query.

Stating what Michael said in a different way:  Entries in a
DateRangeField can be a date range, not just a single timestamp.  How
would you decide what specific date to use in a sort?  The start of the
range?  The end of the range?  The middle of the range?  Any option that
the developers chose would be wrong for somebody, and it's not a
straightforward thing to make that choice configurable.

Michael suggested DatePointField.  That should work, because this type
holds a single timestamp, not a range.

The reason that DateRangeField is deprecated is that it uses a legacy
Lucene class that will no longer be available in 8.0.  Coming up with a
replacement is one of the many things that must be addressed before the
8.0 release.

Thanks,
Shawn
Even if the DateRangeField field can store a range of dates, doesn't 
Solr understand that I have used single timestamps?

I have even stored the dates.
My problem is that I need to use the query formating stated in the 
documentation: 
https://lucene.apache.org/solr/guide/7_1/working-with-dates.html#date-range-formatting
For example, if "financialYear" is a date range, I can do 
q=financialYear:2014 and it will return everything that has a date 
within 2014. If the field is date point, will it work?


George

Re: How to sort on dates?

[Georgios Petasis]

Στις 15/12/2017 20:05, ο Shawn Heisey έγραψε:

On 12/15/2017 2:53 AM, Georgios Petasis wrote:

I have a field of type "date_range" defined as:



The problem is that sorting on this field does not work (despite the
fact that I put dates in there). Instead I get an error prompting to
perform sorting through a query.

Stating what Michael said in a different way:  Entries in a
DateRangeField can be a date range, not just a single timestamp.  How
would you decide what specific date to use in a sort?  The start of the
range?  The end of the range?  The middle of the range?  Any option that
the developers chose would be wrong for somebody, and it's not a
straightforward thing to make that choice configurable.

Michael suggested DatePointField.  That should work, because this type
holds a single timestamp, not a range.

The reason that DateRangeField is deprecated is that it uses a legacy
Lucene class that will no longer be available in 8.0.  Coming up with a
replacement is one of the many things that must be addressed before the
8.0 release.

Thanks,
Shawn
Even if the DateRangeField field can store a range of dates, doesn't 
Solr understand that I have used single timestamps?

I have even stored the dates.
My problem is that I need to use the query formating stated in the 
documentation: 
https://lucene.apache.org/solr/guide/7_1/working-with-dates.html#date-range-formatting
For example, if "financialYear" is a date range, I can do 
q=financialYear:2014 and it will return everything that has a date 
within 2014. If the field is date point, will it work?


George

How to sort on dates?

[Georgios Petasis]

Hi all,

I have a field of type "date_range" defined as:

multiValued="false" indexed="true" stored="true"/>


The problem is that sorting on this field does not work (despite the 
fact that I put dates in there). Instead I get an error prompting to 
perform sorting through a query.


How can I do that? There is no documentation that I could find, that 
shows an alternative.


Also, I think that I saw a warning somewhere, that DateRangeField is 
deprecated. But no alternative is suggested:


https://lucene.apache.org/solr/guide/7_1/working-with-dates.html

I am using solr 7.1.

George

How to use facet pivoting with paging?

[Georgios Petasis]

Hi all,

I am performing a facet with pivot query. The result I get is correct, 
but when I use "facet.offset=1" in the query, the results I get differ, 
as in pivot removed.


Instead of getting:

"facet_pivot":{
  "buyer.name,agreedPrice.hasCurrency.notation":[{
  "field":"buyer.name",
  "value":"KOMPANIA W\u0118GLOWA S.A.",
  "count":11228,
  "pivot":[{
  "field":"agreedPrice.hasCurrency.notation",
  "value":"EUR",
  "count":11084,
  "stats":{
"stats_fields":{
  "agreedPrice.hasCurrencyValue":{
"sum":5.069190961199988E9],
  "stats":{
"stats_fields":{
  "agreedPrice.hasCurrencyValue":{
"sum":5.069190961199988E9,

I get:

"facet_pivot":{
  "buyer.name,agreedPrice.hasCurrency.notation":[{
  "field":"buyer.name",
  "value":"KOMPANIA W\u0118GLOWA S.A.",
  "count":11228,
  "stats":{
"stats_fields":{
  "agreedPrice.hasCurrencyValue":{
"sum":5.069190961199988E9,

There is no "pivot" element. Is there a way to use paging and get the 
results I get with facet.offset=0?


I am using Solr 6.4.2

Regards,

George

Removing duplicate values from fields filled with copyField...

[Georgios Petasis]

Hi all,

It seems that this is a popular request (remove duplicates generated 
from copyField), but I am not sure that I have understood the answer. 
Can somebody point to a correct answer for this issue?


I have understand that this involves "update request processors", but I 
am not sure I understand how to configure them.


Regards,

George

Removing duplicate values from fields filled with copyField...

[Georgios Petasis]

Hi all,

It seems that this is a popular request (remove duplicates generated 
from copyField), but I am not sure that I have understood the answer. 
Can somebody point to a correct answer for this issue?


I have understand that this involves "update request processors", but I 
am not sure I understand how to configure them.


Regards,

George

Sort by date? (solr.DateRangeField)

[Georgios Petasis]

Dear all,

I have defined a field "date_range" as:
multiValued="true" indexed="true"/>


The field works (and I can perform range queries for these dates). but I 
have not found a way to sort
results based on this field. I have tried with the dist(), ms() 
functions, to use the boost plugin, etc. and everything fails.

Either with an error message, like:

fl=startDate={!boost+b=$dateboost+v=$qq}=recip(ms(NOW,startDate),3.16e-11,1,1)=*
A ValueSource isn't directly available from this field. Instead try a 
query using the distance as the score.


or a stack trace:
fl=startDate=*="ms(startDate)"+asc"
   java.lang.UnsupportedOperationException
at 
org.apache.lucene.queries.function.FunctionValues.doubleVal(FunctionValues.java:47)
at 
org.apache.lucene.queries.function.ValueSource$ValueSourceComparator.copy(ValueSource.java:153)
at 
org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:206)
at 
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:221)
at 
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:172)

at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at 
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
at 
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
at 
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:211)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1765)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1582)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:581)
at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:528)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)

at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)

at java.lang.Thread.run(Thread.java:745)


 I am using Solr 6.2.1.

Regards,
George

Sort by date? (solr.DateRangeField)

[Georgios Petasis]

Dear all,

I have defined a field "date_range" as:
multiValued="true" indexed="true"/>


The field works (and I can perform range queries for these dates). but I 
have not found a way to sort
results based on this field. I have tried with the dist(), ms() 
functions, to use the boost plugin, etc. and everything fails.

Either with an error message, like:

fl=startDate={!boost+b=$dateboost+v=$qq}=recip(ms(NOW,startDate),3.16e-11,1,1)=*
A ValueSource isn't directly available from this field. Instead try a 
query using the distance as the score.


or a stack trace:
fl=startDate=*="ms(startDate)"+asc"
   java.lang.UnsupportedOperationException
at 
org.apache.lucene.queries.function.FunctionValues.doubleVal(FunctionValues.java:47)
at 
org.apache.lucene.queries.function.ValueSource$ValueSourceComparator.copy(ValueSource.java:153)
at 
org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:206)
at 
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:221)
at 
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:172)

at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at 
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
at 
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:473)
at 
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:211)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1765)
at 
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1582)
at 
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:581)
at 
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:528)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)

at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)

at java.lang.Thread.run(Thread.java:745)


 I am using Solr 6.2.1.

Regards,
George

Re: confirm subscribe to solr-user@lucene.apache.org

[Georgios Petasis]

On 7/11/2016 13:50, solr-user-h...@lucene.apache.org wrote:

Hi! This is the ezmlm program. I'm managing the
solr-user@lucene.apache.org mailing list.

I'm working for my owner, who can be reached
at solr-user-ow...@lucene.apache.org.

To confirm that you would like

petas...@yahoo.gr

added to the solr-user mailing list, please send
a short reply to this address:


solr-user-sc.1478519438.cplbgddohdekmdpnkjoa-petasisg=yahoo...@lucene.apache.org

Usually, this happens when you just hit the "reply" button.
If this does not work, simply copy the address and paste it into
the "To:" field of a new message.

or click here:

mailto:solr-user-sc.1478519438.cplbgddohdekmdpnkjoa-petasisg=yahoo...@lucene.apache.org

This confirmation serves two purposes. First, it verifies that I am able
to get mail through to you. Second, it protects you in case someone
forges a subscription request in your name.

Please note that ALL Apache dev- and user- mailing lists are publicly
archived.  Do familiarize yourself with Apache's public archive policy at

 http://www.apache.org/foundation/public-archives.html

prior to subscribing and posting messages to solr-user@lucene.apache.org.
If you're not sure whether or not the policy applies to this mailing list,
assume it does unless the list name contains the word "private" in it.

Some mail programs are broken and cannot handle long addresses. If you
cannot reply to this request, instead send a message to
 and put the
entire address listed above into the "Subject:" line.


--- Administrative commands for the solr-user list ---

I can handle administrative requests automatically. Please
do not send them to the list address! Instead, send
your message to the correct command address:

To subscribe to the list, send a message to:


To remove your address from the list, send a message to:


Send mail to the following for info and FAQ for this list:



Similar addresses exist for the digest list:



To get messages 123 through 145 (a maximum of 100 per request), mail:


To get an index with subject and author for messages 123-456 , mail:


They are always returned as sets of 100, max 2000 per request,
so you'll actually get 100-499.

To receive all messages with the same subject as message 12345,
send a short message to:


The messages should contain one line or word of text to avoid being
treated as sp@m, but I will ignore their content.
Only the ADDRESS you send to is important.

You can start a subscription for an alternate address,
for example "john@host.domain", just add a hyphen and your
address (with '=' instead of '@') after the command word: