Are you comparing strings in here or timestamp? Filter ((cast(registration#37 as string) >= 2015-05-28) && (cast(registration#37 as string) <= 2015-05-29))
Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 14 April 2016 at 18:04, Kiran Chitturi <kiran.chitt...@lucidworks.com> wrote: > Hi, > > Timestamp range filter queries in SQL are not getting pushed down to the > PrunedFilteredScan instances. The filtering is happening at the Spark layer. > > The physical plan for timestamp range queries is not showing the pushed > filters where as range queries on other types is working fine as the > physical plan is showing the pushed filters. > > Please see below for code and examples. > > *Example:* > > *1.* Range filter queries on Timestamp types > > *code: * > >> sqlContext.sql("SELECT * from events WHERE `registration` >= >> '2015-05-28' AND `registration` <= '2015-05-29' ") > > *Full example*: > https://github.com/lucidworks/spark-solr/blob/master/src/test/scala/com/lucidworks/spark/EventsimTestSuite.scala#L151 > * plan*: > https://gist.github.com/kiranchitturi/4a52688c9f0abe3d4b2bd8b938044421#file-time-range-sql > > *2. * Range filter queries on Long types > > *code*: > >> sqlContext.sql("SELECT * from events WHERE `length` >= '700' and >> `length` <= '1000'") > > *Full example*: > https://github.com/lucidworks/spark-solr/blob/master/src/test/scala/com/lucidworks/spark/EventsimTestSuite.scala#L151 > *plan*: > https://gist.github.com/kiranchitturi/4a52688c9f0abe3d4b2bd8b938044421#file-length-range-sql > > The SolrRelation class we use extends > <https://github.com/lucidworks/spark-solr/blob/master/src/main/scala/com/lucidworks/spark/SolrRelation.scala#L37> > the PrunedFilteredScan. > > Since Solr supports date ranges, I would like for the timestamp filters to > be pushed down to the Solr query. > > Are there limitations on the type of filters that are passed down with > Timestamp types ? > Is there something that I should do in my code to fix this ? > > Thanks, > -- > Kiran Chitturi > >
