A collection gone missing: uninteresting collection

2017-01-20 Thread Chetas Joshi 
Hello,

I have been running Solr (5.5.0) on HDFS.

Recently a collection just went missing with all the instanceDirs and
Datadirs getting deleted. The following logs in the solrCloud overseer.

2017-01-20 20:42:39,515 INFO org.apache.solr.core.SolrCore:
[3044_01_17_shard4_replica1]  CLOSING SolrCore
org.apache.solr.core.SolrCore@2e2e0a23

2017-01-20 20:42:39,665 INFO org.apache.solr.core.SolrCore:
[3044_01_17_shard4_replica1] Closing main searcher on request.

2017-01-20 20:42:39,690 INFO org.apache.solr.core.CachingDirectoryFactory:
looking to close hdfs://Ingest/solr53/3044_01_17/core_node3/data
[CachedDir<>]

2017-01-20 20:42:39,690 INFO org.apache.solr.core.CachingDirectoryFactory:
looking to close hdfs://Ingest/solr53/3044_01_17/core_node3/data/index
[CachedDir<>,
CachedDir<>]

2017-01-20 20:42:39,690 INFO org.apache.solr.core.CachingDirectoryFactory:
Closing directory: hdfs://Ingest/solr53/3044_01_17/core_node3/data/index

2017-01-20 20:42:39,712 INFO org.apache.solr.store.hdfs.HdfsDirectory:
Closing hdfs directory hdfs://Ingest/solr53/3044_01_17/core_node3/data/index

2017-01-20 20:42:39,713 INFO org.apache.solr.core.CachingDirectoryFactory:
Closing directory: hdfs://Ingest/solr53/3044_01_17/core_node3/data

2017-01-20 20:42:39,713 INFO org.apache.solr.store.hdfs.HdfsDirectory:
Closing hdfs directory hdfs://Ingest/solr53/3044_01_17/core_node3/data

2017-01-20 20:42:39,713 INFO org.apache.solr.core.CachingDirectoryFactory:
looking to close hdfs://Ingest/solr53/3044_01_17/core_node3/data []

2017-01-20 20:42:39,713 INFO org.apache.solr.core.CachingDirectoryFactory:
Removing directory after core close:
hdfs://Ingest/solr53/3044_01_17/core_node3/data

  "core":"3044_01_17_shard13_replica1",

  "collection":"3044_01_17",

2017-01-20 20:42:39,780 INFO org.apache.solr.cloud.overseer.ZkStateWriter:
going to update_collection /collections/3044_01_17/state.json version: 2164

2017-01-20 20:42:39,782 INFO org.apache.solr.common.cloud.ZkStateReader: A
cluster state change: [WatchedEvent state:SyncConnected
type:NodeDataChanged path:/collections/3044_01_17/state.json] for
collection [3044_01_17] has occurred - updating... (live nodes size: [85])

2017-01-20 20:42:39,783 INFO org.apache.solr.common.cloud.ZkStateReader:
Updating data for [3044_01_17] from [2164] to [2165]

  "core":"3044_01_17_shard32_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard22_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard27_replica1",

  "collection":"3044_01_17",

2017-01-20 20:42:39,822 INFO
org.apache.solr.cloud.ShardLeaderElectionContextBase: Removing leader
registration node on cancel: /collections/3044_01_17/leaders/shard4/leader 9

2017-01-20 20:42:39,832 INFO org.apache.solr.cloud.ElectionContext:
Canceling election
/collections/3044_01_17/leader_elect/shard4/election/241183598302995297-core_node3-n_08

2017-01-20 20:42:39,833 INFO org.apache.solr.common.cloud.ZkStateReader:
Removing watch for uninteresting collection [3044_01_17]

  "core":"3044_01_17_shard4_replica1",

  "collection":"3044_01_17",

2017-01-20 20:42:39,852 INFO org.apache.solr.servlet.HttpSolrCall: [admin]
webapp=null path=/admin/cores
params={deleteInstanceDir=true&action=UNLOAD&core=3044_01_17_shard4_replica1&wt=javabin&qt=/admin/cores&deleteDataDir=true&version=2}
status=0 QTime=341

2017-01-20 20:42:39,938 INFO org.apache.solr.cloud.overseer.ZkStateWriter:
going to update_collection /collections/3044_01_17/state.json version: 2165

2017-01-20 20:42:39,940 INFO org.apache.solr.common.cloud.ZkStateReader:
Uninteresting collection [3044_01_17]

  "core":"3044_01_17_shard14_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard35_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard11_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard3_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard1_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard2_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard5_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard42_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard28_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard45_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard8_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard49_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard9_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard34_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard25_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard7_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard10_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard36_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard47_replica1",

  "collection":"3044_01_17",

  "core":"3044_01_17_shard6_replica1",

  "collection

NPE when using timeAllowed in the /export handler

2017-01-20 Thread radha krishnan 
Hi,

am trying to query a core with 60 million docs by specifying timeAllowed as
100 ms just to test the timeAllowed feature.

This is the query

http://10.20.132.162:8983/solr/large_core/export?indent=on&q=*:*&distrib=false&fl=logtype&timeAllowed=100&sort=logtype+asc&wt=json&version=2.2

when i ran in the browser, i got the below NPE

the /export query  has 17 million docs has hits but there was an NPE after
the /export was called.

Can you tell if any thing is wrong in the query or is there is a known bug
for the NPE

HTTP ERROR 500

Problem accessing /solr/logs_core_new/export. Reason:

{trace=java.lang.NullPointerException
at org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:61)
at 
org.apache.solr.response.SortingResponseWriter.write(SortingResponseWriter.java:176)
at 
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at 
org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:728)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
,code=500}



RELATED SERVER LOGS









2017-01-21 01:55:12.257 WARN  (qtp1397616978-21) [   x:logs_core_new]
org.apache.solr.search.SolrIndexSearcher Query: []; Elapsed time: 120.
Exceeded allowed search time: 100 ms.

2017-01-21 01:55:12.258 INFO  (qtp1397616978-21) [   x:logs_core_new]
org.apache.solr.core.SolrCore.Request [logs_core_new]  webapp=/solr
path=/export 
params={q=*:*&distrib=false&indent=on&fl=vmw_vr_ops_logtype&timeAllowed=100&sort=vmw_vr_ops_logtype+asc&wt=json&version=2.2}
hits=17579862 status=0 QTime=120

2017-01-21 01:55:12.718 ERROR (qtp1397616978-21) [   x:logs_core_new]
org.apache.solr.servlet.HttpSolrCall
null:java.lang.NullPointerException

at org.apache.lucene.util.BitSetIterator.(BitSetIterator.java:61)

at 
org.apache.solr.response.SortingResponseWriter.write(SortingResponseWriter.java:176)

at 
org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)

at 
org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:728)

at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)

at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)

at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)

at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)

at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)

at 
org.eclipse.jetty.server.

Re: Phonetic Search

2017-01-20 Thread Rishikant Snigh 
As suggested phonetic search won't work in this case. try spellcheck
component

On 19-Jan-2017 10:53 PM, "Walter Underwood"  wrote:

> Phonetic search will not match “satpuda” and “satpura” because they sound
> different. You want fuzzy search.
>
> To get fuzzy search that is easy to use in edismax, apply the patch in
> SOLR-629.
>
> https://issues.apache.org/jira/browse/SOLR-629  jira/browse/SOLR-629>
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
> > On Jan 19, 2017, at 5:14 AM, Vivek Pathak  wrote:
> >
> > https://cwiki.apache.org/confluence/display/solr/Phonetic+Matching
> >
> > didnt work for you?
> >
> >
> > On 01/19/2017 05:58 AM, PAVAN wrote:
> >> Hi,
> >>
> >> I am trying to implement phonetic search in my application. Below are
> >> indexed terms in solr.
> >>
> >> "satpura private limited"
> >>
> >> when user search with "satpuda"  it has to display the above result.
> Below
> >> is the configuration
> >>
> >> 
> >>  
> >>  
> >>  
> >>   ignoreCase="true"
> >> words="stopwords.txt" />
> >>   nameType="GENERIC"
> >>  ruleType="APPROX" concat="true" languageSet="auto"
> />
> >>  
> >>   
> >>
> >> Please help me how to implement phonetic search.
> >>
> >>
> >>
> >>
> >>
> >>
> >> --
> >> View this message in context: http://lucene.472066.n3.
> nabble.com/Phonetic-Search-tp4314828.html
> >> Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
>

RE: Joining Across Collections

2017-01-20 Thread Moenieb Davids 
Hi Guys

Just a quick question on search and join:

I have a few cores which is based on a mainframe extract, 1 core per extracted 
file which resembles a "DB Table"
The cores are all somehow linked via 1 to many fields, with a structure similar 
to a normal ERD

Is it possible to return the result from a query that joins lets say 3 cores in 
the following format:

"core1_id":"XXX",
"_childDocuments_":[
{
  "core2_id":"yyy",
  "core_2_fieldx":"ABC",
  "_childDocuments_":[
  {
"core3_id":"zzz",
"core_3_fieldx":"ABC",
"core3_fieldy":"123",
  {
  "core2_fieldy":"123",
{

Regards
Moenieb Davids

-Original Message-
From: nabil Kouici [mailto:koui...@yahoo.fr.INVALID] 
Sent: 20 January 2017 03:43 PM
To: solr-user@lucene.apache.org
Subject: Re: Joining Across Collections

Hi Mikhail,
I'm not we can do a negation because Field1 and Field2 are not in the join 
condition.
Regards,Nabil.

  De : Mikhail Khludnev 
 À : solr-user ; nabil Kouici  
 Envoyé le : Jeudi 19 janvier 2017 9h00
 Objet : Re: Joining Across Collections
   
It seems like it can be done by just negating join query or I'm missing
something.

On Wed, Jan 18, 2017 at 11:32 AM, nabil Kouici 
wrote:

> Hi All,
> I'm using  join across collection feature to do an inner join between 2
> collections. It works fine.
> Is it possible to use this feature to compare between fields from
> different collections. For exemple:
> Collection1 Field1Collection2 Field2
> search document from Collection1 where Field1 != Field2
> In sql, this will translated to:
> Select A.* From Collection1 A inner join Collection2 B on  A.id=B.idWhere
> A.Field1<>B.Field2
>
> Thank you.
> Regards,NKI.
>



-- 
Sincerely yours
Mikhail Khludnev


   










===
GPAA e-mail Disclaimers and confidential note 

This e-mail is intended for the exclusive use of the addressee only.
If you are not the intended recipient, you should not use the contents 
or disclose them to any other person. Please notify the sender immediately 
and delete the e-mail. This e-mail is not intended nor 
shall it be taken to create any legal relations, contractual or otherwise. 
Legally binding obligations can only arise for the GPAA by means of 
a written instrument signed by an authorised signatory.
===

Re: Boolean disjunction with nested documents

2017-01-20 Thread Mikhail Khludnev 
It could be more clear if you post debugQuery=true response. Anyway, you've
find a solution, it's great.
There is no a performance pitfall in nested queries (_query_).

On Fri, Jan 20, 2017 at 5:24 PM, Ivan Bianchi  wrote:

> Hi Mikhail,
>
> thanks for the answer, although both of queries still are giving me the
> same "*child query must only match non-parent docs*" IllegalState error, as
> I understand it is still seeing the second clause (has_schedules:false) as
> part of the child filter.
>
> Even using the v parameter inside the {!parent} part is not recognising the
> second clause.
>
>
> Anyway, finally the nested queries have been a solution for me. Using the
> following filter query has made the desired behaviour:
>
> _query_:"{!parent which='content_type:activity'}(schedule.weekday:1 OR
> schedule.weekday:2)" OR _query_:"has_new_filters:false"
>
> Any thoughts on this kind of nested queries (_query_) performance?
>
> Best regards,
> Ivan
>
> 2017-01-19 18:40 GMT+01:00 Mikhail Khludnev :
>
> > It's my pet peeve. Try
> > ?q={!parent which=content_type:activity}(schedule.weekday:1) OR
> > has_schedules:false&debugQuery=true
> > vs
> > ?q= {!parent which=content_type:activity}(schedule.weekday:1) OR
> > has_schedules:false&debugQuery=true
> > and you'll see how space matters.
> > The pro's way is to
> >
> > ?q={!parent which=content_type:activity v=$childq} has_schedules:false
> > &debugQuery=true&childq=schedule.weekday:1
> >
> > Also check this, it deserves.
> >
> > https://lucidworks.com/blog/2009/03/31/nested-queries-in-solr/Why Not
> AND,
> > OR, And NOT? | Lucidworks.com
> >  > web&cd=1&cad=rja&uact=8&ved=0ahUKEwjF34DJ387RAhWKKcAKHQiFC
> > DAQFggmMAA&url=https%3A%2F%2Flucidworks.com%2Fblog%
> > 2F2011%2F12%2F28%2Fwhy-not-and-or-and-not%2F&usg=
> > AFQjCNH5ilVhXIevavTAURQuLzTZORbTvQ&bvm=bv.144224172,d.bGs>
> > http://blog-archive.griddynamics.com/2013/12/grandchildren-and-siblings-
> > with-block.html
> >
> >
> > On Thu, Jan 19, 2017 at 7:33 PM, Ivan Bianchi  wrote:
> >
> > > I hope someone can help me because I have spent too many time looking
> for
> > > this issue :(
> > >
> > > I have 2 kind of documents related with an 1-n relation, in my example
> > this
> > > is 1 activity has many schedules.
> > > To achieve this I have some inner child document with schedule fields
> > > inside the activity document. The identifier field of the document is
> > > called content_type.
> > >
> > > In the activity I have a boolean field to detect if the document has
> > > children (schedules) called 'has_schedules'.
> > >
> > > I'm trying to find the documents whose children have a weekday = 1 OR
> > they
> > > don't have children. Here's the filter query:
> > > "{!parent which=content_type:activity}(schedule.weekday:1)" OR
> > > has_schedules:false
> > >
> > > I don't understand how the " char works in here, wherever I enclose the
> > > condition with " char, Solr ignores it. For example the previous query
> is
> > > equal to has_schedules:false, on the contrary I only get the first
> > > condition. Also, if I use the "(" it gives me a syntax error.
> > > The other issue here, is that if I remove the " chars, Solr gets the
> > second
> > > condition (has_schedules:false) as part of the nested child query,
> which
> > > implies an error as this is a parent's field.
> > >
> > > Thank you for any help,
> > >
> > >
> > > --
> > > Ivan
> > >
> >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
>
>
> --
> Ivan
>



-- 
Sincerely yours
Mikhail Khludnev

Re: DIH do not work. Child-entity cannot refer parent's id.

2017-01-20 Thread Shawn Heisey 
On 1/20/2017 7:40 AM, Shawn Heisey wrote:
> One thing you might want to try doing is enclosing the property
> ${books.book_id} in single quotes. The example configs on the
> dataimport wiki page have the properties referenced from parent
> entities surrounded by single quotes:

A second look revealed that book_id is numeric, so this is probably NOT
something you need to do.

I've really got no idea what might be wrong here.  Your config looks
correct.

Thanks,
Shawn

Re: DIH do not work. Child-entity cannot refer parent's id.

2017-01-20 Thread Shawn Heisey 
On 1/20/2017 5:45 AM, Keiichi MORITA wrote:
> DataImportHandler *can't* work out with Oracle 12c and Solr 6.3.
> Query in nested entities are called, the mapping values are not in child's
> WHERE clause.
> What is the cause of this error? I want some help.
>
>
> ## data-config.xml
>
> 
>  driver="oracle.jdbc.driver.OracleDriver"
>   url="jdbc:oracle:thin:@localhost:1521/bookstore" 
>   user="bookadmin" 
>   password="bookpass"/>
>   
> 
>   
>   
>   
> 
> 
> 
>   
> 
>   
> 



> o.a.s.h.d.DataImporter Full Import failed:java.lang.RuntimeException:
> java.lang.RuntimeException:
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> execute query: select book_id, content_id, content from contents where
> book_id =  and deleted = 0 Processing Document # 1

The query actually sent to JDBC is missing the value after "book_id ="
... I'm not sure why this is the case.  One thing you might want to try
doing is enclosing the property ${books.book_id} in single quotes.  The
example configs on the dataimport wiki page have the properties
referenced from parent entities surrounded by single quotes:

https://wiki.apache.org/solr/DataImportHandler#Full_Import_Example

Other than this, which you should fix, your usage looks correct to me,
but it's always possible that I'm missing something.

>From what I can tell, the query logged in the error message comes from
DIH, not from JDBC, so this doesn't appear to be an Oracle issue, but
something in Solr or its configuration.  You can verify that the string
is missing in the query that DIH constructs by making DIH log all its
queries.  To do this, go to the Logging tab of the admin UI and change
the logging level for JdbcDataSource to DEBUG.  The queries will be
logged in the solr.log file -- you most likely won't see them in the
admin UI.

It is possible that your DIH config contains invisible characters in the
section where the property is referenced, that is causing the property
to not be correctly inserted.  I don't know how likely this is, but it
is something that could happen.  This can usually be seen in a hex editor.

A bug in DIH is not impossible, although this is the first time I've
heard of this problem happening.  I would have expected a lot of people
to mention this on the mailing list if 6.3 had a bug that caused this.

Are you using the Oracle JVM?  This is recommended.  Version 1.8.x (Java
8) is required for Solr 6.3.0.

Thanks,
Shawn

Re: Boolean disjunction with nested documents

2017-01-20 Thread Ivan Bianchi 
Hi Mikhail,

thanks for the answer, although both of queries still are giving me the
same "*child query must only match non-parent docs*" IllegalState error, as
I understand it is still seeing the second clause (has_schedules:false) as
part of the child filter.

Even using the v parameter inside the {!parent} part is not recognising the
second clause.


Anyway, finally the nested queries have been a solution for me. Using the
following filter query has made the desired behaviour:

_query_:"{!parent which='content_type:activity'}(schedule.weekday:1 OR
schedule.weekday:2)" OR _query_:"has_new_filters:false"

Any thoughts on this kind of nested queries (_query_) performance?

Best regards,
Ivan

2017-01-19 18:40 GMT+01:00 Mikhail Khludnev :

> It's my pet peeve. Try
> ?q={!parent which=content_type:activity}(schedule.weekday:1) OR
> has_schedules:false&debugQuery=true
> vs
> ?q= {!parent which=content_type:activity}(schedule.weekday:1) OR
> has_schedules:false&debugQuery=true
> and you'll see how space matters.
> The pro's way is to
>
> ?q={!parent which=content_type:activity v=$childq} has_schedules:false
> &debugQuery=true&childq=schedule.weekday:1
>
> Also check this, it deserves.
>
> https://lucidworks.com/blog/2009/03/31/nested-queries-in-solr/Why Not AND,
> OR, And NOT? | Lucidworks.com
>  web&cd=1&cad=rja&uact=8&ved=0ahUKEwjF34DJ387RAhWKKcAKHQiFC
> DAQFggmMAA&url=https%3A%2F%2Flucidworks.com%2Fblog%
> 2F2011%2F12%2F28%2Fwhy-not-and-or-and-not%2F&usg=
> AFQjCNH5ilVhXIevavTAURQuLzTZORbTvQ&bvm=bv.144224172,d.bGs>
> http://blog-archive.griddynamics.com/2013/12/grandchildren-and-siblings-
> with-block.html
>
>
> On Thu, Jan 19, 2017 at 7:33 PM, Ivan Bianchi  wrote:
>
> > I hope someone can help me because I have spent too many time looking for
> > this issue :(
> >
> > I have 2 kind of documents related with an 1-n relation, in my example
> this
> > is 1 activity has many schedules.
> > To achieve this I have some inner child document with schedule fields
> > inside the activity document. The identifier field of the document is
> > called content_type.
> >
> > In the activity I have a boolean field to detect if the document has
> > children (schedules) called 'has_schedules'.
> >
> > I'm trying to find the documents whose children have a weekday = 1 OR
> they
> > don't have children. Here's the filter query:
> > "{!parent which=content_type:activity}(schedule.weekday:1)" OR
> > has_schedules:false
> >
> > I don't understand how the " char works in here, wherever I enclose the
> > condition with " char, Solr ignores it. For example the previous query is
> > equal to has_schedules:false, on the contrary I only get the first
> > condition. Also, if I use the "(" it gives me a syntax error.
> > The other issue here, is that if I remove the " chars, Solr gets the
> second
> > condition (has_schedules:false) as part of the nested child query, which
> > implies an error as this is a parent's field.
> >
> > Thank you for any help,
> >
> >
> > --
> > Ivan
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>



-- 
Ivan

Re: Joining Across Collections

2017-01-20 Thread nabil Kouici 
Hi Mikhail,
I'm not we can do a negation because Field1 and Field2 are not in the join 
condition.
Regards,Nabil.

  De : Mikhail Khludnev 
 À : solr-user ; nabil Kouici  
 Envoyé le : Jeudi 19 janvier 2017 9h00
 Objet : Re: Joining Across Collections
   
It seems like it can be done by just negating join query or I'm missing
something.

On Wed, Jan 18, 2017 at 11:32 AM, nabil Kouici 
wrote:

> Hi All,
> I'm using  join across collection feature to do an inner join between 2
> collections. It works fine.
> Is it possible to use this feature to compare between fields from
> different collections. For exemple:
> Collection1 Field1Collection2 Field2
> search document from Collection1 where Field1 != Field2
> In sql, this will translated to:
> Select A.* From Collection1 A inner join Collection2 B on  A.id=B.idWhere
> A.Field1<>B.Field2
>
> Thank you.
> Regards,NKI.
>



-- 
Sincerely yours
Mikhail Khludnev

Upgrade SOLR version - facets perfomance regression

2017-01-20 Thread SOLR4189 
Before few months we upgraded our SOLR in production from 4.10.1 to 5.4.1.
And at once we noticed perfomance regressions. After searchings in internet
we found  SOLR-8096 issue  
. So we had to downgrade SOLR version to 4.10.1. 
We want to be in the latest Solr version (5 or 6), but  
  1. Facets perfomance is very important for us.
  2. We can't use new Facet API, because our client application uses
blacklight-4.5.0 (without option for upgrade).

What can we do in this situations? If the truth that new Facet API have to
return perfomance of SOLR-4.10.1? Maybe does some convertor from new api to
old api exist?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Upgrade-SOLR-version-facets-perfomance-regression-tp4315027.html
Sent from the Solr - User mailing list archive at Nabble.com.

DIH do not work. Child-entity cannot refer parent's id.

2017-01-20 Thread Keiichi MORITA 
DataImportHandler *can't* work out with Oracle 12c and Solr 6.3.
Query in nested entities are called, the mapping values are not in child's
WHERE clause.
What is the cause of this error? I want some help.


## data-config.xml


  
  

  
  
  



  

  



## Tables

CREATE TABLE books (
 book_id number(11)   NOT NULL PRIMARY KEY,
 title   varchar2(128)DEFAULT NULL,
 deleted number(1)DEFAULT 0
);

CREATE TABLE contents (
 book_id  number(11)  NOT NULL,
 content_id   number(11)  NOT NULL,
 content  varchar2(2048)  NOT NULL,
 deleted  number(1)   DEFAULT 0
);
ALTER TABLE contents ADD CONSTRAINT contents_pk PRIMARY KEY (book_id,
content_id);
ALTER TABLE contents ADD CONSTRAINT contents_fk FOREIGN KEY (book_id)
REFERENCES books(book_id);


## Data

INSERT INTO books(book_id, title) VALUES ('1', 'First');
INSERT INTO books(book_id, title) VALUES ('2', 'Second');
INSERT INTO books(book_id, title) VALUES ('3', 'Third');

INSERT INTO contents(book_id, content_id, content) VALUES (1,1,'During the
Middle Ages');
INSERT INTO contents(book_id, content_id, content) VALUES (1,2,'The
Renaissance began in Italy');
INSERT INTO contents(book_id, content_id, content) VALUES (2,1,'By the
mid-19th century');
INSERT INTO contents(book_id, content_id, content) VALUES (3,1,'Italy has
the third largest economy');


## schema.xml







## Error messages in /var/solr/logs/solr.log

2017-01-20 12:29:00.853 ERROR (Thread-17) [   x:bookstore]
o.a.s.h.d.DocBuilder Exception while processing: books document :
SolrInputDocument(fields: [book_id=1,
title=First]):org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query: select book_id, content_id, content from contents
where book_id =  and deleted = 0 Processing Document # 1
at
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:69)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:327)
at
org.apache.solr.handler.dataimport.JdbcDataSource.createResultSetIterator(JdbcDataSource.java:288)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:283)
at
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:52)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:516)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:416)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:475)
at
org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:458)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLSyntaxErrorException: ORA-00936: missing expression

at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1059)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:522)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:210)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:30)
at
oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:762)
at
oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:925)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:)
at
oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1792)
at
oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1745)
at
oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:334)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.executeStatement(JdbcDataSource.java:349)
at
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.(JdbcDataSource.java:321)
... 15 more

2017-01-20 12:29:00.854 INFO  (Thread-17) [   x:bookstore]
o.a.s.u.p.LogUpdateProcessor

Do i need to modify PivotFacetProcessor Code??

2017-01-20 Thread Sumit Jain 
First I will explain my problem-

My Solr Db schema is as follows
*(SNo,Id,Name,Date,Metadata1,Metadata2.Metadata30)*.
/Here Id field is not unique. /

Data Set Example:- 
<"1","1","Alex","20/1/2017","blah1","blah2"...>
<"2","1","Alex","12/1/2017","blah1","blah2"...>
<"3","2","Amanda","14/1/2017","blah1","blah3"...>
<"4","2","Amanda","15/1/2017","blah1","blah3"...>
<"5","2","Amanda","16/1/2017","blah1","blah3"...>
<"6","3","Charles","16/1/2017","blah1","blah"...>
<"7","3","charles","17/1/2017","blah1","blah"...>
<"8","4","Alex","20/1/2017","blah2","blah2"...>
<"9","4","Alex","12/1/2017","blah2","blah2"...>
<"10","5","Amanda","14/1/2017","blah3","blah3"...>
<"11","5","Amanda","15/1/2017","blah3","blah3"...>
<"12","5","Amanda","16/1/2017","blah4","blah3"...>
<"13","6","Charles","16/1/2017","blah4","blah"...>
<"14","6","charles","17/1/2017","blah5","blah"...>

When i execute facet query on *Metadata1 *
Expected Result  - *blah1(3),blah2(1),blah3(1),blah4(2),blah5(1)*
Actual Result  -  *blah1(7),blah2(2),blah3(2),blah4(2),blah5(1)*

So I reckoned that the count is calculated on the basis of *Doucment Unique
Id(Sno)*.
Then i found out about *Pivot Faceting* which is a very cool feature so i
modified my query like this- (*pivot=Metadata1,Id*)
Now the result i get is 

Output:-
*Actual result*
Facet field *(Metadata1)* {
 *blah1 (7)*,
 Pivot filed (*Id*)  {
 *1 (2)
 2 (3)
 3 (2)*
 },
 *blah2 (2)*,
 Pivot filed (*Id*)  {
 *4 (2)*
 },
 *blah3 (2)*,
 Pivot filed (*Id*)  {
 *5 (2)*
 },
 *blah4 (2)*,
 Pivot filed (*Id*)  {
 *4 (1)
 5 (1)*
 },
 *blah5 (1)*,
 Pivot filed (*Id*)  {
 *5 (1)*
 },
}

*Expected Result*
*blah1(3),blah2(1),blah3(1),blah4(2),blah5(1)*

The above approach solves my problem but i get alot of unnecessary content.
*All i need is Unique count on a specific field per metadata field*

I saw the code and i know exactly where i am supposed to make the changes.
But do in need to modify the code ??
I can not afford to have that much network traffic so please suggest me what
are the available options??



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Do-i-need-to-modify-PivotFacetProcessor-Code-tp4315013.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solr Query Debugger

2017-01-20 Thread Vincenzo D'Amore 
Hi all,

about an year ago, trying to understand what's behind the results I had by
my solr instances I wrote a program able to present in a decent way what
was returned by the debugQuery parameter and others.

I found my program extremely interesting and I have ported the code in a
google chrome free plugin:

Solr Query Debugger
https://chrome.google.com/webstore/detail/solr-query-debugger/gmpkeiamnmccifccnbfljffkcnacmmdl

Solr Query Debugger aims to help Solr developers and users with queries.
You can modify Solr queries, execute, debug and, very important, see the
explain in a clear format.

Well, now Solr Query Debugger is quite stable, less bug and more feature, I
write here just present it to the Solr users mailing list and ask, if you
haven't done already, to try and tell me what about.

If you found the plugin interesting or useful.

Thanks for your time,
Vincenzo



-- 
Vincenzo D'Amore
email: v.dam...@gmail.com
skype: free.dev
mobile: +39 349 8513251

Bug in admin website creating collections?

2017-01-20 Thread Steggink, Frank 
Hi,

I'm using Solr 6.2.1  (using the Docker image solr:6.2-alpine) in cloud mode.
When creating a collection through the admin website I noticed that when 
numShards is being omitted, the collection doesn't get created. I'm using 
implicit routing, and the documentation [1] says that numShards is optional. A 
500 error is returned. This is because the numShards parameter is being sent, 
but it is empty. When this parameter is being omitted entirely, I'm able to 
create a collection.
Should this be considered a bug or a "feature"? If I enter a value which is 
different than the actual number of shards I'm interested in (specified by the 
shards field), I assume this value is ignored, correct?

Thanks in advance,

Frank

[1] 
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CREATE:CreateaCollection


Disclaimer:
De inhoud van dit bericht is uitsluitend bestemd voor geadresseerde.
Gebruik van de inhoud van dit bericht door anderen zonder toestemming van het 
Kadaster 
is onrechtmatig. Mocht dit bericht ten onrechte bij u terecht komen, dan 
verzoeken wij u 
dit direct te melden aan de verzender en het bericht te vernietigen. 
Aan de inhoud van dit bericht kunnen geen rechten worden ontleend.

Disclaimer:
The content of this message is meant to be received by the addressee only.
Use of the content of this message by anyone other than the addressee without 
the consent 
of the Kadaster is unlawful. If you have received this message, but are not the 
addressee, 
please contact the sender immediately and destroy the message.
No rights can be derived from the content of this message

Bug in admin website creating collection in SolrCloud using routerField?

2017-01-20 Thread Steggink, Frank 
Hi,

I'm using Apache Solr version 6.2.1, based on the Docker container 
solr:6.2-alpine.
It appears to me that there is a bug when creating a collection in SolrCloud 
using the admin website. I'm using implicit routing and I would like to create 
two shards. The value which is entered in the routerField text is being sent as 
parameter routerField=xxx in the query string, instead of router.field=xxx. 
Consequently, in state.json router.field is not defined at all, only 
router.name is. Can anyone confirm this? Should this be reported in Jira? So 
far I haven't checked if this issue has already been fixed in 6.3 or 6.4. I 
plan to use version 6.3 soon.
As a workaround, I'll use the Collections API.

Thanks in advance,

Frank


Disclaimer:
De inhoud van dit bericht is uitsluitend bestemd voor geadresseerde.
Gebruik van de inhoud van dit bericht door anderen zonder toestemming van het 
Kadaster 
is onrechtmatig. Mocht dit bericht ten onrechte bij u terecht komen, dan 
verzoeken wij u 
dit direct te melden aan de verzender en het bericht te vernietigen. 
Aan de inhoud van dit bericht kunnen geen rechten worden ontleend.

Disclaimer:
The content of this message is meant to be received by the addressee only.
Use of the content of this message by anyone other than the addressee without 
the consent 
of the Kadaster is unlawful. If you have received this message, but are not the 
addressee, 
please contact the sender immediately and destroy the message.
No rights can be derived from the content of this message