Recently moved to Solr 8.4.1

[Paresh]

Hi All,

We recently moved to SOlr 8.4.1 from Solr 7.7.0. I am seeing below getting
dumpled in Solr.log file -

2020-03-23 09:40:59.385 ERROR
(updateExecutor-3-thread-4-processing-x:collection1) [   ] o.a.s.c.SolrCore
Too many close [count:-1] on org.apache.solr.core.SolrCore@4c134748. Please
report this exception to solr-user@lucene.apache.org

Any help is appreciated.

Reagrds,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

JOIN query

[Paresh]

Hi,

I have two collections: collection1 and collection2
I have fields like -
colleciton1: id, prop1, prop2, prop3
collection2: id, col1, col2, col3

I am doing a join query with collection1.prop1 = collection2.col1 on
collection2.

As a result, I can get any field from collection2 in 'fl'.

Is there any way to get field from collection1 while performing query from
collection2 joining with collection1?


Regards,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: $deleteDocByQuery is not working for me

[Paresh]

Hi,

I am able to get it done using following way -







Thanks,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: $deleteDocByQuery and $deleteDocByID

[Paresh]

Hi,

I am able to get it done using following way -







Thanks,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: $deleteDocByQuery and $deleteDocByID

[Paresh]

Hi,

I am able to get it done using following way -







Thanks,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: $deleteDocByQuery and $deleteDocByID

[Paresh]

Hi Erik,

I am also looking for some example of deleteDocByQuery. Here is my
requirement -

I want to do the database query and get the list of values of which matching
documents should be deleted from Solr.

I want to delete the docs which matches following query 
SolrColumnName:

This  will come from query executed on RDBMS -
select columnName from Table where state = 'deleted'

This columnName is the value populated in Solr for SolrColulmnName.

Regards,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7.7.0: Log file not getting generated

[Paresh]

Hi Paras,

I have db-data-config.xml file with entities defined for data population and
update.

As a part of some process, I want to delete the documents from Solr for data
deleted in RDBMS. For this purpose I am writing SQL to fetch data from RDBMS
and delete it in Solr using $deleteDocByQuery.

I want to debug if the query getting formed is correct or not -

Here is the snippet of entity -






The said entity, I am not able to delete the docs from solr using
delta-import operation, so wanted to debug it.

Regards,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7.7.0: Log file not getting generated

[Paresh]

How can we see SQL in log file?



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7.7.0: Log file not getting generated

[Paresh]

In my environment I was creating a windows service for Solr and there I
forgot to specify -Dsolr.log.dir as a command line for Windows Service.
After defining it to correct path and reinstalling solr as service solved
the issue.



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

$deleteDocByQuery is not working for me

[Paresh]

Hi,

I am trying to write an entity to delete documents for the records marked as
deleted in my rdbms database using db-data-config.xml file with following
entry in an entity -

query="SELECT CONCAT( 'ColName:', dbCol ) AS '$deleteDocByQuery' FROM TABLE1
t1 WHERE t1.state = 1 AND t1.lmd gt; TO_DATE
('${dih.last_index_time}','-MM-DD HH24:MI:SS')"

Through SOLR-Admin UI I am selecting "delta-import", entity-name and giving
proper jdbc string in custom-parameters.

I have marked the ColName represented in Solr as deleted in RDBMS Table1
using state column value of 1.

ColName is Solr column-name for the collection. dbCol - database column name
in Table1

I am trying to use $deleteDocByQuery clause to remove documents.

Any help is appreciated.

Regards,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 7.7.0: Log file not getting generated

[Paresh]

Thanks Paras. It solved my problem. I am now able to see the logs.



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr 7.7.0: Log file not getting generated

[Paresh]

Hi,

We just moved to Solr 7.7.0 and log file is not getting generated. Even log
folder is not present there.

I manually enabled all the logs through Solr-admin UI but it is not working.

Is there any other way to enable the logs and how can we enable it at the
start of Solr?

Any help is appreciated.

Thanks,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Wild-card query behavior

[Paresh]

E.g. In query, join with wild-card query using parenthesis I get error -

"error-class","org.apache.solr.common.SolrException",
  "root-error-class","org.apache.solr.parser.ParseException"],
"msg":"org.apache.solr.search.SyntaxError: Cannot parse
'solrField:(12*': Encountered \"\" at line 1, column 57.\r\nWas
expecting one of:\r\n ...\r\n ...\r\n ...\r\n   
\"+\" ...\r\n\"-\" ...\r\n ...\r\n\"(\" ...\r\n   
\")\" ...\r\n\"*\" ...\r\n\"^\" ...\r\n ...\r\n   
 ...\r\n ...\r\n ...\r\n
...\r\n ...\r\n\"[\" ...\r\n\"{\" ...\r\n   
 ...\r\n\"filter(\" ...\r\n ...\r\n",
"code":400}}

When using the same query with parenthesis in filter query (fq), I get the
expected results.




--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Wild-card query behavior

[Paresh]

Hi All,

I am trying wild-card query with query, filter query with and without !join
and finding it difficult to understand the SOLR behavior.

(-) wild-card like 12* in query: field:12* works well
(-) wild-card like 12* in query with {!join to=... from=...}field:12* -->
works well
(-) wild-card like (12*) in query with {!join to=... from=...}field:(12*)
--> doesn't work 
(-) wild-card like (12*) in filter query with ={!join to=...
from=...}field:12* --> doesn't work
(-) wild-card like (12*) in filter query with ={!join to=...
from=...}field:"12*" --> doesn't work
(-) wild-card like (12*) in filter query with ={!join to=...
from=...}field:(12*) --> works well

Why wild-card query does not work with {!join}?

Regards,
Paresh



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

How to read DebugQuery output

[Paresh Khandelwal]

Hi All,

I tried to get the debug information about the query for my INNER JOIN and
ACROSS JOIN and trying to understand it.

See the query below - 1487 msec
{
  "responseHeader":{
"status":0, "QTime":1487,
"params":{  "q":"*:*",
  "fq.op":"AND",   "indent":"on",
  "fl":"TC_0Y0_Item_ID",
  "fq":["TC_0Y0_Occurrence_Name:\"6935 style rear MY11+\"",
"TC_0Y0_ProductScope:xtWNf_fTAaLUgD",
"{!join to=TC_0Y0_Item_ID
from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773"],
  "wt":"json",   "debugQuery":"on",
  "group.field":"TC_0Y0_Item_ID", ..
  "debug":{
"join":{
  "{!join from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id
to=TC_0Y0_Item_ID
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773":{
"time":955,
"fromSetSize":3,
"toSetSize":14560,
"fromTermCount":6632106,
"fromTermTotalDf":6632106,
"fromTermDirectCount":6632106,
"fromTermHits":1,
"fromTermHitsTotalDf":1,
"toTermHits":1,
"toTermHitsTotalDf":14560,
"toTermDirectCount":0,
"smallSetsDeferred":1,
"toSetDocsAdded":14560}},
"rawquerystring":"*:*",
"querystring":"*:*",
"parsedquery":"MatchAllDocsQuery(*:*)",
"parsedquery_toString":"*:*",
"explain":{
  "AZD1uV0qgj6GxC":"\n1.0 = *:*, product of:\n  1.0 = boost\n
 1.0 = queryNorm\n"},
"QParser":"LuceneQParser",
"filter_queries":["TC_0Y0_Occurrence_Name:\"6935 style rear
MY11+\"",
  "TC_0Y0_ProductScope:xtWNf_fTAaLUgD",
  "{!join to=TC_0Y0_Item_ID
from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773"],
"parsed_filter_queries":["TC_0Y0_Occurrence_Name:6935 style
rear MY11+",
  "TC_0Y0_ProductScope:xtWNf_fTAaLUgD",
  "JoinQuery({!join
from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id to=TC_0Y0_Item_ID
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773)"],
"timing":{
  "time":1487.0,  ..

I am trying to see why fromTermCount is so high when fromSetSize and
toSetSize is less?

Where can I find the details about all the contents of debugQuery and how
to read each component?

Any help is appreciated.

Regards,
Paresh

Understanding DebugQuery

[Paresh Khandelwal]

Hi All,

I tried to get the debug information about the query for my INNER JOIN and
ACROSS JOIN and trying to understand it.

See the query below - 1487 msec
{
  "responseHeader":{
"status":0, "QTime":1487,
"params":{  "q":"*:*",
  "fq.op":"AND",   "indent":"on",
  "fl":"TC_0Y0_Item_ID",
  "fq":["TC_0Y0_Occurrence_Name:\"6935 style rear MY11+\"",
"TC_0Y0_ProductScope:xtWNf_fTAaLUgD",
"{!join to=TC_0Y0_Item_ID
from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773"],
  "wt":"json",   "debugQuery":"on",
  "group.field":"TC_0Y0_Item_ID", ..
  "debug":{
"join":{
  "{!join from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id
to=TC_0Y0_Item_ID
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773":{
"time":955,
"fromSetSize":3,
"toSetSize":14560,
"fromTermCount":6632106,
"fromTermTotalDf":6632106,
"fromTermDirectCount":6632106,
"fromTermHits":1,
"fromTermHitsTotalDf":1,
"toTermHits":1,
"toTermHitsTotalDf":14560,
"toTermDirectCount":0,
"smallSetsDeferred":1,
"toSetDocsAdded":14560}},
"rawquerystring":"*:*",
"querystring":"*:*",
"parsedquery":"MatchAllDocsQuery(*:*)",
"parsedquery_toString":"*:*",
"explain":{
  "AZD1uV0qgj6GxC":"\n1.0 = *:*, product of:\n  1.0 = boost\n
 1.0 = queryNorm\n"},
"QParser":"LuceneQParser",
"filter_queries":["TC_0Y0_Occurrence_Name:\"6935 style rear
MY11+\"",
  "TC_0Y0_ProductScope:xtWNf_fTAaLUgD",
  "{!join to=TC_0Y0_Item_ID
from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773"],
"parsed_filter_queries":["TC_0Y0_Occurrence_Name:6935 style
rear MY11+",
  "TC_0Y0_ProductScope:xtWNf_fTAaLUgD",
  "JoinQuery({!join
from=TC_0Y0_ItemRevision_0Y0_awp0Item_item_id to=TC_0Y0_Item_ID
fromIndex=collection1}TC_0Y0_ItemRevision_0Y0_awp0Item_item_id:92138773)"],
"timing":{
  "time":1487.0,  ..

I am trying to see why fromTermCount is so high when fromSetSize and
toSetSize is less?

Where can I find the details about all the contents of debugQuery and how
to read each component?

Any help is appreciated.

Regards,
Paresh

JOIN Execution Plan

[Paresh]

Hi,

I have two collections - collection1 and collection2.
I am doing HTTP request on collection2 using
http://localhost:8983/solr/collection1/tcfts/?params={q=col:value AND
_query_:{!join}...AND _query_:{!join}..}

If my query is like - fieldOnCollection1:somevalue AND INNER JOIN (with
collection1) AND ACROSS JOIN( with collection2)

(Q)Drawing analogy to Oracle, how will SOLR execute this query? 
Is there anyway to see in the logs the way SOLR executes all criterias? I
mean the sequence in which it is executed.

When I do JOINs in RDBMS, I specify smallest table first. 
(Q) Do I also need to specify the first criteria as the one where I am
supposed to get less matching documents from SOLR collection?

Any help is appreciated.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

JOIN Execution Plan

[Paresh]

Hi,

I have two collections - collection1 and collection2.
I am doing HTTP request on collection2 using
http://localhost:8983/solr/collection1/tcfts/?params={q=col:value AND
_query_:{!join}...AND _query_:{!join}..}

If my query is like - fieldOnCollection1:somevalue AND INNER JOIN (with
collection1) AND ACROSS JOIN( with collection2)

(Q)Drawing analogy to Oracle, how will SOLR execute this query? 
Is there anyway to see in the logs the way SOLR executes all criterias? I
mean the sequence in which it is executed.

When I do JOINs in RDBMS, I specify smallest table first. 
(Q) Do I also need to specify the first criteria as the one where I am
supposed to get less matching documents from SOLR collection?

Any help is appreciated.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR JOIN

[Paresh]

They are logically different that is why kept in different collections.
Collection1: The purpose is to do Global Text Search for all objects through
multiple document types
Collection3: The purpose is to have search for some other type of objects
whose documents are in Millions and takes GBs size.

On very specific purpose we need to go to collection3 and do JOIN with
collection1 to fetch some data and facets
Then go to collection1 and  JOIN with collection3 to get facets of
collection1 but no data.





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR JOIN

[Paresh]

I am actually able to make this work by doing multiple JOINs -
1. Do query on Collection1
2. JOIN with Collection3 (document type1 field)
3. JOIN with Collection3 (document type2 field)
4. Do not fetch any data except faceting information

But the other problem is, it is very slow.

Are there any other solution when we have two collections and want data that
satisfy fields from both collections and also get facets from both
collection?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR JOIN

[Paresh]

Hi Erick,

I am able to achieve querying on Collection3 with INNER JOIN between two
document types and JOIN across collection1 using below mechanism. I am also
getting facetting information from collection3 along with data.

http://localhost:8983/solr/collection3/tcfts?wt=json=on=0=50=AND=OC2:(9350)
AND _query_:"{!join from=col3_Oc1 to=Oc1}ID1:xtWNf_fTAaLUgD" AND
_query_:{!join to=col3_Field1 from=Field1
fromIndex=collection1}Field2:12010340

Now, I want to collect the faceting information for collection1 with the
same type of query but no data.
So what is needed is -
Query on collection1, match few columns with some expression and then do
JOIN across collection3 do INNER JOIN as above.

Could you help on this?




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: SOLR JOIN

[Paresh]

Hi Erick,

I am able to achieve querying on Collection3 with INNER JOIN between two
document types and JOIN across collection1 using below mechanism. I am also
getting facetting information from collection3 along with data.

http://localhost:8983/solr/collection3/tcfts?wt=json=on=0=50=AND=OC2:(9350)
AND _query_:"{!join from=col3_Oc1 to=Oc1}ID1:xtWNf_fTAaLUgD" AND
_query_:{!join to=col3_Field1 from=Field1
fromIndex=collection1}Field2:12010340

Now, I want to collect the faceting information for collection1 with the
same type of query but no data.
So what is needed is -
Query on collection1, match few columns with some expression and then do
JOIN across collection3 do INNER JOIN as above.

Could you help on this?




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

SOLR JOIN

[Paresh]

Hi,

I have two collections both having different schema.
Collection1: ID, Field1, Field2
Collection3: ID, Oc1, OC2, col3_Field1
 ID1, col3_Oc1
I want to do following JOINs

(-)Query on Collection1
(-)Field2:value2
(-)JOIN across collection3 Field1 = collection3.col3_Field1
   Inner JOIN in collection 3
   col3_oc1 = Oc1
I am able to achieve SELF JOIN within a collection and across JOIN as well.

But I am not able do both together.

Any help appreciated.




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr Core not able access latest data indexed by multiple server.

[Paresh Modi]

Hi,

I am using Solr multicore approach in my app. we have two different servers
(ServerA1 and ServerA2) for load balancing, both the server accessing the
same index repository and request will go to any server as per load balance
algorithm. 

Problem occurs in following way [Note that both the servers accessing the
same physical location(index)].

- ADD TO INDEX request for File1 go to ServerA1 for core CR1, core CR1
loaded in ServerA1 and indexing done.
- ADD TO INDEX request for File2 go to ServerA2 for core CR1, core CR1
loaded in ServerA2 and indexing done.
- SEARCH request for File2 go to ServerA1, now here core CR1 is already
loaded so it directly access the index but File2 added by ServerA2 is not
found in core loaded by ServerA1.

So this is the problem, File2 indexed by core CR1 loaded in ServerA2 is not
available in core CR1 loaded by ServerA1.


I have searched and found that the solution to this problem is reload the
CORE. when you reload the core, it will have latest indexed data. but
reloading the Core for every request is very heavy and time consuming
process.

Please let me know if anyone has any solution for this.


Waiting for your expert advice.


Thanks
Paresh

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Core-not-able-access-latest-data-indexed-by-multiple-server-tp3919113p3919113.html
Sent from the Solr - User mailing list archive at Nabble.com.