Solt with no text search

[Bsr]

 0
down vote
favorite


I have successfuly implemented the custom search in solr by extending
CustomScoreProvider. But the problem is i need to pass some user info like
31,1 which i need in custom sort.

*q={!mycustomparser}31+1+*:**

and it is acting as a text search. Is there any way that i can skip this
text search and can just pass the user info.

*some code.*

public class MyCustomParserPlugin extends QParserPlugin {

@Override
public QParser createParser(String querystring, SolrParams slocParams,
SolrParams params, SolrQueryRequest req) {
return new MyCustomParser(querystring, slocParams, params, req);
}

private static class MyCustomParser extends QParser{

private Query inQuery;

public MyCustomParser(String querystring, SolrParams slocParams,
SolrParams params, SolrQueryRequest req) {
super(querystring, slocParams, params, req);
try {
QParser parser = getParser(querystring, getReq());
this.inQuery = parser.parse();
}catch(SyntaxError ex) {
throw new RuntimeException("error parsing query", ex);
}
}

@Override
public Query parse() throws SyntaxError {
return new MatchingQuery(inQuery);
}

}

}

=

public class MatchingQuery extends CustomScoreQuery {

private Query subQuery;

public MatchingQuery(Query subQuery) {
super(subQuery);
this.subQuery = subQuery;
}

@Override
protected CustomScoreProvider getCustomScoreProvider(LeafReaderContext
context) {
return new MyCustomSortClz(subQuery, context);
}
}

=

public class MyCustomSortClz extends CustomScoreProvider {

  // ---> my sort
}





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr Phrase Count : How to get count of a phrase in a text field solr

[aneeshkappu]

Hi All, I want to get the count of a phrase from a document .
Currently im using Shingle Filter factory but it consuming a large disk
space. Any alternate ways or any way to optimize this.
currently it consuming 40GB for just 46K records

my schema setting is given below 




  

 
   

 
   


 
  




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Could not find collection when upgrading to Solr 7

[Marvin Bredal Lillehaug]

This is for doing local development and running tests.
So starting with embedded zookeeper in SolrCloud mode is intentional. The
idea is to just run the script setting up the folder structure for the
cores and everything is up and running, ready for indexing and querying
locally.
The applications using Solr also use Zookeeper for configuration and
coordination of their own stuff.

There are no persmission issues.

I have put the scripts here https://github.com/computerlove/start-solr

The file config specifies what version to use. When running run-dev-solr.sh
that version is downloaded, solrhome is copied to the right place and the
instance is started.


On Sat, Feb 24, 2018 at 1:20 PM, Shawn Heisey  wrote:

> On 2/24/2018 3:13 AM, Marvin Bredal Lillehaug wrote:
>
>> We have a multicore Solr-instance that currently is running version 6.6.2.
>> For local development we run
>>
>> ./solr start -v -d $solrDir/server -h localhost -p $SOLR_PORT -c -f -s
>> $solrHomeDir -a "-Dbootstrap_conf=true -Djetty.host=localhost" -m 4096m
>>
>>
> Are you aware that the -c option, without the -z option, will start an
> embedded zookeeper server and put Solr into SolrCloud mode?  The embedded
> ZK server is not recommended except for setting up a proof of concept.  For
> production, the ZK ensemble needs at least three servers.
>
> Setting jetty.host to localhost will make it so that Solr cannot be
> accessed remotely, it will only work on the local machine.  This defeats
> one of the primary purposes of SolrCloud -- a highly available fault
> tolerant cluster.
>
> The bootstrap_conf setting is intended to be used precisely once -- in a
> situation where you want to convert a non-cloud install to SolrCloud.  It
> is not meant to be defined every time you start Solr.  I would strongly
> recommended that you do not use bootstrap at all.  Instead, start up a new
> install, create your collection how you desire, and then run your indexing
> so you create the index fresh.
>
> On Solr 6 everything starts fine, and cores and collections are created on
>> startup.
>> When running the same setup on Solr 7.x.x it crash with Could not find
>> collection : nvdb-transactionlog when running 
>> CoreContainer.repairCoreProperty.
>> Logs for Solr 6 and 7 attached.
>>
>
> That error probably means that the collection was not found in the
> ZooKeeper database.  Are you doing any kind of configuration or changes on
> the embedded ZK server before you start Solr?  Does the user that's running
> Solr have enough permissions to create and write to the ZK data directory?
> The problem *might* be the bootstrap_conf setting, but I am not sure
> whether that is correct.  Unless something is very broken in your install,
> then I would suspect the bootstrap option is causing a problem.
>
> Thanks,
> Shawn
>
>


-- 
med vennlig hilsen,
Marvin B. Lillehaug

Solt with no text search

[Bsr]

 0
down vote
favorite


I have successfuly implemented the custom search in solr by extending
CustomScoreProvider. But the problem is i need to pass some user info like
31,1 which i need in custom sort.

q={!mycustomparser}31+1+*:*

and it is acting as a text search. Is there any way that i can skip this
text search and can just pass the user info.

some code.

public class MyCustomParserPlugin extends QParserPlugin {

@Override
public QParser createParser(String querystring, SolrParams slocParams,
SolrParams params, SolrQueryRequest req) {
return new MyCustomParser(querystring, slocParams, params, req);
}

private static class MyCustomParser extends QParser{

private Query inQuery;

public MyCustomParser(String querystring, SolrParams slocParams,
SolrParams params, SolrQueryRequest req) {
super(querystring, slocParams, params, req);
try {
QParser parser = getParser(querystring, getReq());
this.inQuery = parser.parse();
}catch(SyntaxError ex) {
throw new RuntimeException("error parsing query", ex);
}
}

@Override
public Query parse() throws SyntaxError {
return new MatchingQuery(inQuery);
}

}

}

=

public class MatchingQuery extends CustomScoreQuery {

private Query subQuery;

public MatchingQuery(Query subQuery) {
super(subQuery);
this.subQuery = subQuery;
}

@Override
protected CustomScoreProvider getCustomScoreProvider(LeafReaderContext
context) {
return new MyCustomSortClz(subQuery, context);
}
}

=

public class MyCustomSortClz extends CustomScoreProvider {

  // ---> my sort
}





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Could not find collection when upgrading to Solr 7

[Shawn Heisey]

On 2/24/2018 3:13 AM, Marvin Bredal Lillehaug wrote:
We have a multicore Solr-instance that currently is running version 
6.6.2.

For local development we run

./solr start -v -d $solrDir/server -h localhost -p $SOLR_PORT -c -f -s 
$solrHomeDir -a "-Dbootstrap_conf=true -Djetty.host=localhost" -m 4096m




Are you aware that the -c option, without the -z option, will start an 
embedded zookeeper server and put Solr into SolrCloud mode?  The 
embedded ZK server is not recommended except for setting up a proof of 
concept.  For production, the ZK ensemble needs at least three servers.


Setting jetty.host to localhost will make it so that Solr cannot be 
accessed remotely, it will only work on the local machine.  This defeats 
one of the primary purposes of SolrCloud -- a highly available fault 
tolerant cluster.


The bootstrap_conf setting is intended to be used precisely once -- in a 
situation where you want to convert a non-cloud install to SolrCloud.  
It is not meant to be defined every time you start Solr.  I would 
strongly recommended that you do not use bootstrap at all.  Instead, 
start up a new install, create your collection how you desire, and then 
run your indexing so you create the index fresh.


On Solr 6 everything starts fine, and cores and collections are 
created on startup.
When running the same setup on Solr 7.x.x it crash with Could not find 
collection : nvdb-transactionlog when 
running CoreContainer.repairCoreProperty. Logs for Solr 6 and 7 attached.


That error probably means that the collection was not found in the 
ZooKeeper database.  Are you doing any kind of configuration or changes 
on the embedded ZK server before you start Solr?  Does the user that's 
running Solr have enough permissions to create and write to the ZK data 
directory?  The problem *might* be the bootstrap_conf setting, but I am 
not sure whether that is correct.  Unless something is very broken in 
your install, then I would suspect the bootstrap option is causing a 
problem.


Thanks,
Shawn

Re: Indexing timeout issues with SolrCloud 7.1

[Deepak Goel]

>From the error list, i can see multiple errors:

1. Failure to recover replica
2. Peer sync error
3. Failure to download file

On 24 Feb 2018 03:10, "Tom Peters"  wrote:

I included the last 25 lines from the logs from each of the five nodes
during that time period.

I _think_ I'm running into issues with bulking up deleteByQuery. Quick
background: we have objects in our system that may have multiple
availability windows. So when we index an object, will store it as separate
documents each with their own begins and expires date. At index time we
don't know if the all of the windows are still valid or not, so we remove
all of them with a deleteByQuery (e.g. deleteByQuery=object_id:12345) and
then index one or more documents.

I ran an isolated test a number of times where I indexed 1500 documents in
this manner (deletes then index). In Solr 3.4, it takes about 15s to
complete. In Solr 7.1, it's taking about 5m. If I remove the deleteByQuery,
the indexing times are nearly identical.

When run in normal production mode where we have lots of processes indexing
at once (~20 or so), it starts to cause lots of issues (which you see
below).


Please let me know if anything I mentioned is unclear. Thanks!




solr2-a:
2018-02-23 04:09:36.551 ERROR (updateExecutor-2-thread-2672-
processing-http:solr2-b:8080//solr//mycollection_shard1_replica_n1
x:mycollection_shard1_replica_n6 r:core_node9 n:solr2-a.vam.be.cmh.
mycollection.com:8080_solr s:shard1 c:mycollection) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6] o.a.s.u.
ErrorReportingConcurrentUpdateSolrClient error
2018-02-23 04:09:36.551 ERROR (updateExecutor-2-thread-2692-
processing-http:solr2-d:8080//solr//mycollection_shard1_replica_n11
x:mycollection_shard1_replica_n6 r:core_node9 n:solr2-a.vam.be.cmh.
mycollection.com:8080_solr s:shard1 c:mycollection) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6] o.a.s.u.
ErrorReportingConcurrentUpdateSolrClient error
2018-02-23 04:09:36.551 ERROR (updateExecutor-2-thread-2711-
processing-http:solr2-e:8080//solr//mycollection_shard1_replica_n4
x:mycollection_shard1_replica_n6 r:core_node9 n:solr2-a.vam.be.cmh.
mycollection.com:8080_solr s:shard1 c:mycollection) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6] o.a.s.u.
ErrorReportingConcurrentUpdateSolrClient error
2018-02-23 04:09:36.552 ERROR (qtp1595212853-32739) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6]
o.a.s.u.p.DistributedUpdateProcessor
Setting up to try to start recovery on replica http://solr2-b:8080/solr/
mycollection_shard1_replica_n1/
2018-02-23 04:09:36.552 ERROR (qtp1595212853-32739) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6]
o.a.s.u.p.DistributedUpdateProcessor
Setting up to try to start recovery on replica http://solr2-d:8080/solr/
mycollection_shard1_replica_n11/
2018-02-23 04:09:36.552 ERROR (qtp1595212853-32739) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6]
o.a.s.u.p.DistributedUpdateProcessor
Setting up to try to start recovery on replica http://solr2-e:8080/solr/
mycollection_shard1_replica_n4/
2018-02-23 04:09:38.217 ERROR (updateExecutor-2-thread-2712-
processing-http:solr2-e:8080//solr//mycollection_shard1_replica_n4
x:mycollection_shard1_replica_n6 r:core_node9 n:solr2-a.vam.be.cmh.
mycollection.com:8080_solr s:shard1 c:mycollection) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6] o.a.s.u.
ErrorReportingConcurrentUpdateSolrClient error
2018-02-23 04:09:38.217 ERROR (updateExecutor-2-thread-2726-
processing-http:solr2-d:8080//solr//mycollection_shard1_replica_n11
x:mycollection_shard1_replica_n6 r:core_node9 n:solr2-a.vam.be.cmh.
mycollection.com:8080_solr s:shard1 c:mycollection) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6] o.a.s.u.
ErrorReportingConcurrentUpdateSolrClient error
2018-02-23 04:09:38.217 ERROR (updateExecutor-2-thread-2727-
processing-http:solr2-b:8080//solr//mycollection_shard1_replica_n1
x:mycollection_shard1_replica_n6 r:core_node9 n:solr2-a.vam.be.cmh.
mycollection.com:8080_solr s:shard1 c:mycollection) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6] o.a.s.u.
ErrorReportingConcurrentUpdateSolrClient error
2018-02-23 04:09:38.218 ERROR (qtp1595212853-32260) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6]
o.a.s.u.p.DistributedUpdateProcessor
Setting up to try to start recovery on replica http://solr2-b:8080/solr/
mycollection_shard1_replica_n1/
2018-02-23 04:09:38.218 ERROR (qtp1595212853-32260) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6]
o.a.s.u.p.DistributedUpdateProcessor
Setting up to try to start recovery on replica http://solr2-d:8080/solr/
mycollection_shard1_replica_n11/
2018-02-23 04:09:38.218 ERROR (qtp1595212853-32260) [c:mycollection
s:shard1 r:core_node9 x:mycollection_shard1_replica_n6]