Problem using user.timezone in DIH

[Jamie Jackson]

Solr 6.6.0

I'm getting a weird issue and a misleading error when I try to use the
user.timezone property in a data import handler configuration which has
child entities:

Caused by: java.sql.SQLInvalidAuthorizationSpecException: Could not
connect: Access denied for user '{}'@'111.222.111.222' (using password: YES)

I was trying this in a query for a delta import:

CONVERT_TZ('${dataimporter.last_index_time}','${user.timezone}','${
custom.dataimporter.datasource.tz}')

It works great for cores with no child entities in the DIH config.

However, if I have any child entities, it dies. Here's a failing
configuration and its stack trace:
https://gist.github.com/jamiejackson/1ec544e6b87607098fbb1077a4eece30

I isolated it down to the use of user.timezone. Even if I simply add the
following in a SQL _comment_, it causes the problem. For example:

select /* ${user.timezone} */ column_a, column_b, ...

Bug?

Thanks,
Jamie

Re: Replicates not recovering after rolling restart

[Erick Erickson]

Gah! Don't you hate it when you spend days on something like this?

Slight clarification. _version_ is used for optimistic locking, not
replication. Let's say you have two clients updating the same document
and sending it to Solr at the same time. The _version_ field is filled
out automagically and one of the updates will be rejected. Otherwise
there'd be no good way to fail a document due to this kind of thing.

Thanks for letting us know what the problem really was.

Best,
Erick

On Fri, Sep 22, 2017 at 2:57 PM, Bill Oconnor  wrote:
>
> Thanks everyone for the responses.
>
>
> I believe I have found the problem.
>
>
> The type of __version__ is incorrect in our schema. This is a required field 
> that is primarily used by Solr.
>
>
> Our schema has typed it as type=int instead of  type=long
>
>
> I believe that this number is used by the replication process to figure out 
> what needs to be sync'd on an
>
> individual replicate. In our case Solr puts the value in during indexing. It 
> appears that Solr has chosen a
>
> number that cannot be represented by "int". As the replicates query the 
> leader to determine if a sync is
>
> necessary the the leader throws an error as it try's to format the response 
> with the large _version_ .
>
> This process continues until the replicates give up.
>
>
> I finally verified this by doing a simple query _version_:*which throws 
> the same error but gives
>
> more helpful info "re-index your documents"
>
>
> Thanks.
>
>
>
>
>
> 
> From: Rick Leir 
> Sent: Friday, September 22, 2017 12:34:57 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Replicates not recovering after rolling restart
>
> Wunder, Erick
>
> $ dc
> 16o
> 1578578283947098112p
> 15E83C95E8D0
>
> That is an interesting number. Is it, as a guess, machine instructions
> or an address pointer? It does not look like UTF-8 or ASCII. Machine
> code looks promising:
>
>
> Disassembly:
>
> 0:  15 e8 3c 95 e8  adceax,0xe8953ce8
> 5:  d0 00   rolBYTE PTR [rax],1
> 
>
> /ADC/dest,src Modifies flags: AF CF OF SF PF ZF Sums two binary operands
> placing the result in the destination.
>
> *ROL - Rotate Left*
>
> Registers: the/64-bit/extension of/eax/is called/rax/.
>
> Is that code possibly in the JVM executable? Or a random memory page.
>
> cheers -- Rick
>
> On 2017-09-20 07:21 PM, Walter Underwood wrote:
>> 1578578283947098112 needs 61 bits. Is it being parsed into a 32 bit target?
>>
>> That doesn’t explain where it came from, of course.
>>
>> wunder
>> Walter Underwood
>> wun...@wunderwood.org
>> http://observer.wunderwood.org/  (my blog)
>>
>>
>>> On Sep 20, 2017, at 3:35 PM, Erick Erickson  wrote:
>>>
>>> The numberformatexception is...odd. Clearly that's too big a number
>>> for an integer, did anything in the underlying schema change?
>>>
>>> Best,
>>> Erick
>>>
>>> On Wed, Sep 20, 2017 at 3:00 PM, Walter Underwood  
>>> wrote:
 Rolling restarts work fine for us. I often include installing new configs 
 with that. Here is our script. Pass it any hostname in the cluster. I use 
 the load balancer name. You’ll need to change the domain and the install 
 directory of course.

 #!/bin/bash

 cluster=$1

 hosts=`curl -s 
 "http://${cluster}:8983/solr/admin/collections?action=CLUSTERSTATUS=json;
  | jq -r '.cluster.live_nodes[]' | sort`

 for host in $hosts
 do
 host="${host}.cloud.cheggnet.com"
 echo restarting Solr on $host
 ssh $host 'cd /apps/solr6 ; sudo -u bin bin/solr stop; sudo -u bin 
 bin/solr start -cloud -h `hostname`'
 done


 Walter Underwood
 wun...@wunderwood.org
 http://observer.wunderwood.org/  (my blog)


> On Sep 20, 2017, at 1:42 PM, Bill Oconnor  wrote:
>
> Hello,
>
>
> Background:
>
>
> We have been successfully using Solr for over 5 years and we recently 
> made the decision to move into SolrCloud. For the most part that has been 
> easy but we have repeated problems with our rolling restart were server 
> remain functional but stay in Recovery until they stop trying. We 
> restarted because we increased the memory from 12GB to 16GB on the JVM.
>
>
> Does anyone have any insight as to what is going on here?
>
> Is there a special procedure I should use for starting a stopping host?
>
> Is it ok to do a rolling restart on all the nodes in s shard?
>
>
> Any insight would be appreciated.
>
>
> Configuration:
>
>
> We have a group of servers with multiple collections. Each collection 
> consist of one shard and multiple replicates. We are running the latest 
> stable version of SolrClound 6.6 on Ubuntu LTS and Oracle Corporation 
> Java HotSpot(TM) 64-Bit Server VM 

Re: Replicates not recovering after rolling restart

[Bill Oconnor]

Thanks everyone for the responses.


I believe I have found the problem.


The type of __version__ is incorrect in our schema. This is a required field 
that is primarily used by Solr.


Our schema has typed it as type=int instead of  type=long


I believe that this number is used by the replication process to figure out 
what needs to be sync'd on an

individual replicate. In our case Solr puts the value in during indexing. It 
appears that Solr has chosen a

number that cannot be represented by "int". As the replicates query the leader 
to determine if a sync is

necessary the the leader throws an error as it try's to format the response 
with the large _version_ .

This process continues until the replicates give up.


I finally verified this by doing a simple query _version_:*which throws the 
same error but gives

more helpful info "re-index your documents"


Thanks.






From: Rick Leir 
Sent: Friday, September 22, 2017 12:34:57 AM
To: solr-user@lucene.apache.org
Subject: Re: Replicates not recovering after rolling restart

Wunder, Erick

$ dc
16o
1578578283947098112p
15E83C95E8D0

That is an interesting number. Is it, as a guess, machine instructions
or an address pointer? It does not look like UTF-8 or ASCII. Machine
code looks promising:


Disassembly:

0:  15 e8 3c 95 e8  adceax,0xe8953ce8
5:  d0 00   rolBYTE PTR [rax],1


/ADC/dest,src Modifies flags: AF CF OF SF PF ZF Sums two binary operands
placing the result in the destination.

*ROL - Rotate Left*

Registers: the/64-bit/extension of/eax/is called/rax/.

Is that code possibly in the JVM executable? Or a random memory page.

cheers -- Rick

On 2017-09-20 07:21 PM, Walter Underwood wrote:
> 1578578283947098112 needs 61 bits. Is it being parsed into a 32 bit target?
>
> That doesn’t explain where it came from, of course.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Sep 20, 2017, at 3:35 PM, Erick Erickson  wrote:
>>
>> The numberformatexception is...odd. Clearly that's too big a number
>> for an integer, did anything in the underlying schema change?
>>
>> Best,
>> Erick
>>
>> On Wed, Sep 20, 2017 at 3:00 PM, Walter Underwood  
>> wrote:
>>> Rolling restarts work fine for us. I often include installing new configs 
>>> with that. Here is our script. Pass it any hostname in the cluster. I use 
>>> the load balancer name. You’ll need to change the domain and the install 
>>> directory of course.
>>>
>>> #!/bin/bash
>>>
>>> cluster=$1
>>>
>>> hosts=`curl -s 
>>> "http://${cluster}:8983/solr/admin/collections?action=CLUSTERSTATUS=json;
>>>  | jq -r '.cluster.live_nodes[]' | sort`
>>>
>>> for host in $hosts
>>> do
>>> host="${host}.cloud.cheggnet.com"
>>> echo restarting Solr on $host
>>> ssh $host 'cd /apps/solr6 ; sudo -u bin bin/solr stop; sudo -u bin 
>>> bin/solr start -cloud -h `hostname`'
>>> done
>>>
>>>
>>> Walter Underwood
>>> wun...@wunderwood.org
>>> http://observer.wunderwood.org/  (my blog)
>>>
>>>
 On Sep 20, 2017, at 1:42 PM, Bill Oconnor  wrote:

 Hello,


 Background:


 We have been successfully using Solr for over 5 years and we recently made 
 the decision to move into SolrCloud. For the most part that has been easy 
 but we have repeated problems with our rolling restart were server remain 
 functional but stay in Recovery until they stop trying. We restarted 
 because we increased the memory from 12GB to 16GB on the JVM.


 Does anyone have any insight as to what is going on here?

 Is there a special procedure I should use for starting a stopping host?

 Is it ok to do a rolling restart on all the nodes in s shard?


 Any insight would be appreciated.


 Configuration:


 We have a group of servers with multiple collections. Each collection 
 consist of one shard and multiple replicates. We are running the latest 
 stable version of SolrClound 6.6 on Ubuntu LTS and Oracle Corporation Java 
 HotSpot(TM) 64-Bit Server VM 1.8.0_66 25.66-b17


 (collection)  (shard)  (replicates)

 journals_stage   ->  shard1  ->  solr-220 (leader) , solr-223, solr-221, 
 solr-222 (replicates)


 Problem:


 Restarting the system puts the replicates in a recovery state they never 
 exit from. They eventually give up after 500 tries.  If I go to the 
 individual replicates and execute a query the data is still available.


 Using tcpdump I find the replicates sending this request to the leader 
 (the leader appears to be active).


 The exchange goes  like this - :


 solr-220 is the leader.

 Solr-221 to Solr-220


 10:18:42.426823 IP solr-221:54341 > 

RE: AEM SOLR integaration

[Davis, Daniel (NIH/NLM) [C]]

Gunalan,

I think this depends on your system environment.   It is a general "service 
discovery" issue.   On-premise, my organization uses f5 BigIP as a load 
balancer, and so we merely have f5 LTM direct traffic from one name to any of a 
number of Solr instances.   If they are all SolrCloud, it mostly just works.

In AWS Cloud, the same thing could work with an Elastic Load Balancer (ELB) or 
Application Load Balancer (ALB), which is more flexible.

AEM Solr Search appears to be for embedding search results into AEM, rather 
than to index AEM content in a structured manner.   These are two different but 
related features.   Which are you looking to do?

Hope this helps,

-Dan

-Original Message-
From: Gunalan V [mailto:visagan2...@gmail.com] 
Sent: Friday, September 22, 2017 7:32 AM
To: solr-user@lucene.apache.org
Subject: Re: AEM SOLR integaration

Thank You!

I was looking for some suggestions in building the SOLR infrastructure.

Like how each AEM instance should point to ? Might be one AEM instance to one 
SOLR cloud (With internal zookeeper) in all environments or any specific 
architecture we need to follow while going with AEM.



Thanks,
GVK

On Fri, Sep 22, 2017 at 02:58 Atita Arora  wrote:

>
> https://www.slideshare.net/DEEPAKKHETAWAT/basics-of-solr-and-solr-inte
> gration-with-aem6-61150010
>
> This could probably help too along with the link Nicole shared.
>
> On Fri, Sep 22, 2017 at 12:28 PM, Nicole Bilić 
> 
> wrote:
>
> > Hi,
> >
> > Maybe this could help you out http://www.aemsolrsearch.com/
> >
> > Regards,
> > Nicole
> >
> > On Sep 22, 2017 05:41, "Gunalan V"  wrote:
> >
> > > Hello,
> > >
> > > I'm looking for suggestion in building the SOLR infrastructure so
> Kindly
> > > let me know if anyone has integerated AEM (Adobe Experience 
> > > Manager)
> with
> > > SOLR?
> > >
> > >
> > >
> > > Thanks,
> > > GVK
> > >
> >
>

Re: How to build solr

[Erick Erickson]

1> do you have ant installed? My guess is yes but thought I'd check
2> what target are you executing? "ant server"? "ant dist"? ???
3> the first time you execute an ant task you should see a message
about a target to install "ivy", did you see it and did you follow it?
That should install ivy-2.4.0.jar (or similar) in ./ant/lib

Best,
Erick

On Fri, Sep 22, 2017 at 3:11 AM, Sidana, Mohit
 wrote:
> Here's a very nice article on " How to Debug Solr With Eclipse" which I have 
> used previously to debug solr with eclipse.
>
> http://opensourceconnections.com/blog/2013/04/13/how-to-debug-solr-with-eclipse/
>
>
> -Original Message-
> From: Atita Arora [mailto:atitaar...@gmail.com]
> Sent: Friday, September 22, 2017 10:07 AM
> To: solr-user@lucene.apache.org
> Subject: Re: How to build solr
>
> http://www.gingercart.com/Home/search-and-crawl/build-and-run-solr-from-source
>
> and  follow thread
>
> http://lucene.472066.n3.nabble.com/running-solr-in-debug-through-eclipse-td4159777.html
>
> to run solr server in debug mode through eclipse.
>
> Should give you some hint.
>
> Let me go through your error again to see , if I get some clue there.
>
> -Atita
>
> On Fri, Sep 22, 2017 at 11:41 AM, srini sampath > wrote:
>
>> Thanks Aman,
>> Erick, I followed the link and I am getting the following error,
>>
>> Buildfile: ${user.home}\git\lucene-solr\build.xml
>>
>> compile:
>>
>> -check-git-state:
>>
>> -git-cleanroot:
>>
>> -copy-git-state:
>>
>> git-autoclean:
>>
>> resolve:
>>
>> ivy-availability-check:
>>
>> BUILD FAILED
>> ${user.home}\git\lucene-solr\build.xml:309: The following error
>> occurred while executing this line:
>> ${user.home}\git\lucene-solr\lucene\build.xml:124: The following error
>> occurred while executing this line:
>> ${user.home}\git\lucene-solr\lucene\common-build.xml:424:
>> ${user.home}\.ant\lib does not exist.
>>
>> Total time: 0 seconds
>>
>> Any Idea?
>> How can I run solr server In debug mode.
>>
>> Here is the thing I am trying to do,
>> Change a custom plugin called solrTextTagger
>> and add some extra
>> query parameters to it.
>>
>> I defined my custom handler in the following way
>>
>>- >class="org.opensextant.solrtexttagger.TaggerRequestHandler">
>>
>>
>>
>>- And I defined my custom handler jar file location location in
>>solrschema.xml in the following way
>>
>>   
>> (solr-text-tagger.jar
>> location)
>>
>>- I made some changes to the solrTextTagger,
>> And built a jar using
>>maven.
>>- I am running solr as a service. And sending a request using HTTP Post
>>method.
>>- But the problem is how can I debug solr-text-tagger.jar code to check
>>and make changes. (I mean how to do remote debugging?)
>>
>>
>> I am using eclipse IDE for development.
>> I found similar problem here
>> > in-Solr-Request-Handler-plugin-and-its-debugging-td4077533.html>.
>> But I could not understand the solution.
>>
>> .Best,
>> Srini Sampth.
>>
>>
>>
>>
>>
>> On Thu, Sep 21, 2017 at 8:51 PM, Erick Erickson
>> 
>> wrote:
>>
>> > And did you follow the link provided on that page?
>> >
>> > Best,
>> > Erick
>> >
>> > On Thu, Sep 21, 2017 at 3:07 AM, Aman Tandon
>> > 
>> > wrote:
>> > > Hi Srini,
>> > >
>> > > Kindly refer to the READ.ME section of this link of GitHub, this
>> should
>> > > work.
>> > > https://github.com/apache/lucene-solr/blob/master/README.md
>> > >
>> > > With regards,
>> > > Aman Tandon
>> > >
>> > >
>> > > On Sep 21, 2017 1:53 PM, "srini sampath"
>> > > 
>> > > wrote:
>> > >
>> > >> Hi,
>> > >> How to build and compile solr in my locale machine? it seems the
>> > >> https://wiki.apache.org/solr/HowToCompileSolr page became obsolete.
>> > >> Thanks in advance
>> > >>
>> >
>>

Re: Error Opening new IndexSearcher - LockObtainFailedException

[Erick Erickson]

Hmmm, 6.4 was considerably before the refactoring that this patch
addresses so it's not a surprise that it doesn't apply.

On Thu, Sep 21, 2017 at 10:28 PM, Shashank Pedamallu
 wrote:
> Hi Luiz,
>
> Unfortunately, I’m on version Solr-6.4.2 and the patch does not apply 
> straight away.
>
> Thanks,
> Shashank
>
> On 9/21/17, 8:35 PM, "Luiz Armesto"  wrote:
>
> Hi Shashank,
>
> There is an open issue about this exception [1]. Can you take a look and
> test the patch to see if it works in your case?
>
> [1] 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SOLR-2D11297=DwIFaQ=uilaK90D4TOVoH58JNXRgQ=blJD2pBapH3dDkoajIf9mT9SSbbs19wRbChNde1ErNI=EBLEhJ6TlQpK4rJngNBxBwypGpdbAuhnuqmgiRGcxZg=j69wKZOK2Ve9oeIPl92iyiQLSZS38Qe-ZLj-2OeN-u0=
>
> On Sep 21, 2017 10:19 PM, "Shashank Pedamallu" 
> wrote:
>
> Hi,
>
> I’m seeing the following exception in Solr that gets automatically 
> resolved
> eventually.
> 2017-09-22 00:18:17.243 ERROR (qtp1702660825-17) [   x:spedamallu1-core-1]
> o.a.s.c.CoreContainer Error creating core [spedamallu1-core-1]: Error
> opening new searcher
> org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.(SolrCore.java:952)
> at org.apache.solr.core.SolrCore.(SolrCore.java:816)
> at 
> org.apache.solr.core.CoreContainer.create(CoreContainer.java:890)
> at org.apache.solr.core.CoreContainer.getCore(
> CoreContainer.java:1167)
> at 
> org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)
> at 
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:418)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:345)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:296)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1691)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:582)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)
> at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:226)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:512)
> at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1112)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:213)
> at org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:119)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:534)
> at 
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:251)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(
> AbstractConnection.java:273)
> at 
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(
> SelectChannelEndPoint.java:93)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> executeProduceConsume(ExecuteProduceConsume.java:303)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> produceConsume(ExecuteProduceConsume.java:148)
> at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
> ExecuteProduceConsume.java:136)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:671)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:589)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Error opening new 
> searcher
> at 
> org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1891)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2011)
> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1041)
> at org.apache.solr.core.SolrCore.(SolrCore.java:925)
> 

Re: Solr nodes crashing (OOM) after 6.6 upgrade

[shamik]

Susheel, my inference was based on the Qtime value from Solr log and not
based on application log. Before the CPU spike, the query time didn’t give
any indication that they are slow in the process of slowing down. As the GC
suddenly triggers a high CPU usage, query execution slows down or chocks,
but that can easily be attributed to the lack of available processing power.

I’m curious to know what’s the recommended hardware for 6.6 having 50gb
index size and 15 million+ documents.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: mm is not working if you have same term multiple times in query

[Aman Deep Singh]

We can't use shingles as user can query lock and lock ,or any other
combination although and and some other words can be passed in stop word
processing but can't rely on that completely.

On 22-Sep-2017 7:00 PM, "Emir Arnautović" 
wrote:

It seems to me that all OOTB solution would include some query parsing on
client side.
If those are adjacent values, you could try play with shingles to get it to
work.
Brainstorming: custom token filter that would assign token occurrence
number to each token: e.g.
“foo lock bar lock” would be indexed as foo1 lock1 bar1 lock2, but that
would mess score…

Maybe there is something specific about your usecase that could be used to
make it work.

Emir

> On 22 Sep 2017, at 15:17, Aman Deep Singh 
wrote:
>
> Hi Emir,
> Thanks for the reply,
> I understand how the dismax/edismax works ,my problem is I don't want to
> show the results with one token only ,
> I cannot use phrase query here because the phrase query doesn't work with
> single word query so to do so we need to change the search request (qf or
> pf )dynamically ,will definitely try to use the function query.
>
> Thanks,
> Aman Deep Singh
>
> On 22-Sep-2017 6:25 PM, "Emir Arnautović" 
> wrote:
>
>> Hi Aman,
>> You have wrong expectations: Edismax does respect mm, it’s just that it
is
>> met. If you take a look at parsed query, it’ll be something like:
>> +(((name:lock) (name:lock))~2)
>> And from dismax perspective it found both terms. It will not start
>> searching for the next term after first is found or look at term
frequency.
>> You can use phrase query to make sure that lock is close to lock or use
>> function query to make sure tf requirement is met.
>> Not sure what is your usecase.
>>
>> HTH,
>> Emir
>>
>>> On 22 Sep 2017, at 12:52, Aman Deep Singh 
>> wrote:
>>>
>>> Hi,
>>> I'm using Solr 6.6.0 i have set mm as 100% but when i have the repeated
>>> search term then mm param is not honoured
>>>
>>> I have 2 docs in index
>>> Doc1-
>>> name=lock
>>> Doc 2-
>>> name=lock lock
>>>
>>> Now when i'm quering the solr with query
>>> *
>> http://localhost:8983/solr/test2/select?defType=dismax;
qf=name=on=100%25=lock%20lock=json
>>> <
>> http://localhost:8983/solr/test2/select?defType=dismax;
qf=name=on=100%25=lock%20lock=json
>>> *
>>> then it is returning both results but it should return only Doc 2 as no
>> of
>>> frequency is 2 in query while doc1 has frequency of 1 (lock term
>> frequency).
>>> Any Idea what to do ,to avoid getting doc 1 in resultset as i don't want
>>> user to get the Doc1.
>>> Schema
>>> > stored="true"/>
>>> >> autoGeneratePhraseQueries="false" positionIncrementGap="100"> > type
>>> ="index">  > class=
>>> "solr.LowerCaseFilterFactory"/>   <
>>> tokenizer class="solr.StandardTokenizerFactory"/> >> "solr.ManagedSynonymFilterFactory" managed="synonyms_gdn"/> > class=
>>> "solr.LowerCaseFilterFactory"/>  
>>>
>>> Their is no synonym is added also.
>>>
>>> Thanks,
>>> Aman Deep Singh
>>
>>

Re: Strange Behavior When Extracting Features

[alessandro.benedetti]

I think this has nothing to do with the LTR plugin.
The problem here should be just the way you use the local params,
to properly pass multi term local params in Solr you need to use *'* :

efi.case_description='added couple of fiber channel'

This should work.
If not only the first term will be passed as a local param and then passed
in the efi map to LTR.

I will update the Jira issue as well.

Cheers





-
---
Alessandro Benedetti
Search Consultant, R Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: mm is not working if you have same term multiple times in query

[Emir Arnautović]

It seems to me that all OOTB solution would include some query parsing on 
client side. 
If those are adjacent values, you could try play with shingles to get it to 
work. 
Brainstorming: custom token filter that would assign token occurrence number to 
each token: e.g.
“foo lock bar lock” would be indexed as foo1 lock1 bar1 lock2, but that would 
mess score…

Maybe there is something specific about your usecase that could be used to make 
it work.

Emir

> On 22 Sep 2017, at 15:17, Aman Deep Singh  wrote:
> 
> Hi Emir,
> Thanks for the reply,
> I understand how the dismax/edismax works ,my problem is I don't want to
> show the results with one token only ,
> I cannot use phrase query here because the phrase query doesn't work with
> single word query so to do so we need to change the search request (qf or
> pf )dynamically ,will definitely try to use the function query.
> 
> Thanks,
> Aman Deep Singh
> 
> On 22-Sep-2017 6:25 PM, "Emir Arnautović" 
> wrote:
> 
>> Hi Aman,
>> You have wrong expectations: Edismax does respect mm, it’s just that it is
>> met. If you take a look at parsed query, it’ll be something like:
>> +(((name:lock) (name:lock))~2)
>> And from dismax perspective it found both terms. It will not start
>> searching for the next term after first is found or look at term frequency.
>> You can use phrase query to make sure that lock is close to lock or use
>> function query to make sure tf requirement is met.
>> Not sure what is your usecase.
>> 
>> HTH,
>> Emir
>> 
>>> On 22 Sep 2017, at 12:52, Aman Deep Singh 
>> wrote:
>>> 
>>> Hi,
>>> I'm using Solr 6.6.0 i have set mm as 100% but when i have the repeated
>>> search term then mm param is not honoured
>>> 
>>> I have 2 docs in index
>>> Doc1-
>>> name=lock
>>> Doc 2-
>>> name=lock lock
>>> 
>>> Now when i'm quering the solr with query
>>> *
>> http://localhost:8983/solr/test2/select?defType=dismax=name=on=100%25=lock%20lock=json
>>> <
>> http://localhost:8983/solr/test2/select?defType=dismax=name=on=100%25=lock%20lock=json
>>> *
>>> then it is returning both results but it should return only Doc 2 as no
>> of
>>> frequency is 2 in query while doc1 has frequency of 1 (lock term
>> frequency).
>>> Any Idea what to do ,to avoid getting doc 1 in resultset as i don't want
>>> user to get the Doc1.
>>> Schema
>>> > stored="true"/>
>>> >> autoGeneratePhraseQueries="false" positionIncrementGap="100"> > type
>>> ="index">  > class=
>>> "solr.LowerCaseFilterFactory"/>   <
>>> tokenizer class="solr.StandardTokenizerFactory"/> >> "solr.ManagedSynonymFilterFactory" managed="synonyms_gdn"/> > class=
>>> "solr.LowerCaseFilterFactory"/>  
>>> 
>>> Their is no synonym is added also.
>>> 
>>> Thanks,
>>> Aman Deep Singh
>> 
>> 

Re: Indexing CSV files with filenames

[sophia250]

how did you find the file name? When i do posting, i use wildcard *.csv. Your
way seems work only when you are posting by each



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: mm is not working if you have same term multiple times in query

[Aman Deep Singh]

Hi Emir,
Thanks for the reply,
I understand how the dismax/edismax works ,my problem is I don't want to
show the results with one token only ,
I cannot use phrase query here because the phrase query doesn't work with
single word query so to do so we need to change the search request (qf or
pf )dynamically ,will definitely try to use the function query.

Thanks,
Aman Deep Singh

On 22-Sep-2017 6:25 PM, "Emir Arnautović" 
wrote:

> Hi Aman,
> You have wrong expectations: Edismax does respect mm, it’s just that it is
> met. If you take a look at parsed query, it’ll be something like:
> +(((name:lock) (name:lock))~2)
> And from dismax perspective it found both terms. It will not start
> searching for the next term after first is found or look at term frequency.
> You can use phrase query to make sure that lock is close to lock or use
> function query to make sure tf requirement is met.
> Not sure what is your usecase.
>
> HTH,
> Emir
>
> > On 22 Sep 2017, at 12:52, Aman Deep Singh 
> wrote:
> >
> > Hi,
> > I'm using Solr 6.6.0 i have set mm as 100% but when i have the repeated
> > search term then mm param is not honoured
> >
> > I have 2 docs in index
> > Doc1-
> > name=lock
> > Doc 2-
> > name=lock lock
> >
> > Now when i'm quering the solr with query
> > *
> http://localhost:8983/solr/test2/select?defType=dismax=name=on=100%25=lock%20lock=json
> > <
> http://localhost:8983/solr/test2/select?defType=dismax=name=on=100%25=lock%20lock=json
> >*
> > then it is returning both results but it should return only Doc 2 as no
> of
> > frequency is 2 in query while doc1 has frequency of 1 (lock term
> frequency).
> > Any Idea what to do ,to avoid getting doc 1 in resultset as i don't want
> > user to get the Doc1.
> > Schema
> >  stored="true"/>
> >  > autoGeneratePhraseQueries="false" positionIncrementGap="100">  type
> > ="index">   class=
> > "solr.LowerCaseFilterFactory"/>   <
> > tokenizer class="solr.StandardTokenizerFactory"/>  > "solr.ManagedSynonymFilterFactory" managed="synonyms_gdn"/>  class=
> > "solr.LowerCaseFilterFactory"/>  
> >
> > Their is no synonym is added also.
> >
> > Thanks,
> > Aman Deep Singh
>
>

Re: Solr nodes crashing (OOM) after 6.6 upgrade

[Susheel Kumar]

It may happen that you may never find the queries/query time being logged
for the queries which caused OOM and your app never got chance to log how
much time it took...

So if you had proper exception handled in your client code, you may see
exception being logged but not see the query time for such queries.

Thnx

On Fri, Sep 22, 2017 at 6:32 AM, shamik  wrote:

> I usually log queries that took more than 1sec. Based on the logs, I
> haven't
> seen anything alarming or surge in terms of slow queries, especially around
> the time when the CPU spike happened.
>
> I don't necessarily have the data for deep paging, but the usage of sort
> parameter (date in our case) has been typically low. We also restrict 10
> results per page for pagination. Are there are recommendations around this?
>
> Again, I don't want to sound like a broken record, but I still don't get
> the
> part why these issues crop in 6.6 as compared to 5.5
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: mm is not working if you have same term multiple times in query

[Emir Arnautović]

Hi Aman,
You have wrong expectations: Edismax does respect mm, it’s just that it is met. 
If you take a look at parsed query, it’ll be something like:
+(((name:lock) (name:lock))~2)
And from dismax perspective it found both terms. It will not start searching 
for the next term after first is found or look at term frequency. You can use 
phrase query to make sure that lock is close to lock or use function query to 
make sure tf requirement is met.
Not sure what is your usecase.

HTH,
Emir

> On 22 Sep 2017, at 12:52, Aman Deep Singh  wrote:
> 
> Hi,
> I'm using Solr 6.6.0 i have set mm as 100% but when i have the repeated
> search term then mm param is not honoured
> 
> I have 2 docs in index
> Doc1-
> name=lock
> Doc 2-
> name=lock lock
> 
> Now when i'm quering the solr with query
> *http://localhost:8983/solr/test2/select?defType=dismax=name=on=100%25=lock%20lock=json
> *
> then it is returning both results but it should return only Doc 2 as no of
> frequency is 2 in query while doc1 has frequency of 1 (lock term frequency).
> Any Idea what to do ,to avoid getting doc 1 in resultset as i don't want
> user to get the Doc1.
> Schema
> 
>  autoGeneratePhraseQueries="false" positionIncrementGap="100">  ="index">   "solr.LowerCaseFilterFactory"/>   <
> tokenizer class="solr.StandardTokenizerFactory"/>  "solr.ManagedSynonymFilterFactory" managed="synonyms_gdn"/>  "solr.LowerCaseFilterFactory"/>  
> 
> Their is no synonym is added also.
> 
> Thanks,
> Aman Deep Singh

Re: AEM SOLR integaration

[Gunalan V]

Thank You!

I was looking for some suggestions in building the SOLR infrastructure.

Like how each AEM instance should point to ? Might be one AEM instance to
one SOLR cloud (With internal zookeeper) in all environments or any
specific architecture we need to follow while going with AEM.



Thanks,
GVK

On Fri, Sep 22, 2017 at 02:58 Atita Arora  wrote:

>
> https://www.slideshare.net/DEEPAKKHETAWAT/basics-of-solr-and-solr-integration-with-aem6-61150010
>
> This could probably help too along with the link Nicole shared.
>
> On Fri, Sep 22, 2017 at 12:28 PM, Nicole Bilić 
> wrote:
>
> > Hi,
> >
> > Maybe this could help you out http://www.aemsolrsearch.com/
> >
> > Regards,
> > Nicole
> >
> > On Sep 22, 2017 05:41, "Gunalan V"  wrote:
> >
> > > Hello,
> > >
> > > I'm looking for suggestion in building the SOLR infrastructure so
> Kindly
> > > let me know if anyone has integerated AEM (Adobe Experience Manager)
> with
> > > SOLR?
> > >
> > >
> > >
> > > Thanks,
> > > GVK
> > >
> >
>

mm is not working if you have same term multiple times in query

[Aman Deep Singh]

Hi,
I'm using Solr 6.6.0 i have set mm as 100% but when i have the repeated
search term then mm param is not honoured

I have 2 docs in index
Doc1-
name=lock
Doc 2-
name=lock lock

Now when i'm quering the solr with query
*http://localhost:8983/solr/test2/select?defType=dismax=name=on=100%25=lock%20lock=json
*
then it is returning both results but it should return only Doc 2 as no of
frequency is 2 in query while doc1 has frequency of 1 (lock term frequency).
Any Idea what to do ,to avoid getting doc 1 in resultset as i don't want
user to get the Doc1.
Schema

  <
tokenizer class="solr.StandardTokenizerFactory"/>

Their is no synonym is added also.

Thanks,
Aman Deep Singh

Re: Solr nodes crashing (OOM) after 6.6 upgrade

[shamik]

I usually log queries that took more than 1sec. Based on the logs, I haven't
seen anything alarming or surge in terms of slow queries, especially around
the time when the CPU spike happened.

I don't necessarily have the data for deep paging, but the usage of sort
parameter (date in our case) has been typically low. We also restrict 10
results per page for pagination. Are there are recommendations around this?

Again, I don't want to sound like a broken record, but I still don't get the
part why these issues crop in 6.6 as compared to 5.5  



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr nodes crashing (OOM) after 6.6 upgrade

[Michael Kuhlmann]

Hi Shamik,

funny enough, we had a similar issue with our old legacy application
that still used plain Lucene code in a JBoss container.

Same, there were no specific queries or updates causing this, the
performance just broke completely without unusual usage. GC was raising
up to 99% or so. Sometimes it came back after some while but most often
we had to completely restart JBoss for that.

I never figured out what the root cause was, but my suspicion still is
that Lucene was innocent. I rather suspect Rackspace's hypervisor to be
the blamable.

So maybe you can give it a try and have a look at the Amazon cloud settings?

Best,
Michael

Am 22.09.2017 um 12:00 schrieb shamik:
> All the tuning and scaling down of memory seemed to be stable for a couple of
> days but then came down due to a huge spike in CPU usage, contributed by G1
> Old Generation GC. I'm really puzzled why the instances are suddenly
> behaving like this. It's not that a sudden surge of load contributed to
> this, query and indexing load seemed to be comparable with the previous time
> frame. Just wondering if the hardware itself is not adequate enough for 6.6.
> The instances are all running on 8 CPU / 30gb m3.2xlarge EC2 instances.
> 
> Does anyone ever face issues similar to this?
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> 

RE: How to build solr

[Sidana, Mohit]

Here's a very nice article on " How to Debug Solr With Eclipse" which I have 
used previously to debug solr with eclipse.

http://opensourceconnections.com/blog/2013/04/13/how-to-debug-solr-with-eclipse/


-Original Message-
From: Atita Arora [mailto:atitaar...@gmail.com] 
Sent: Friday, September 22, 2017 10:07 AM
To: solr-user@lucene.apache.org
Subject: Re: How to build solr

http://www.gingercart.com/Home/search-and-crawl/build-and-run-solr-from-source

and  follow thread

http://lucene.472066.n3.nabble.com/running-solr-in-debug-through-eclipse-td4159777.html

to run solr server in debug mode through eclipse.

Should give you some hint.

Let me go through your error again to see , if I get some clue there.

-Atita

On Fri, Sep 22, 2017 at 11:41 AM, srini sampath  wrote:

> Thanks Aman,
> Erick, I followed the link and I am getting the following error,
>
> Buildfile: ${user.home}\git\lucene-solr\build.xml
>
> compile:
>
> -check-git-state:
>
> -git-cleanroot:
>
> -copy-git-state:
>
> git-autoclean:
>
> resolve:
>
> ivy-availability-check:
>
> BUILD FAILED
> ${user.home}\git\lucene-solr\build.xml:309: The following error 
> occurred while executing this line:
> ${user.home}\git\lucene-solr\lucene\build.xml:124: The following error 
> occurred while executing this line:
> ${user.home}\git\lucene-solr\lucene\common-build.xml:424:
> ${user.home}\.ant\lib does not exist.
>
> Total time: 0 seconds
>
> Any Idea?
> How can I run solr server In debug mode.
>
> Here is the thing I am trying to do,
> Change a custom plugin called solrTextTagger 
> and add some extra 
> query parameters to it.
>
> I defined my custom handler in the following way
>
>- class="org.opensextant.solrtexttagger.TaggerRequestHandler">
>
>
>
>- And I defined my custom handler jar file location location in
>solrschema.xml in the following way
>
>   
> (solr-text-tagger.jar
> location)
>
>- I made some changes to the solrTextTagger,
> And built a jar using
>maven.
>- I am running solr as a service. And sending a request using HTTP Post
>method.
>- But the problem is how can I debug solr-text-tagger.jar code to check
>and make changes. (I mean how to do remote debugging?)
>
>
> I am using eclipse IDE for development.
> I found similar problem here
>  in-Solr-Request-Handler-plugin-and-its-debugging-td4077533.html>.
> But I could not understand the solution.
>
> .Best,
> Srini Sampth.
>
>
>
>
>
> On Thu, Sep 21, 2017 at 8:51 PM, Erick Erickson 
> 
> wrote:
>
> > And did you follow the link provided on that page?
> >
> > Best,
> > Erick
> >
> > On Thu, Sep 21, 2017 at 3:07 AM, Aman Tandon 
> > 
> > wrote:
> > > Hi Srini,
> > >
> > > Kindly refer to the READ.ME section of this link of GitHub, this
> should
> > > work.
> > > https://github.com/apache/lucene-solr/blob/master/README.md
> > >
> > > With regards,
> > > Aman Tandon
> > >
> > >
> > > On Sep 21, 2017 1:53 PM, "srini sampath" 
> > > 
> > > wrote:
> > >
> > >> Hi,
> > >> How to build and compile solr in my locale machine? it seems the 
> > >> https://wiki.apache.org/solr/HowToCompileSolr page became obsolete.
> > >> Thanks in advance
> > >>
> >
>

Re: Solr nodes crashing (OOM) after 6.6 upgrade

[Emir Arnautović]

It does not have to be query load - it can be one heavy query that cause memory 
consumption (heavy faceting, deep paging,…) and after that GC jumps in. Maybe 
you could start with log and see if there are queries that have large QTime,

Emir

> On 22 Sep 2017, at 12:00, shamik  wrote:
> 
> All the tuning and scaling down of memory seemed to be stable for a couple of
> days but then came down due to a huge spike in CPU usage, contributed by G1
> Old Generation GC. I'm really puzzled why the instances are suddenly
> behaving like this. It's not that a sudden surge of load contributed to
> this, query and indexing load seemed to be comparable with the previous time
> frame. Just wondering if the hardware itself is not adequate enough for 6.6.
> The instances are all running on 8 CPU / 30gb m3.2xlarge EC2 instances.
> 
> Does anyone ever face issues similar to this?
> 
> 
> 
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr nodes crashing (OOM) after 6.6 upgrade

[shamik]

All the tuning and scaling down of memory seemed to be stable for a couple of
days but then came down due to a huge spike in CPU usage, contributed by G1
Old Generation GC. I'm really puzzled why the instances are suddenly
behaving like this. It's not that a sudden surge of load contributed to
this, query and indexing load seemed to be comparable with the previous time
frame. Just wondering if the hardware itself is not adequate enough for 6.6.
The instances are all running on 8 CPU / 30gb m3.2xlarge EC2 instances.

Does anyone ever face issues similar to this?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Sorting by distance resources with WKT polygon data

[Ere Maijala]

Hi,

our strategy is to have a separate center coordinate field that we use 
for sorting. This has the additional benefit that it's possible to have 
the indexed center point differ from the polygon's centroid, which can 
be useful e.g. with cities, where the city center can be quite a bit 
offset from the centroid.


--Ere

Grondin Luc kirjoitti 13.9.2017 klo 0.07:

Hello,

I am having difficulties with sorting by distance resources indexed with WKT 
geolocation data. I have tried different field configurations and query 
parameters and I did not get working results.

I am using SOLR 6.6 and JTS-core 1.14. My test sample includes resources with point coordinates 
plus one associated with a polygon. I tried using both fieldtypes 
"solr.SpatialRecursivePrefixTreeFieldType" and 
"solr.RptWithGeometrySpatialField". In both cases, I get good results if I do not care 
about sorting. The problem arises when I include sorting.

With SpatialRecursivePrefixTreeFieldType:

The best request I used, based on the documentation I could find, was:
select?fl=*,score={!geofilt%20sfield=PositionGeo%20pt=45.52,-73.53%20d=10%20score=distance}=score%20asc

The distance appears to be correctly evaluated for resources indexed with point 
coordinates. However, it is wrong for the resource with a polygon


   2.3913236
   4.3242383
   4.671504
   4.806902
   20015.115


(Please note that I have verified the polygon externally and it is correct)

With solr.RptWithGeometrySpatialField:

I get an exception triggered by the presence of « score=distance » in the 
request « 
q={!geofilt%20sfield=PositionGeo%20pt=45.52,-73.53%20d=10%20score=distance} »

java.lang.UnsupportedOperationException
 at 
org.apache.lucene.spatial.composite.CompositeSpatialStrategy.makeDistanceValueSource(CompositeSpatialStrategy.java:92)
 at 
org.apache.solr.schema.AbstractSpatialFieldType.getValueSourceFromSpatialArgs(AbstractSpatialFieldType.java:412)
 at 
org.apache.solr.schema.AbstractSpatialFieldType.getQueryFromSpatialArgs(AbstractSpatialFieldType.java:359)
 at 
org.apache.solr.schema.AbstractSpatialFieldType.createSpatialQuery(AbstractSpatialFieldType.java:308)
 at 
org.apache.solr.search.SpatialFilterQParser.parse(SpatialFilterQParser.java:80)

 From there, I am rather stuck with no ideas on how to resolve these problems. 
So advises in that regards would be much appreciated. I can provide more 
details if necessary.

Thank you in advance,


  ---
   Luc Grondin
   Analyste en gestion de l'information numérique
   Centre d'expertise numérique pour la recherche - Université de Montréal
   téléphone: 514-343-6111 p. 3988  --  
luc.gron...@umontreal.ca




--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

Re: How to build solr

[Atita Arora]

http://www.gingercart.com/Home/search-and-crawl/build-and-run-solr-from-source

and  follow thread

http://lucene.472066.n3.nabble.com/running-solr-in-debug-through-eclipse-td4159777.html

to run solr server in debug mode through eclipse.

Should give you some hint.

Let me go through your error again to see , if I get some clue there.

-Atita

On Fri, Sep 22, 2017 at 11:41 AM, srini sampath  wrote:

> Thanks Aman,
> Erick, I followed the link and I am getting the following error,
>
> Buildfile: ${user.home}\git\lucene-solr\build.xml
>
> compile:
>
> -check-git-state:
>
> -git-cleanroot:
>
> -copy-git-state:
>
> git-autoclean:
>
> resolve:
>
> ivy-availability-check:
>
> BUILD FAILED
> ${user.home}\git\lucene-solr\build.xml:309: The following error occurred
> while executing this line:
> ${user.home}\git\lucene-solr\lucene\build.xml:124: The following error
> occurred while executing this line:
> ${user.home}\git\lucene-solr\lucene\common-build.xml:424:
> ${user.home}\.ant\lib does not exist.
>
> Total time: 0 seconds
>
> Any Idea?
> How can I run solr server In debug mode.
>
> Here is the thing I am trying to do,
> Change a custom plugin called solrTextTagger
> and add some extra query
> parameters to it.
>
> I defined my custom handler in the following way
>
>- class="org.opensextant.solrtexttagger.TaggerRequestHandler">
>
>
>
>- And I defined my custom handler jar file location location in
>solrschema.xml in the following way
>
>   
> (solr-text-tagger.jar
> location)
>
>- I made some changes to the solrTextTagger,
> And built a jar using
>maven.
>- I am running solr as a service. And sending a request using HTTP Post
>method.
>- But the problem is how can I debug solr-text-tagger.jar code to check
>and make changes. (I mean how to do remote debugging?)
>
>
> I am using eclipse IDE for development.
> I found similar problem here
>  in-Solr-Request-Handler-plugin-and-its-debugging-td4077533.html>.
> But I could not understand the solution.
>
> .Best,
> Srini Sampth.
>
>
>
>
>
> On Thu, Sep 21, 2017 at 8:51 PM, Erick Erickson 
> wrote:
>
> > And did you follow the link provided on that page?
> >
> > Best,
> > Erick
> >
> > On Thu, Sep 21, 2017 at 3:07 AM, Aman Tandon 
> > wrote:
> > > Hi Srini,
> > >
> > > Kindly refer to the READ.ME section of this link of GitHub, this
> should
> > > work.
> > > https://github.com/apache/lucene-solr/blob/master/README.md
> > >
> > > With regards,
> > > Aman Tandon
> > >
> > >
> > > On Sep 21, 2017 1:53 PM, "srini sampath" 
> > > wrote:
> > >
> > >> Hi,
> > >> How to build and compile solr in my locale machine? it seems the
> > >> https://wiki.apache.org/solr/HowToCompileSolr page became obsolete.
> > >> Thanks in advance
> > >>
> >
>

Re: AEM SOLR integaration

[Atita Arora]

https://www.slideshare.net/DEEPAKKHETAWAT/basics-of-solr-and-solr-integration-with-aem6-61150010

This could probably help too along with the link Nicole shared.

On Fri, Sep 22, 2017 at 12:28 PM, Nicole Bilić 
wrote:

> Hi,
>
> Maybe this could help you out http://www.aemsolrsearch.com/
>
> Regards,
> Nicole
>
> On Sep 22, 2017 05:41, "Gunalan V"  wrote:
>
> > Hello,
> >
> > I'm looking for suggestion in building the SOLR infrastructure so Kindly
> > let me know if anyone has integerated AEM (Adobe Experience Manager) with
> > SOLR?
> >
> >
> >
> > Thanks,
> > GVK
> >
>

Re: Replicates not recovering after rolling restart

[Rick Leir]

Wunder, Erick

$ dc
16o
1578578283947098112p
15E83C95E8D0

That is an interesting number. Is it, as a guess, machine instructions 
or an address pointer? It does not look like UTF-8 or ASCII. Machine 
code looks promising:



Disassembly:

0:  15 e8 3c 95 e8  adceax,0xe8953ce8
5:  d0 00   rolBYTE PTR [rax],1


/ADC/dest,src Modifies flags: AF CF OF SF PF ZF Sums two binary operands 
placing the result in the destination.


*ROL - Rotate Left*

Registers: the/64-bit/extension of/eax/is called/rax/.

Is that code possibly in the JVM executable? Or a random memory page.

cheers -- Rick

On 2017-09-20 07:21 PM, Walter Underwood wrote:

1578578283947098112 needs 61 bits. Is it being parsed into a 32 bit target?

That doesn’t explain where it came from, of course.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)



On Sep 20, 2017, at 3:35 PM, Erick Erickson  wrote:

The numberformatexception is...odd. Clearly that's too big a number
for an integer, did anything in the underlying schema change?

Best,
Erick

On Wed, Sep 20, 2017 at 3:00 PM, Walter Underwood  wrote:

Rolling restarts work fine for us. I often include installing new configs with 
that. Here is our script. Pass it any hostname in the cluster. I use the load 
balancer name. You’ll need to change the domain and the install directory of 
course.

#!/bin/bash

cluster=$1

hosts=`curl -s 
"http://${cluster}:8983/solr/admin/collections?action=CLUSTERSTATUS=json; | 
jq -r '.cluster.live_nodes[]' | sort`

for host in $hosts
do
host="${host}.cloud.cheggnet.com"
echo restarting Solr on $host
ssh $host 'cd /apps/solr6 ; sudo -u bin bin/solr stop; sudo -u bin bin/solr 
start -cloud -h `hostname`'
done


Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)



On Sep 20, 2017, at 1:42 PM, Bill Oconnor  wrote:

Hello,


Background:


We have been successfully using Solr for over 5 years and we recently made the 
decision to move into SolrCloud. For the most part that has been easy but we 
have repeated problems with our rolling restart were server remain functional 
but stay in Recovery until they stop trying. We restarted because we increased 
the memory from 12GB to 16GB on the JVM.


Does anyone have any insight as to what is going on here?

Is there a special procedure I should use for starting a stopping host?

Is it ok to do a rolling restart on all the nodes in s shard?


Any insight would be appreciated.


Configuration:


We have a group of servers with multiple collections. Each collection consist 
of one shard and multiple replicates. We are running the latest stable version 
of SolrClound 6.6 on Ubuntu LTS and Oracle Corporation Java HotSpot(TM) 64-Bit 
Server VM 1.8.0_66 25.66-b17


(collection)  (shard)  (replicates)

journals_stage   ->  shard1  ->  solr-220 (leader) , solr-223, solr-221, 
solr-222 (replicates)


Problem:


Restarting the system puts the replicates in a recovery state they never exit 
from. They eventually give up after 500 tries.  If I go to the individual 
replicates and execute a query the data is still available.


Using tcpdump I find the replicates sending this request to the leader (the 
leader appears to be active).


The exchange goes  like this - :


solr-220 is the leader.

Solr-221 to Solr-220


10:18:42.426823 IP solr-221:54341 > solr-220:8983:


POST /solr/journals_stage_shard1_replica1/update HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
User-Agent: 
Solr[org.apache.solr.client.solrj.impl.HttpSolrClient]
 1.0
Content-Length: 108
Host: solr-220:8983
Connection: Keep-Alive


commit_end_point=true=false=true=false=true=javabin=2


Solr-220 back to Solr-221


IP solr-220:8983 > solr-221:54341: Flags [P.], seq 1:5152, ack 385, win 235, 
options [nop,nop,
TS val 85813 ecr 858107069], length 5151
..HTTP/1.1 500 Server Error
Content-Type: application/octet-stream
Content-Length: 5060


.responseHeader..%QTimeC.%error..#msg?.For input string: 
"1578578283947098112".%trace?.: For
input string: "1578578283947098112"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:583)
at java.lang.Integer.parseInt(Integer.java:615)
at 
org.apache.lucene.queries.function.docvalues.IntDocValues.getRangeScorer(IntDocValues.java:89)
at 
org.apache.solr.search.function.ValueSourceRangeFilter$1.iterator(ValueSourceRangeFilter.java:83)
at 
org.apache.solr.search.SolrConstantScoreQuery$ConstantWeight.scorer(SolrConstantScoreQuery.java:100)
at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:126)
at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:400)
at 

Re: AEM SOLR integaration

[Nicole Bilić]

Hi,

Maybe this could help you out http://www.aemsolrsearch.com/

Regards,
Nicole

On Sep 22, 2017 05:41, "Gunalan V"  wrote:

> Hello,
>
> I'm looking for suggestion in building the SOLR infrastructure so Kindly
> let me know if anyone has integerated AEM (Adobe Experience Manager) with
> SOLR?
>
>
>
> Thanks,
> GVK
>

Re: CloudSolrServer set http request timeout

[Vincenzo D'Amore]

Thanks for the suggestion, it's working like a charm. 

Ciao,
Vincenzo


> On 21 Sep 2017, at 19:51, Jason Gerlowski  wrote:
> 
> Hi Vincenzo,
> 
> Have you tried setting the read/socket timeout on your client?
> CloudSolrServer uses a LBHttpSolrServer under the hood, which you can
> get with the getLBServer method
> (https://lucene.apache.org/solr/4_1_0/solr-solrj/org/apache/solr/client/solrj/impl/CloudSolrServer.html#getLbServer()).
> Once you have access to LBHttpSolrServer, you can use the
> "setSoTimeout" method
> (https://lucene.apache.org/solr/4_1_0/solr-solrj/org/apache/solr/client/solrj/impl/LBHttpSolrServer.html#setSoTimeout(int))
> to choose an appropriate maximum timeout.
> 
> At least, that's how the Javadocs make it look in 4.x, and how I know
> it works in more recent versions.  Hope that helps.
> 
> Jason
> 
>> On Thu, Sep 21, 2017 at 1:07 PM, Vincenzo D'Amore  wrote:
>> Hi,
>> 
>> I have a huge problem with few queries in SolrCloud 4.8.1 that hangs the
>> client.
>> 
>> Actually I'm unable to understand even if the cluster really receives the
>> requests.
>> 
>> How can I set a timeout when Solrj client wait too much ?
>> 
>> Best regards,
>> Vincenzo
>> 
>> --
>> Vincenzo D'Amore
>> email: v.dam...@gmail.com
>> skype: free.dev
>> mobile: +39 349 8513251

Re: How to build solr

[srini sampath]

PS: I have Installed both Ant and Ivy in my system. But there is no
${user.home}\.ant\lib
folder

On Fri, Sep 22, 2017 at 11:41 AM, srini sampath  wrote:

> Thanks Aman,
> Erick, I followed the link and I am getting the following error,
>
> Buildfile: ${user.home}\git\lucene-solr\build.xml
>
> compile:
>
> -check-git-state:
>
> -git-cleanroot:
>
> -copy-git-state:
>
> git-autoclean:
>
> resolve:
>
> ivy-availability-check:
>
> BUILD FAILED
> ${user.home}\git\lucene-solr\build.xml:309: The following error occurred
> while executing this line:
> ${user.home}\git\lucene-solr\lucene\build.xml:124: The following error
> occurred while executing this line:
> ${user.home}\git\lucene-solr\lucene\common-build.xml:424:
> ${user.home}\.ant\lib does not exist.
>
> Total time: 0 seconds
>
> Any Idea?
> How can I run solr server In debug mode.
>
> Here is the thing I am trying to do,
> Change a custom plugin called solrTextTagger
> and add some extra query
> parameters to it.
>
> I defined my custom handler in the following way
>
>- 
>
>
>
>- And I defined my custom handler jar file location location in
>solrschema.xml in the following way
>
>
> (solr-text-tagger.jar
> location)
>
>- I made some changes to the solrTextTagger,
> And built a jar using
>maven.
>- I am running solr as a service. And sending a request using HTTP
>Post method.
>- But the problem is how can I debug solr-text-tagger.jar code to
>check and make changes. (I mean how to do remote debugging?)
>
>
> I am using eclipse IDE for development.
> I found similar problem here
> .
> But I could not understand the solution.
>
> .Best,
> Srini Sampth.
>
>
>
>
>
> On Thu, Sep 21, 2017 at 8:51 PM, Erick Erickson 
> wrote:
>
>> And did you follow the link provided on that page?
>>
>> Best,
>> Erick
>>
>> On Thu, Sep 21, 2017 at 3:07 AM, Aman Tandon 
>> wrote:
>> > Hi Srini,
>> >
>> > Kindly refer to the READ.ME section of this link of GitHub, this should
>> > work.
>> > https://github.com/apache/lucene-solr/blob/master/README.md
>> >
>> > With regards,
>> > Aman Tandon
>> >
>> >
>> > On Sep 21, 2017 1:53 PM, "srini sampath" 
>> > wrote:
>> >
>> >> Hi,
>> >> How to build and compile solr in my locale machine? it seems the
>> >> https://wiki.apache.org/solr/HowToCompileSolr page became obsolete.
>> >> Thanks in advance
>> >>
>>
>
>

Re: How to build solr

[srini sampath]

Thanks Aman,
Erick, I followed the link and I am getting the following error,

Buildfile: ${user.home}\git\lucene-solr\build.xml

compile:

-check-git-state:

-git-cleanroot:

-copy-git-state:

git-autoclean:

resolve:

ivy-availability-check:

BUILD FAILED
${user.home}\git\lucene-solr\build.xml:309: The following error occurred
while executing this line:
${user.home}\git\lucene-solr\lucene\build.xml:124: The following error
occurred while executing this line:
${user.home}\git\lucene-solr\lucene\common-build.xml:424:
${user.home}\.ant\lib does not exist.

Total time: 0 seconds

Any Idea?
How can I run solr server In debug mode.

Here is the thing I am trying to do,
Change a custom plugin called solrTextTagger
and add some extra query
parameters to it.

I defined my custom handler in the following way

   - 

   

   - And I defined my custom handler jar file location location in
   solrschema.xml in the following way

  
(solr-text-tagger.jar
location)

   - I made some changes to the solrTextTagger,
    And built a jar using
   maven.
   - I am running solr as a service. And sending a request using HTTP Post
   method.
   - But the problem is how can I debug solr-text-tagger.jar code to check
   and make changes. (I mean how to do remote debugging?)


I am using eclipse IDE for development.
I found similar problem here
.
But I could not understand the solution.

.Best,
Srini Sampth.





On Thu, Sep 21, 2017 at 8:51 PM, Erick Erickson 
wrote:

> And did you follow the link provided on that page?
>
> Best,
> Erick
>
> On Thu, Sep 21, 2017 at 3:07 AM, Aman Tandon 
> wrote:
> > Hi Srini,
> >
> > Kindly refer to the READ.ME section of this link of GitHub, this should
> > work.
> > https://github.com/apache/lucene-solr/blob/master/README.md
> >
> > With regards,
> > Aman Tandon
> >
> >
> > On Sep 21, 2017 1:53 PM, "srini sampath" 
> > wrote:
> >
> >> Hi,
> >> How to build and compile solr in my locale machine? it seems the
> >> https://wiki.apache.org/solr/HowToCompileSolr page became obsolete.
> >> Thanks in advance
> >>
>