solr boost query

[sara hajili]

Hi
I have some docs in my solr index. and now i want to run a query. please
help me how create this query?
i have documents like this:
{
topic_code:"A"
},
{
topic_code:"B"
},
{
topic_code:"A"
},
{
topic_code:"A"
},
{
topic_code:"C"
},
{
topic_code:"A"
},..
now i want to get respanse from a solr 100 dosuments , and i want to have
70 document with topic_code:A and 10 documents with topic_code:B and 20
document with topic_code:C.
so i guess i need SOLR BOOST QUERY.
some thing like this :
q=(topic_code:A)^70 OR (topic_code:B)^20 OR (topic_code:C)^10 & row=100
but this isn't work.
could u help me to create a suitable query and get my best result ?
tnx.

Re: solr nested schema

[sara hajili]

i need to have nested object to solr document , and query in nested objects
and children. i create this schema :
  
 and add this doc :
{
"id":12,
"topic":{"code":12,"name":"hi"}
}

this file added completly but have a probelm in search. i didn't get any
result when is search : "topic.code:12"
i didn't get any result from solr.
what is my problem?
schema? adding doc ? or query ?
On Sat, Jul 14, 2018 at 4:24 PM Andrea Gazzarini 
wrote:

> Hi,
> Please expand a bit your needs, because the answer to your question could
> be different.
>
> Specifically: is that nesting needed only for visualization purposes? Could
> you please expand your access pattern (i.e. queries requirements)?
>
> Even if Solr supports nested documents (just google "Solr nested
> documents") , depending on the answers above, it could be something that
> you don't need.
>
> Best,
> Andrea
>
> Il sab 14 lug 2018, 13:27 sara hajili  ha scritto:
>
> > hi
> > i need add nested document in solr . how can i do this ?
> > how add nested document field in schema ?
> > for example i need to add this :
> > {
> >   "id":12,
> >'topic":[ {
> >code:1,
> > name:"hi"
> > },
> > {
> > code:2,
> >  name:"hi"
> > }
> >   ]
> > }
> >
>

solr nested schema

[sara hajili]

hi
i need add nested document in solr . how can i do this ?
how add nested document field in schema ?
for example i need to add this :
{
  "id":12,
   'topic":[ {
   code:1,
name:"hi"
},
{
code:2,
 name:"hi"
}
  ]
}

solr cloud vs standalone solr

[sara hajili]

hi all,
I want to know when standalone solr can't be sufficient for storing data
and we need to migrate to solr cloud?for example standalone solr take too
much time to return query result or to store document or etc.

in other word ,what is best capacity and data index size in  standalone
solr  that doesn't bad effect on query running and data inserting
performance?and after passing this index size i must switch to solr cloud?

Solr 6 more like this

[sara hajili]

hi i switch from solr 5 to solr 6 .
i create my more like this handler that use solr more like this handler and
expand query by adding some word to query.
and now my question is about mlt parameter .
i wanna to know about mlt.mindf and mlt.mintf.what are these doing exactly?
when i didn't set mlt.mindf and mlt.mintf and theses set to default value.
every thing is ok.and i got answer of my handler queickly.
but when i set both of these 1 .
i got heap error.and when i check my solr with jconsole.i saw that these
new query with mlt.mintf=1 and mlt.mindf=1 nead more than 4G heap while
when i execute my query with default mlt.mintf=2 and mlt.mindf=5 i did not
get heap space error .and query execute with less than 512M heap size.

mintf and mindf how can affect on memory use (heap size) of my solr system?

Re: solr extends query

[sara hajili]

i saw synonym Filter but it is not sufficient for me.as i saw this, i must
build synonym filter map . that this is built like a map i mean i must put
exactly in synonym map for example "home" is synonym of "house" (for
example).
so in query when user insert home it filter add house to it and expand this.
but i don't want this. i have a algorithm that get n words and according to
it and use of wordnet ,it make a decision that what word must be add to
expand this query. it depends on all words exist on query.
so i can not tokenize my query word by word.
because of if i tokenize query word by word .then in synonym filter factory
i have just one token and i can not make a decision what word must be added
to query. i need to have all word together.
and also i can not tokenize my query with "." for example because i wanna
to tokenize my query word by word to normalize every word. so i can not
change tokenize format.
and i can not use synonymfilterfactory.
how i can do this?i got all query words and expand this with my algorithm
based on using wordnet?

On Mon, Jul 25, 2016 at 5:32 AM, Erik Hatcher <erik.hatc...@gmail.com>
wrote:

> You’re going to need to tokenize to look up words, so a TokenFilter is a
> better place to put this sort of thing, I think.
>
> Build off of Lucene’s SynonymFilter (and corresponding
> SynonymFilterFactory)
> https://github.com/apache/lucene-solr/blob/5e5fd662575105de88d8514b426bccdcb4c76948/lucene/analysis/common/src/java/org/apache/lucene/analysis/synonym/SynonymFilter.java
> <
> https://github.com/apache/lucene-solr/blob/5e5fd662575105de88d8514b426bccdcb4c76948/lucene/analysis/common/src/java/org/apache/lucene/analysis/synonym/SynonymFilter.java
> >
>
> I’m not sure exactly what your code is trying to do (please share that as
> well, for best assistance), but I do not recommend putting custom code into
> Solr’s package namespaces (and obviously that has issues here, because of
> the separate JAR and visibility/access).
>
> Erik
>
>
> > On Jul 25, 2016, at 2:25 AM, sara hajili <hajili.s...@gmail.com> wrote:
> >
> > i use solr 6-1-0.
> > and i write my own search Handler .i got this error:
> >
> > java.lang.IllegalAccessError: tried to access field
> > org.apache.solr.handler.component.ResponseBuilder.requestInfo from
> > class org.apache.solr.handler.component.MyResponseBuilder
> >   at
> org.apache.solr.handler.component.MyResponseBuilder.getRequestInfo(MyResponseBuilder.java:19)
> >   at
> org.apache.solr.handler.component.MySearchHandler.handleRequestBody(MySearchHandler.java:94)
> >   at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> >   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
> >   at
> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:657)
> >   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
> >   at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
> >   at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
> >   at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
> >   at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> >   at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> >   at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> >   at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> >   at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
> >   at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> >   at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> >   at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
> >   at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> >   at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
> >   at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> >   at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> >   at org.eclipse.jetty.server.Server.handle(Server.java:518)
> >   at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> >   at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> >   at
> org.eclipse.jetty.i

Re: solr extends query

[sara hajili]

er
> implementations.  The synonym token filter is the closest to what you want.
>
> But maybe first - please detail (copy/paste) the exact error you were
> getting in your handler.  It's still not my recommended approach but it
> sounds like you're new to Solr development and need to iron out some issues
> first.
>
>Erik
>
> > On Jul 24, 2016, at 13:20, sara hajili <hajili.s...@gmail.com> wrote:
> >
> > How i can implement this as q token filter.?
> > I wanna to expand query with wordnet.
> > So i try to implement my search handler. In this way i got solrparam. Q
> > And send it to wordnet and expand this.
> > I write this to mysearch handler. So when user insert a query like q=tree
> > and apple.
> > I expand this query with orange.
> >> On Jul 24, 2016 8:37 PM, "Erik Hatcher" <erik.hatc...@gmail.com> wrote:
> >>
> >> My first inclination would be to implement this as a TokenFilter
> instead.
> >> Can you provide some examples of what you want?
> >>
> >>> On Jul 24, 2016, at 08:22, sara hajili <hajili.s...@gmail.com> wrote:
> >>>
> >>> hi all.
> >>> i wanna to expand my queries with wordnet ...
> >>> so i try to write my own SearchHandler class."/mysearchHandler"
> >>> so i add new requestHandler in solr-config  /mySearchHandler
> >>> but i faced with problem. i found some field that they were package
> >> access .
> >>> so i get access error in run query.(this field does n't have any getter
> >> to
> >>> use them)
> >>> and now my question is what is best approach to expand my queris with
> >>> wordnet?
> >>> is good to write my own searchHandler?and overwrite it to get my
> result?
> >>> tnx
> >>
>

Re: solr extends query

[sara hajili]

How i can implement this as q token filter.?
I wanna to expand query with wordnet.
So i try to implement my search handler. In this way i got solrparam. Q
And send it to wordnet and expand this.
I write this to mysearch handler. So when user insert a query like q=tree
and apple.
I expand this query with orange.
On Jul 24, 2016 8:37 PM, "Erik Hatcher" <erik.hatc...@gmail.com> wrote:

> My first inclination would be to implement this as a TokenFilter instead.
>  Can you provide some examples of what you want?
>
> > On Jul 24, 2016, at 08:22, sara hajili <hajili.s...@gmail.com> wrote:
> >
> > hi all.
> > i wanna to expand my queries with wordnet ...
> > so i try to write my own SearchHandler class."/mysearchHandler"
> > so i add new requestHandler in solr-config  /mySearchHandler
> > but i faced with problem. i found some field that they were package
> access .
> > so i get access error in run query.(this field does n't have any getter
> to
> > use them)
> > and now my question is what is best approach to expand my queris with
> > wordnet?
> > is good to write my own searchHandler?and overwrite it to get my result?
> > tnx
>

solr extends query

[sara hajili]

hi all.
i wanna to expand my queries with wordnet ...
so i try to write my own SearchHandler class."/mysearchHandler"
so i add new requestHandler in solr-config  /mySearchHandler
but i faced with problem. i found some field that they were package access .
so i get access error in run query.(this field does n't have any getter to
use them)
and now my question is what is best approach to expand my queris with
wordnet?
is good to write my own searchHandler?and overwrite it to get my result?
tnx

Re: solr.textfield in solr 6

[sara hajili]

yes it occur because of exsitance text field in my solr-config.xml file
.and text field dos not exist in my schema file
tnx

On Wed, Jul 13, 2016 at 6:04 PM, Erick Erickson <erickerick...@gmail.com>
wrote:

> You have to show us exactly what you're doing when you
> add and remove "solr.TextField".
>
> This error _usually_ means that your solrconfig.xml
> file for some request handler has
> text
> and thus when you send a query in without a field
> qualifier it is searched for in a "text" field. I suspect
> that what you're _really_ doing is somehow removing
> the  somehow.
>
> Best,
> Erick
>
> On Wed, Jul 13, 2016 at 7:00 AM, sara hajili <hajili.s...@gmail.com>
> wrote:
> > hi i switched from solr 5.3 to solr 6.1.0.
> > in my schema file i have a solr.textField
> > now when i start solr and add core . i get this exception
> >
> > org.apache.solr.common.SolrException: undefined field text
> > at
> org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1300)
> > at
> org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:452)
> > at
> org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74)
> > at
> org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:169)
> > at
> org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:206)
> > at
> org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:372)
> > at
> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742)
> > at
> org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:530)
> > at org.apache.solr.parser.QueryParser.Term(QueryParser.java:315)
> > at
> org.apache.solr.parser.QueryParser.Clause(QueryParser.java:186)
> > at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107)
> > at
> org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96)
> > at
> org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:153)
> > at
> org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50)
> > at org.apache.solr.search.QParser.getQuery(QParser.java:141)
> > at
> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:162)
> > at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:267)
> > at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
> > at
> org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:68)
> > at
> org.apache.solr.core.SolrCore.lambda$getSearcher$4(SolrCore.java:1810)
> > at
> org.apache.solr.core.SolrCore$$Lambda$24/1861899333.call(Unknown Source)
> > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$22(ExecutorUtil.java:229)
> > at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$1/728115831.run(Unknown
> > Source)
> > at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> > at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> > at java.lang.Thread.run(Thread.java:745)
> >
> > and when i delete solr.textField type every thing is ok.
> >
> > how i solve it?
> >
> > tnx
>

solr.textfield in solr 6

[sara hajili]

hi i switched from solr 5.3 to solr 6.1.0.
in my schema file i have a solr.textField
now when i start solr and add core . i get this exception

org.apache.solr.common.SolrException: undefined field text
at 
org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1300)
at 
org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:452)
at 
org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74)
at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:169)
at 
org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:206)
at 
org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:372)
at 
org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742)
at 
org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:530)
at org.apache.solr.parser.QueryParser.Term(QueryParser.java:315)
at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:186)
at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107)
at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96)
at 
org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:153)
at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50)
at org.apache.solr.search.QParser.getQuery(QParser.java:141)
at 
org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:162)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:267)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2036)
at 
org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:68)
at 
org.apache.solr.core.SolrCore.lambda$getSearcher$4(SolrCore.java:1810)
at org.apache.solr.core.SolrCore$$Lambda$24/1861899333.call(Unknown 
Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$22(ExecutorUtil.java:229)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$1/728115831.run(Unknown
Source)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

and when i delete solr.textField type every thing is ok.

how i solve it?

tnx

Re: Query optimization

[sara hajili]

as i know when you use docValue=true
solr when indexing doc,
solr although store doc and docValue=true field in memory.to use that in
facet query and sort query result.
so maybe use a lot docvalue=true use a lot  memory of ur system.
but use it in logical way.can make better query response time

On Wed, Jul 13, 2016 at 5:11 AM, Midas A <test.mi...@gmail.com> wrote:

> Is there any draw back of using docValues=true ?
>
> On Wed, Jul 13, 2016 at 2:28 PM, sara hajili <hajili.s...@gmail.com>
> wrote:
>
> > Hi.
> > Facet query take a long time.you vcan use group query.
> > Or in fileds in schema that you run facet query on that filed.
> > Set doc value=true.
> > To get better answer.in quick time.
> > On Jul 13, 2016 11:54 AM, "Midas A" <test.mi...@gmail.com> wrote:
> >
> > > http://
> > >
> > >
> >
> #:8983/solr/prod/select?q=id_path_ids:166=sort_price:[0%20TO%20*]=status:A=company_status:A=true=1=show_meta_id=show_brand=product_amount_available=by_processor=by_system_memory=by_screen_size=by_operating_system=by_laptop_type=by_processor_brand=by_hard_drive_capacity=by_touchscreen=by_warranty=by_graphic_memory=is_trm=show_merchant=is_cod=show_market={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]=500=true=sort_price=0=10=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
> > >
> > >
> > > What kind of optimization we can do in above query . it is taking 2400
> > ms .
> > >
> >
>

Re: Query optimization

[sara hajili]

Hi.
Facet query take a long time.you vcan use group query.
Or in fileds in schema that you run facet query on that filed.
Set doc value=true.
To get better answer.in quick time.
On Jul 13, 2016 11:54 AM, "Midas A"  wrote:

> http://
>
> #:8983/solr/prod/select?q=id_path_ids:166=sort_price:[0%20TO%20*]=status:A=company_status:A=true=1=show_meta_id=show_brand=product_amount_available=by_processor=by_system_memory=by_screen_size=by_operating_system=by_laptop_type=by_processor_brand=by_hard_drive_capacity=by_touchscreen=by_warranty=by_graphic_memory=is_trm=show_merchant=is_cod=show_market={!ex=p_r%20key=product_rating:[4-5]}product_rating:[4%20TO%205]={!ex=p_r%20key=product_rating:[3-5]}product_rating:[3%20TO%205]={!ex=p_r%20key=product_rating:[2-5]}product_rating:[2%20TO%205]={!ex=p_r%20key=product_rating:[1-5]}product_rating:[1%20TO%205]={!ex=m_r%20key=merchant_rating:[4-5]}merchant_rating:[4%20TO%205]={!ex=m_r%20key=merchant_rating:[3-5]}merchant_rating:[3%20TO%205]={!ex=m_r%20key=merchant_rating:[2-5]}merchant_rating:[2%20TO%205]={!ex=m_r%20key=merchant_rating:[1-5]}merchant_rating:[1%20TO%205]=500=true=sort_price=0=10=product_amount_available%20desc,boost_index%20asc,popularity%20desc,is_cod%20desc
>
>
> What kind of optimization we can do in above query . it is taking 2400 ms .
>

Re: solr server heap out

[sara hajili]

Hi .u can monitor heap size by jvm.
Run solr and open jvisualvm in "javahome/bin "
Monitor your heap size that solr use.
You can create heap image and check objects that are in heap.and check why
you get heap size error.
On Jul 13, 2016 9:55 AM, "Midas A"  wrote:

> Hi,
> I frequently getting solr heap out once or twice a day. what could be the
> possible reasons for the same and is there any way to log memory used by
> the query in solr.log .
>
> Thanks ,
> Abhishek Tiwari
>

Solr more like this

[sara hajili]

Hi
I indexed pdf files yo solr.and now I wanna to know is there any way to
uplaod  a pdf file and solr return related pdf in result?
I mean I don't want to index pdf file (the file that I wanna to get pdf
more like this for this pdf).and just upload pdf file and get mlt
result.can I do this??

solr index size issue

[sara hajili]

hi i have a about 500 doc that stored that in solr.
when i added this 500 doc i see solr index size it was  about 300 KB .
but it become bigger more and more ,and now after about 2 hours solr index
size become 3500KB . i did n't add any new doc to solr. but index size
become bigger and bigger.why solr treat in this way?
why solr index size become bigger and bigger without adding  any new doc?

Re: boolean query with score and with out score

[sara hajili]

i have exactly this 2 query :
query = caption:apple  caption:bannana^1.0003  caption:pineapple^1.0023
 and this query
query = caption:apple  caption:bannana  caption:pineapple
and get different result.!

On Fri, Feb 12, 2016 at 11:55 PM, Erik Hatcher <erik.hatc...@gmail.com>
wrote:

> What are the parsed queries from debug=true?  Maybe it's an Or/OR thing?
>
> > On Feb 12, 2016, at 23:47, sara hajili <hajili.s...@gmail.com> wrote:
> >
> > hi i have a Boolean query
> > like this
> > query = caption:apple Or caption:bannana^1.0003  OR
> > caption:pineapple^1.0023
> > and get a result like
> > doc1
> > doc2
> > doc3
> >
> > but this result does not satisfy me at all.
> > because i had a doc that contain some of this term but i did not get
> these
> > docs.
> >
> > but when i change my query to :
> > query = caption:apple Or caption:bannana  OR caption:pineapple
> > i get appropriate result i get all docs that have even one of this terms.
> >
> > why Boolean query with score and with out score has a different manner?!
>

boolean query with score and with out score

[sara hajili]

hi i have a Boolean query
like this
query = caption:apple Or caption:bannana^1.0003  OR
caption:pineapple^1.0023
and get a result like
doc1
doc2
doc3

but this result does not satisfy me at all.
because i had a doc that contain some of this term but i did not get these
docs.

but when i change my query to :
query = caption:apple Or caption:bannana  OR caption:pineapple
 i get appropriate result i get all docs that have even one of this terms.

why Boolean query with score and with out score has a different manner?!

solr mlt with grouping issue

[sara hajili]

hi all.i have a mlt query.
and i wanna to categorise query result based on special field.
so i wanna to use group solr feature.
but solr mlt does not support group .
i used group = true
group.field =filed1,
group.limit=3
but i did n't get group result.
** i test solr group feature with select handler and get appropriate answer
based on filed1 groups.
i use solr 5.3.1
why mlt does not support group?
how i can make solr result for group based on specific field ?

solr performance issue

[sara hajili]

hi all.
i have a problem with my solr performance and usage hardware like a
ram,cup...
i have a lot of document and so indexed file about 1000 doc in solr that
every doc has about 8 field in average.
and each field has about 60 char.
i set my field as a storedfield = "false" except of  1 field. // i read
that this help performance.
i used copy field and dynamic field if it was necessary . // i read that
this help performance.
and now my question is that when i run a lot of query on solr i faced with
a problem solr use more cpu and ram and after that filled ,it use a lot
 swapped storage and then use hard,but doesn't create a system file! solr
fill hard until i forced to restart server to release hard disk.
and now my question is why solr treat in this way? and how i can avoid solr
to use huge cpu space?
any config need?!

Re: solr performance issue

[sara hajili]

sorry i made a mistake i have a bout 1000 K doc.
i mean about 100 doc.

On Mon, Feb 8, 2016 at 1:35 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:

> Hi Sara,
> Not sure if I am reading this right, but I read it as you have 1000 doc
> index and issues? Can you tell us bit more about your setup: number of
> servers, hw, index size, number of shards, queries that you run, do you
> index at the same time...
>
> It seems to me that you are running Solr on server with limited RAM and
> probably small heap. Swapping for sure will slow things down and GC is most
> likely reason for high CPU.
>
> You can use http://sematext.com/spm to collect Solr and host metrics and
> see where the issue is.
>
> Thanks,
> Emir
>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>
>
> On 08.02.2016 10:27, sara hajili wrote:
>
>> hi all.
>> i have a problem with my solr performance and usage hardware like a
>> ram,cup...
>> i have a lot of document and so indexed file about 1000 doc in solr that
>> every doc has about 8 field in average.
>> and each field has about 60 char.
>> i set my field as a storedfield = "false" except of  1 field. // i read
>> that this help performance.
>> i used copy field and dynamic field if it was necessary . // i read that
>> this help performance.
>> and now my question is that when i run a lot of query on solr i faced with
>> a problem solr use more cpu and ram and after that filled ,it use a lot
>>   swapped storage and then use hard,but doesn't create a system file! solr
>> fill hard until i forced to restart server to release hard disk.
>> and now my question is why solr treat in this way? and how i can avoid
>> solr
>> to use huge cpu space?
>> any config need?!
>>
>>
>

Re: solr performance issue

[sara hajili]

On Mon, Feb 8, 2016 at 3:04 AM, sara hajili <hajili.s...@gmail.com> wrote:

> sorry i made a mistake i have a bout 1000 K doc.
> i mean about 100 doc.
>
> On Mon, Feb 8, 2016 at 1:35 AM, Emir Arnautovic <
> emir.arnauto...@sematext.com> wrote:
>
>> Hi Sara,
>> Not sure if I am reading this right, but I read it as you have 1000 doc
>> index and issues? Can you tell us bit more about your setup: number of
>> servers, hw, index size, number of shards, queries that you run, do you
>> index at the same time...
>>
>> It seems to me that you are running Solr on server with limited RAM and
>> probably small heap. Swapping for sure will slow things down and GC is most
>> likely reason for high CPU.
>>
>> You can use http://sematext.com/spm to collect Solr and host metrics and
>> see where the issue is.
>>
>> Thanks,
>> Emir
>>
>> --
>> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
>> Solr & Elasticsearch Support * http://sematext.com/
>>
>>
>>
>> On 08.02.2016 10:27, sara hajili wrote:
>>
>>> hi all.
>>> i have a problem with my solr performance and usage hardware like a
>>> ram,cup...
>>> i have a lot of document and so indexed file about 1000 doc in solr that
>>> every doc has about 8 field in average.
>>> and each field has about 60 char.
>>> i set my field as a storedfield = "false" except of  1 field. // i read
>>> that this help performance.
>>> i used copy field and dynamic field if it was necessary . // i read that
>>> this help performance.
>>> and now my question is that when i run a lot of query on solr i faced
>>> with
>>> a problem solr use more cpu and ram and after that filled ,it use a lot
>>>   swapped storage and then use hard,but doesn't create a system file!
>>> solr
>>> fill hard until i forced to restart server to release hard disk.
>>> and now my question is why solr treat in this way? and how i can avoid
>>> solr
>>> to use huge cpu space?
>>> any config need?!
>>>
>>>
>>
>

solr score threashold

[sara hajili]

hi all,
i wanna to know about solr search relevency scoreing threashold.
can i change it?
i mean immagine when i searching i get this result
doc1 score =8
doc2 score =6.4
doc3 score=6
doc8score=5.5
doc5 score=2
i wana to change solr score threashold .in this way i set threashold for
example >4
and then i didn't get doc5 as result.can i do this?if yes how?
and if not how i can modified search to don't get docs as a result that
these docs have a lot distance from doc with max score?
in other word i wanna to delete this gap between solr results

Solr relevancy scoring issue

[sara hajili]

Hi all .
I have a issue with solr scoring.
How solr scoring treat ?
I mean is it linearly?

Boost query vs function query in edismax query

[sara hajili]

Hi all as I underestood.
Both of them are for affecting on relevence scoring.but u have more
dominate on relevence scoring when using  boosted  query.is it true?
I am willing to understand more about difference between 2.
And know what is best situation for using each other?
Tnx.

Re: collapse filter query

[sara hajili]

Tnx Joel.
I wanted to get distinct result from solr.so I found to approach collapse
filter  and facet.
And more like this doesn't support facet. And as u said solr 5.3 has bug on
collapse filter.
If I wont to immigrate to solr 5.4.
Is any other approach to get distinct value that I can use in solr  5.3.1?
On Jan 12, 2016 1:39 AM, "Joel Bernstein" <joels...@gmail.com> wrote:

> I went to go work on the issue and found it was already fixed 7 weeks ago.
> The bug fix is available in Solr 5.4.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Mon, Jan 11, 2016 at 3:12 PM, Susheel Kumar <susheel2...@gmail.com>
> wrote:
>
> > You can go to https://issues.apache.org/jira/browse/SOLR/ and create
> Jira
> > ticket after signing in.
> >
> > Thanks,
> > Susheel
> >
> > On Mon, Jan 11, 2016 at 2:15 PM, sara hajili <hajili.s...@gmail.com>
> > wrote:
> >
> > > Tnx.How I can create a jira ticket?
> > > On Jan 11, 2016 10:42 PM, "Joel Bernstein" <joels...@gmail.com> wrote:
> > >
> > > > I believe this is a bug. I think the reason this is occurring is that
> > you
> > > > have an index segment with no values at all in the collapse field. If
> > you
> > > > could create a jira ticket for this I will look at resolving the
> issue.
> > > >
> > > > Joel Bernstein
> > > > http://joelsolr.blogspot.com/
> > > >
> > > > On Mon, Jan 11, 2016 at 2:03 PM, sara hajili <hajili.s...@gmail.com>
> > > > wrote:
> > > >
> > > > > I am using solr 5.3.1
> > > > > On Jan 11, 2016 10:30 PM, "Joel Bernstein" <joels...@gmail.com>
> > wrote:
> > > > >
> > > > > > Which version of Solr are you using?
> > > > > >
> > > > > > Joel Bernstein
> > > > > > http://joelsolr.blogspot.com/
> > > > > >
> > > > > > On Mon, Jan 11, 2016 at 6:39 AM, sara hajili <
> > hajili.s...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > hi all
> > > > > > > i have a MLT query and i wanna to use collapse filter query.
> > > > > > > and i wanna to use collapse expand nullPolicy.
> > > > > > > in this way when i used it :
> > > > > > > {!collapse field=original_post_id nullPolicy=expand}
> > > > > > > i got my appropriate result .
> > > > > > > (in solr web UI)
> > > > > > >
> > > > > > > but in regular search handler "/select",when i used
> > > > > > > {!collapse field=original_post_id nullPolicy=expand}
> > > > > > > i got error:
> > > > > > >
> > > > > > > {
> > > > > > >   "responseHeader":{
> > > > > > > "status":500,
> > > > > > > "QTime":2,
> > > > > > > "params":{
> > > > > > >   "q":"*:*",
> > > > > > >   "indent":"true",
> > > > > > >   "fq":"{!collapse field=original_post_id
> > nullPolicy=expand}",
> > > > > > >   "wt":"json"}},
> > > > > > >   "error":{
> > > > > > > "trace":"java.lang.NullPointerException\n\tat
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:763)\n\tat
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:211)\n\tat
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1678)\n\tat
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocLis

Issue with stemming and lemmatizing

[sara hajili]

I wanna to write my own text tokenizer.
And my question is about what solr treat with stemming  or lemmatizing?
Solr store both lemmatizerd token and orginal token together?
I mean if in index time solr lemmatize creation to create.
And in query time.user want to search about exactly creation not creat.
How solr do that?!
If I lemmatize query string creation  to create
In this way solr find all create not creatin.
How solr behave with stemmer and lemmatizer?index both original and
lemmatized word?

Re: Classes in solr_home /lib cannot import from solr/dist

[sara hajili]

hi Callum.
you can create a directory for your jar file any where,and u must set jar
file location in  tag in solrConfig.xml
and be carefull that add your lib location at the end of the solr config
default  tag,
because some times your jar need class that at first solr must be load own
class after that load your jar to don't face a class not found exception.


On Thu, Jan 14, 2016 at 4:36 AM, Callum Lamb  wrote:

> I've got an extension jar that contains a class which extends from
>
> org.apache.solr.handler.dataimport.DataSource
>
> But it only works if it's within the solr/dist folder. However when stored
> in the lib/ folder within Solr home. When it tries to load the class it
> cannot find it's parent:
>
> Exception in thread "Thread-69" java.lang.NoClassDefFoundError:
> org/apache/solr/handler/dataimport/DataSource
> at
>
> org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance(DataImporter.java:374)
> at
>
> org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextImpl.java:102)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.solr.handler.dataimport.DataSource
>
> The classes in the lib folder don't have access to the class within the
> dist folder in their classpath when they are loaded.
>
> I'd like the keep my solr install separate from my configs/plugins/indexes
> so I want to avoid putting it into the dist folder unless I absolutely have
> to.
>
> Is this by design? Is there some kind of configuration somewhere I can
> tweak to get this to work?
>
> Cheers,
>
> Callum L.
>
> --
>
> Mintel Group Ltd | 11 Pilgrim Street | London | EC4V 6RN
> Registered in England: Number 1475918. | VAT Number: GB 232 9342 72
>
> Contact details for our other offices can be found at
> http://www.mintel.com/office-locations.
>
> This email and any attachments may include content that is confidential,
> privileged
> or otherwise protected under applicable law. Unauthorised disclosure,
> copying, distribution
> or use of the contents is prohibited and may be unlawful. If you have
> received this email in error,
> including without appropriate authorisation, then please reply to the
> sender about the error
> and delete this email and any attachments.
>
>

solr BooleanClauses issue with space

[sara hajili]

hi all,
what is exactly diffrence between sapce and OR in solr query  ?
i mean what is diffrence  between
q = solr OR lucene OR search
and this
q = solr lucene search?

solr default boolean occurence is OR,isn't it?

Re: solr BooleanClauses issue with space

[sara hajili]

tnx.
and my main question is about maxBooleanDefault in solr config.
it is 1024 by default.
and i have a edismax query with about 500 words in this way:
q1= str1 OR str2 OR str3 ...OR strn
it throws exception that cant't parse query too boolean clause.
so if i changed maxBooleanDefault to 1500 it works
but some thing is ambiguous for me is when i don;t change maxBooleanDefault
and it remains 1024.
but i changed query in this way
q2 = str1 str2 str3 ... strn // i eliminated OR between for and inside space
i didn't get exception !!!
why?!
what is diffrence between q1 an q2??

On Wed, Jan 13, 2016 at 6:28 AM, Shawn Heisey <apa...@elyograg.org> wrote:

> On 1/13/2016 5:40 AM, sara hajili wrote:
> > what is exactly diffrence between sapce and OR in solr query  ?
> > i mean what is diffrence  between
> > q = solr OR lucene OR search
> > and this
> > q = solr lucene search?
> >
> > solr default boolean occurence is OR,isn't it?
>
> This depends on what the default operator is.  The default for the
> default operator is OR, and that would produce exactly the same results
> for both of the queries you have mentioned.  If the default operator is
> AND, then those two queries would be different.
>
> The default operator applies to the lucene and edismax parsers.  The
> lucene parser is Solr's default.  In older versions, the default
> operator could be set by a defaultOperator parameter.  I do not remember
> whether that was in solrconfig or schema.  That parameter is deprecated
> and the q.op parameter should be used now.
>
> Thanks,
> Shawn
>
>

collapse filter query

[sara hajili]

hi all
i have a MLT query and i wanna to use collapse filter query.
and i wanna to use collapse expand nullPolicy.
in this way when i used it :
{!collapse field=original_post_id nullPolicy=expand}
i got my appropriate result .
(in solr web UI)

but in regular search handler "/select",when i used
{!collapse field=original_post_id nullPolicy=expand}
i got error:

{
  "responseHeader":{
"status":500,
"QTime":2,
"params":{
  "q":"*:*",
  "indent":"true",
  "fq":"{!collapse field=original_post_id nullPolicy=expand}",
  "wt":"json"}},
  "error":{
"trace":"java.lang.NullPointerException\n\tat
org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:763)\n\tat
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:211)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1678)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1497)\n\tat
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:555)\n\tat
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:522)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)\n\tat
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat
java.lang.Thread.run(Thread.java:745)\n",
"code":500}}


but if i change nullpolicy to ignore or collapse

no error happened.


and i wondered when i used pysolr to create  a more like this query

i get above error again.

so it seems nullpolicy=expand work me just in more like this query in
solr web UI

and my question is how to solve my problem to use nullpolicy=expand in pysolr?

tnx

Re: collapse filter query

[sara hajili]

I am using solr 5.3.1
On Jan 11, 2016 10:30 PM, "Joel Bernstein" <joels...@gmail.com> wrote:

> Which version of Solr are you using?
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Mon, Jan 11, 2016 at 6:39 AM, sara hajili <hajili.s...@gmail.com>
> wrote:
>
> > hi all
> > i have a MLT query and i wanna to use collapse filter query.
> > and i wanna to use collapse expand nullPolicy.
> > in this way when i used it :
> > {!collapse field=original_post_id nullPolicy=expand}
> > i got my appropriate result .
> > (in solr web UI)
> >
> > but in regular search handler "/select",when i used
> > {!collapse field=original_post_id nullPolicy=expand}
> > i got error:
> >
> > {
> >   "responseHeader":{
> > "status":500,
> > "QTime":2,
> > "params":{
> >   "q":"*:*",
> >   "indent":"true",
> >   "fq":"{!collapse field=original_post_id nullPolicy=expand}",
> >   "wt":"json"}},
> >   "error":{
> > "trace":"java.lang.NullPointerException\n\tat
> >
> >
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:763)\n\tat
> >
> >
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:211)\n\tat
> >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1678)\n\tat
> >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1497)\n\tat
> >
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:555)\n\tat
> >
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:522)\n\tat
> >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)\n\tat
> >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)\n\tat
> > org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)\n\tat
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)\n\tat
> >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)\n\tat
> >
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
> >
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
> >
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
> >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)\n\tat
> >
> >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat
> >
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\n\tat
> >
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat
> > org.eclipse.jetty.server.Server.handle(Server.java:499)\n\tat
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)\n\tat
> >
> >
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)\n\tat
> >
> >
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat
> >
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)\n\tat
> >
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)\n\tat
> > java.lang.Thread.run(Thread.java:745)\n",
> > "code":500}}
> >
> >
> > but if i change nullpolicy to ignore or collapse
> >
> > no error happened.
> >
> >
> > and i wondered when i used pysolr to create  a more like this query
> >
> > i get above error again.
> >
> > so it seems nullpolicy=expand work me just in more like this query in
> > solr web UI
> >
> > and my question is how to solve my problem to use nullpolicy=expand in
> > pysolr?
> >
> > tnx
> >
>

Re: collapse filter query

[sara hajili]

Tnx.How I can create a jira ticket?
On Jan 11, 2016 10:42 PM, "Joel Bernstein" <joels...@gmail.com> wrote:

> I believe this is a bug. I think the reason this is occurring is that you
> have an index segment with no values at all in the collapse field. If you
> could create a jira ticket for this I will look at resolving the issue.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
> On Mon, Jan 11, 2016 at 2:03 PM, sara hajili <hajili.s...@gmail.com>
> wrote:
>
> > I am using solr 5.3.1
> > On Jan 11, 2016 10:30 PM, "Joel Bernstein" <joels...@gmail.com> wrote:
> >
> > > Which version of Solr are you using?
> > >
> > > Joel Bernstein
> > > http://joelsolr.blogspot.com/
> > >
> > > On Mon, Jan 11, 2016 at 6:39 AM, sara hajili <hajili.s...@gmail.com>
> > > wrote:
> > >
> > > > hi all
> > > > i have a MLT query and i wanna to use collapse filter query.
> > > > and i wanna to use collapse expand nullPolicy.
> > > > in this way when i used it :
> > > > {!collapse field=original_post_id nullPolicy=expand}
> > > > i got my appropriate result .
> > > > (in solr web UI)
> > > >
> > > > but in regular search handler "/select",when i used
> > > > {!collapse field=original_post_id nullPolicy=expand}
> > > > i got error:
> > > >
> > > > {
> > > >   "responseHeader":{
> > > > "status":500,
> > > > "QTime":2,
> > > > "params":{
> > > >   "q":"*:*",
> > > >   "indent":"true",
> > > >   "fq":"{!collapse field=original_post_id nullPolicy=expand}",
> > > >   "wt":"json"}},
> > > >   "error":{
> > > > "trace":"java.lang.NullPointerException\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.search.CollapsingQParserPlugin$IntScoreCollector.finish(CollapsingQParserPlugin.java:763)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:211)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1678)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1497)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:555)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:522)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:277)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)\n\tat
> > > > org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)\n\tat
> > > >
> > org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)\n\tat
> > > >
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)\n\tat
> > > >
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)\n\tat
> > > >
> > > >
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)\n\tat
> > > >
> > > >
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)\n\tat
> > > >
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
> > > >
> > > >
> > >
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat
> > > >
> > > >
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat
> > > >
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler

problem with solr plugin

[sara hajili]

hi i wanna to have own normalization .
i write 2 class one class form normalization filter factort that extends
token filter factory and imoplement multiTermAwarecomponent
and an other one class is normalization factory that extends token filter.
then i create a jar from this classes with dependencies .
and i add this jar file to solr_home/dist and
solr_home/contrib/extraction/lib
and i use this class in schema in this way:


 

   

but i get this error when i added a core to solr:

org.apache.solr.common.SolrException: Could not load conf for core
post: Plugin init failure for [schema.xml] fieldType
"sample_normalizer": Plugin init failure for [schema.xml]
analyzer/filter: Error loading class
'com.ponila.set.textanalyzer.PersianCustomNormalizerFilterFactory'.
Schema file is D:\solr-5.3.1\example\poinila\solr\post\conf\schema.xml
at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:80)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:721)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:697)
at 
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:629)
at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:214)
at 
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:194)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at 
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:675)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:214)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.solr.common.SolrException: Plugin init failure
for [schema.xml] fieldType "sample_normalizer": Plugin init failure
for [schema.xml] analyzer/filter: Error loading class
'com.ponila.set.textanalyzer.PersianCustomNormalizerFilterFactory'.
Schema file is D:\solr-5.3.1\example\poinila\solr\post\conf\schema.xml
at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:596)
at org.apache.solr.schema.IndexSchema.(IndexSchema.java:175)
at 
org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55)
at 
org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69)
at 
org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:104)
at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:75)
... 30 more
Caused by: org.apache.solr.common.SolrException: Plugin init failure
for [schema.xml] fieldType "sample_normalizer": Plugin init failure
for [schema.xml] analyzer/filter: Error loading class
'com.ponila.set.textanalyzer.PersianCustomNormalizerFilterFactory'
at 
org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:178)
at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:489)
... 35 more
Caused by: org.apache.solr.common.SolrException: 

add text analyzer to solr

[sara hajili]

hi.
i wanna to change solr analyzer , like normalization.
because solr default normalization for persian language does't satisfy me.
so i start reading solr plugins .
and i try to implement my PersianNormalization.
now i have 2 class in this way :
class persianNormalizer extends TokenFilter.
and another class:
class persianNormalizerFilterFactory extends TokenFilterFactory.
and then i create a jar file from in this 2 class.
and then i add this jar file in solr_home/dist and
solr_home/contrib/extraction
(i select this 2 folder because in solrConf.xml in tag lib this to Dir was
set as a lib dir)
and then in schema.xml i create a one textfield  that use
PersianAnlyzerFilterFactory
in this way :

but when solr comes up,i get error :
how i solve my problem.can any one tell me how to use my normalizer in solr?
or can any one tell me about existing tutorial that determine how write
text analyzer and use that in solr step by step?
the error that i faced with it in solr when adding core was :
and core didn't add to solr.
plz help me.
org.apache.solr.common.SolrException: Error CREATEing SolrCore 'post':
Unable to create core [post] Caused by:
solr.PersianCustomNormalizerFilterFactory

Re: integrate solr with preprocessor tools

[sara hajili]

hi Emir,tnx for answering
now my question is how i write this class?
i must use solr interfaces?
i see in above link that i can use solr analyzer.but how i use that?
plz say me how i start to write my own analyzer step by step...
which interface i can use and change to achieve my goal?
tnx

On Wed, Dec 9, 2015 at 1:50 AM, Emir Arnautovic <
emir.arnauto...@sematext.com> wrote:

> Hi Sara,
> You need to wrap your code in tokenizer or token filter
> https://wiki.apache.org/solr/SolrPlugins
>
> If you want to improve existing and believe others can benefit from
> improvement, you can open ticket and submit patch.
>
> Thanks,
> Emir
>
>
> On 09.12.2015 10:41, sara hajili wrote:
>
>> hi i wanna to use solr , and language of my documents that i stored in
>> solr
>> is persian.
>> solr doesn't support persian as well as i want.so i find preprocessor
>> tools
>> like a normalization,tockenizer and etc ...
>> i don't want to use solr persian filter like persian tockenizer,i mean i
>> wanna to improve it.
>>
>> now my question is how i can integrate solr with this external
>> preprocessor
>> tools??
>>
>> tnx
>>
>>
> --
> Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> Solr & Elasticsearch Support * http://sematext.com/
>
>

integrate solr with preprocessor tools

[sara hajili]

hi i wanna to use solr , and language of my documents that i stored in solr
is persian.
solr doesn't support persian as well as i want.so i find preprocessor tools
like a normalization,tockenizer and etc ...
i don't want to use solr persian filter like persian tockenizer,i mean i
wanna to improve it.

now my question is how i can integrate solr with this external preprocessor
tools??

tnx

Re: solr-8983-console.log is huge

[sara hajili]

You can change solr loglevel.bydefault solr logs for every thing.
You can change this by go in solrconsole.inlog/level and edit levels for
just error for example.
And this is temporary way.
You can also change solrconfig.insolr_home
In /log and change logging4j
Config.
For more info look at:
https://cwiki.apache.org/confluence/display/solr/Configuring+Logging
That log file is constantly growing. And it is now ~60GB. what can i change
to fix this?



--
View this message in context:
http://lucene.472066.n3.nabble.com/solr-8983-console-log-is-huge-tp4238613.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: problem with solr auto add core after restart

[sara hajili]

no error occurred,at least as i see in solr log.solr nothing writes in
SOLR_HOME/LOGS.
what can i do now?


On Thu, Oct 29, 2015 at 1:49 PM, Erick Erickson <erickerick...@gmail.com>
wrote:

> What errors, if any, do you see in the Solr logs? The information here
> isn't
> enough to say much.
>
> Best,
> Erick
>
> On Thu, Oct 29, 2015 at 7:44 AM, sara hajili <hajili.s...@gmail.com>
> wrote:
> > hi,
> > i add this in solr.xml file :
> > ${coreRootDirectory:} 
> > and in each core i added these to core.properties
> > loadOnStartup=true
> >
> > and now if i stop and start  solr from solr_home/bin
> > after restart solr start and automatically added cores to solr.
> > but when i restart solr service in linux as :
> > service solr restart
> > solr start and doesn't add cores to solr automatically.
> > how i can solve this problem to stop start solr service and automatically
> > add  cores to it?
> > plz help me.tnx
>

problem with solr auto add core after restart

[sara hajili]

hi,
i add this in solr.xml file :
${coreRootDirectory:} 
and in each core i added these to core.properties
loadOnStartup=true

and now if i stop and start  solr from solr_home/bin
after restart solr start and automatically added cores to solr.
but when i restart solr service in linux as :
service solr restart
solr start and doesn't add cores to solr automatically.
how i can solve this problem to stop start solr service and automatically
add  cores to it?
plz help me.tnx

solr more like this sort / reranking

[sara hajili]

hi all.
i am new in solr.i use more like this (query handler) for get docs that are
similar to specefic doc that i mentioned.
and my issue is when i get some docs , i wanna to changed sort of that, for
example imagine that docs have one field as like_count,
i wanna to get more related docs to doc id = 4 and then sort them based on
like_count and score(i mean score that i get in result of more like this
for each docs ,that shows how this doc is related to my doc),so i need to
sort based on this 2 field and i set score for them for example sort based
on like_count^8 and score^25.
how can i do this?
i found new feature RE-ranking
http://blog.thedigitalgroup.com/vijaym/2015/06/19/query-re-ranking-in-solr/
in solr.
is it good approach?
if yes how i used it in more like this and pysolr?
i set in more like this parameter but i couldn't get answerd.
in this way:
param={
...

"rq":"{!rerank reRankQuery=$rqq reRankDocs=3 reRankWeight=10}",
> "rqq":"poem"
>
> ...
> }
> and if not.what is best approach to get this result from  more like this
query?
> tnx for any help

solr more like this sort / reranking

[sara hajili]

hi all.
i am new in solr.i use more like this (query handler) for get docs that are
similar to specefic doc that i mentioned.
and my issue is when i get some docs , i wanna to changed sort of that, for
example imagine that docs have one field as like_count,
i wanna to get more related docs to doc id = 4 and then sort them based on
like_count and score(i mean score that i get in result of more like this
for each docs ,that shows how this doc is related to my doc),so i need to
sort based on this 2 field and i set score for them for example sort based
on like_count^8 and score^25.
how can i do this?
i found new feature RE-ranking
http://blog.thedigitalgroup.com/vijaym/2015/06/19/query-re-ranking-in-solr/
in solr.
is it good approach?
if yes how i used it in more like this and pysolr?
i set in more like this parameter but i couldn't get answerd.
in this way:
param={
...

"rq":"{!rerank reRankQuery=$rqq reRankDocs=3 reRankWeight=10}",
"rqq":"poem"

...
}
and if not.what is best approach to get this result from  more like this
query?
tnx for any help

solr get score of each doc in edis max search and more like this search result

[sara hajili]

hi all
i wanna to get each doc score in search result + restrict search result to
some doc that their score are above than score that i need (i mean i set
minimum score in search and get doc based on upper than that score)
i need this in normal search with edismax and more like this in pysolr
i undrestand that i can set debug = true
and from search resulrt i get

print(search_result.debug['explain'])

but this explain more and i couldn't get each doc score.
any help ?!
tnx

solr auggestion with copy-field

[sara hajili]

hi all
i wanna to get suggestion from multi field in solr.
i add this to solrConfig


mySuggester
FuzzyLookupFactory
DocumentDictionaryFactory
suggestStr
like_count
string
false




true
10


suggest



and add this to schema:

and

  
  
 

but i didn't get any result from this suggest query:
http://localhost:8983/solr/post/suggest?suggest=true=mySuggester=json=solr

but when i used one field (not copy field ) i got answer.
how i solve my problem with copy field?

search on multiValued field

[sara hajili]

hi
 in my schema i have a tag field.
this field set multiValued="true".
now my question is about search on multivalued field.
if i add thid doc to sor:
{
id:1,
tag:{"tag1","tag2"}
},

{
id:2,
tag:{"tag1","tag2"}
},

when i search on q="tag1"
i get just :
{
id:1,
tag:{"tag1","tag2"}
},

but if i add docs in this way:
{
id:1,
tag:"tag1",
tag:"tag2"
},
{
id:2,
tag:"tag1",
tag:"tag2"
},
now if i search to q="tag1"
i get both of docs as a result.
i'm confused.!what is difference between 2 docs?
and can i search on this doc ?
{
id:1,
tag:{"tag1","tag2"}
},
and get correct result(correct result is getting 2 doc all together)
tnx.

Re: solr now relational between tags?!!

[sara hajili]

tnx.
as i see more like this.i underestand that if  i set field
"termVectors=true"
and i set this in solrconfig:




now i use pysolr and i add this docs:


{
'id': 'doc_11',
'title': 'these trees are amazing',
'tags':['fruit','peach','apple']
}

now i want to find like more this doc that tags=fruit

so i write this:
similar = solr.more_like_this(q='tags:fruit',mltfl='tags')

but i didn't get any answer.

any help?



On Sun, Sep 6, 2015 at 9:20 AM, Erick Erickson <erickerick...@gmail.com>
wrote:

> The closest thing to this I know of would be "more like this",
> you might want to take a look at that capability:
>
> https://cwiki.apache.org/confluence/display/solr/MoreLikeThis
>
> Best,
> Erick
>
> On Sat, Sep 5, 2015 at 11:29 PM, sara hajili <hajili.s...@gmail.com>
> wrote:
> > hi
> > i have a social app and i wanna to index all people posts.
> > in this case i index post data like :
> > title,images,tags,caption,comment,like counts,etc
> > and i need to search on tag"people tags on their post ,this tags are
> > related to their post"
> > i am willing to undrestand that solr know any relational bettwen tags?!!
> > "i know we can set filter to set synonym of a search clause ,and for
> > example if people search football ,(if in synonym file i set
> > "football"="soccer") solr search on football and soccer.
> > i need to know solr can underestand other relational between tags??
> > i mean solr now about companion relation betwwen tags?!!
> > see this:
> > if  i have tags in solr "football,sport,cr7"
> > and in many  tags in solr doc football and sport come to gether.
> > if user search sport,
> > solr underestand that search about sport and football together("because
> > football and sport come together more and more in docs,so solr must
> > conclude that sport and football have comparison relational,so search on
> > both of them ")??
> > so solr know that?if no how implement this?
> > or if yes.how i use this feature?
> > tnx.
>

Re: frequently update field

[sara hajili]

tnx for comparison of  external file and atomic update

On Sat, Sep 5, 2015 at 6:53 AM, Jack Krupansky <jack.krupan...@gmail.com>
wrote:

> The standard recommendation is to create a proof of concept implementation
> and see how well it performs.
>
> The external file approach is intended for bulk update, such as when the
> pricing for many products will be updated all at once.
>
> Atomic update is recommended for incremental updates.
>
> Atomic update does depend of setting all source fields to stored since the
> entire document must be first read from the stored values before updating
> the selected fields.
>
> If storing all source fields is prohibitive, then they must be stored in an
> external database so that the full documents can be reindexed when updating
> is required.
>
> As with any database question, the first thing you must do is identify your
> access patterns - how much data will you be updating and with what
> frequency.
>
> Generally, atomic update is recommended when only a small fraction of the
> data will be updated in some relatively small interval of time, such as
> hundreds of documents per hour or dozens of documents per minute, or a
> handful per second.
>
>
> -- Jack Krupansky
>
> On Sat, Sep 5, 2015 at 1:16 AM, sara hajili <hajili.s...@gmail.com> wrote:
>
> > hi
> > i am new in solr, i face to a problem and need any solution to solve
> that.
> > i have a field that this field need to update frequently.
> > "image i need to index all post of member of a social app"
> > in this case i need to store and index all posts field like caption ,
> > image, title,comments ,etc
> > but question is about some field like
> > "like_count,repost_count,comment_count" this field frequenly changed and
> i
> > need to update that but other like caption ,title are not as the same of
> > like count field.
> > so what is the best solution to handle this frequntly update..
> > i found that in solr 4 people used external file.
> > but now in solr 5.x i see that atomic update appear.
> > atomic update is substitute of extenal file?and what is best approach in
> > this case?
> > "i really worry about cost of re indexing docs when update like count"
> >
>

solr now relational between tags?!!

[sara hajili]

hi
i have a social app and i wanna to index all people posts.
in this case i index post data like :
title,images,tags,caption,comment,like counts,etc
and i need to search on tag"people tags on their post ,this tags are
related to their post"
i am willing to undrestand that solr know any relational bettwen tags?!!
"i know we can set filter to set synonym of a search clause ,and for
example if people search football ,(if in synonym file i set
"football"="soccer") solr search on football and soccer.
i need to know solr can underestand other relational between tags??
i mean solr now about companion relation betwwen tags?!!
see this:
if  i have tags in solr "football,sport,cr7"
and in many  tags in solr doc football and sport come to gether.
if user search sport,
solr underestand that search about sport and football together("because
football and sport come together more and more in docs,so solr must
conclude that sport and football have comparison relational,so search on
both of them ")??
so solr know that?if no how implement this?
or if yes.how i use this feature?
tnx.

Fwd: atomic update in solr with pysolr(python client)

[sara hajili]

hi
have any idea about implement atomic update with pysolr.?
i found this in solrJ:
HttpSolrClient client = new HttpSolrClient("http://localhost:8983/solr;);

// create the document
SolrInputDocument sdoc = new SolrInputDocument();
sdoc.addField("id","book1");
Map fieldModifier = new HashMap<>(1);
fieldModifier.put("add","Cyberpunk");
sdoc.addField("cat", fieldModifier);  // add the map as the field value

client.add( sdoc );  // send it to the solr server

client.close();  // shutdown client before we exit

how i implement this in pysolr?

conn = pysolr.Solr('http://localhost:8983/solr/post')

docs=[{'id':'post1','title':'this is a test','member_id':{"inc":12}}]

conn.add(docs)
this throw an error in :

member_id':{"inc":12}

any idea to implement atomic update?to increament just member_id field

frequently update field

[sara hajili]

hi
i am new in solr, i face to a problem and need any solution to solve that.
i have a field that this field need to update frequently.
"image i need to index all post of member of a social app"
in this case i need to store and index all posts field like caption ,
image, title,comments ,etc
but question is about some field like
"like_count,repost_count,comment_count" this field frequenly changed and i
need to update that but other like caption ,title are not as the same of
like count field.
so what is the best solution to handle this frequntly update..
i found that in solr 4 people used external file.
but now in solr 5.x i see that atomic update appear.
atomic update is substitute of extenal file?and what is best approach in
this case?
"i really worry about cost of re indexing docs when update like count"

Re: plz help me

[sara hajili]

i'm really confused:|
i'm really anxious about cost of update like count.
and as you said:

you indexed like_count field .and i think it cost alot to update and index
again docs.
because like count change more and more
so isn't better to indede="false" that this field name??!!

On Tue, Sep 1, 2015 at 3:08 AM, Upayavira <u...@odoko.co.uk> wrote:

> you don't need to use a dynamic field, just a normal field will work for
> you. But, you *will* want to index it, and you may benefit from
> docValues, so:
>
>  docValues="true"/>
>
> Upayavira
>
> On Tue, Sep 1, 2015, at 10:59 AM, sara hajili wrote:
> > my solr version is 5.2.1
> > i have a question.
> > if i create 2 core .one for post and one for like . i must index like
> > count?
> > i mean in schema for like core i must write:
> >  > stored="true"/>
> >
> > am i true?
> >
> > On Tue, Sep 1, 2015 at 2:42 AM, Upayavira <u...@odoko.co.uk> wrote:
> >
> > > So you want to be able to sort by the "number of likes" value for a
> > > post?
> > >
> > > What version of Solr are you using? How many posts do you have?
> > >
> > > There's a neat feature in Solr 5.2.1 (I'm pretty sure it is there, not
> > > 5.3) called score joins. Using that you can have two cores, one
> > > containing your posts, and another containing your likes.
> > >
> > > You cannot *sort* on these values, but you can include your likes into
> > > the score, which might even be better.
> > >
> > > If this sounds good, I can dig up some syntax for such a query.
> > >
> > > Upayavira
> > >
> > > On Tue, Sep 1, 2015, at 10:36 AM, sara hajili wrote:
> > > > hi.
> > > > at first i.m sorry for my bad english!
> > > > i have a social app.i want to use solr for searching in this app.
> > > > i have many document (in my case people text that posted on my social
> > > > app).
> > > > and i indexed this.
> > > > but i'm have 1 issue and it is :
> > > >
> > > > i have very doc(post) and they have a property "like" is it  good
> > > > approach
> > > > to index like count (people can like eachother post in my social
> app)?
> > > > likecount change more and more in one day.(so as i know it must be
> set
> > > > dynamic field)
> > > > and if i indexed it ,i think  it costs alot , to update and index
> > > > likecount
> > > > more and more even i use bach update.
> > > > so is it approach to didn't index one field in solr but i could sort
> my
> > > > search result according to that unindexed field?
> > > >
> > > > tnx
> > >
>

plz help me

[sara hajili]

hi.
at first i.m sorry for my bad english!
i have a social app.i want to use solr for searching in this app.
i have many document (in my case people text that posted on my social app).
and i indexed this.
but i'm have 1 issue and it is :

i have very doc(post) and they have a property "like" is it  good approach
to index like count (people can like eachother post in my social app)?
likecount change more and more in one day.(so as i know it must be set
dynamic field)
and if i indexed it ,i think  it costs alot , to update and index likecount
more and more even i use bach update.
so is it approach to didn't index one field in solr but i could sort my
search result according to that unindexed field?

tnx

Re: plz help me

[sara hajili]

my solr version is 5.2.1
i have a question.
if i create 2 core .one for post and one for like . i must index like count?
i mean in schema for like core i must write:


am i true?

On Tue, Sep 1, 2015 at 2:42 AM, Upayavira <u...@odoko.co.uk> wrote:

> So you want to be able to sort by the "number of likes" value for a
> post?
>
> What version of Solr are you using? How many posts do you have?
>
> There's a neat feature in Solr 5.2.1 (I'm pretty sure it is there, not
> 5.3) called score joins. Using that you can have two cores, one
> containing your posts, and another containing your likes.
>
> You cannot *sort* on these values, but you can include your likes into
> the score, which might even be better.
>
> If this sounds good, I can dig up some syntax for such a query.
>
> Upayavira
>
> On Tue, Sep 1, 2015, at 10:36 AM, sara hajili wrote:
> > hi.
> > at first i.m sorry for my bad english!
> > i have a social app.i want to use solr for searching in this app.
> > i have many document (in my case people text that posted on my social
> > app).
> > and i indexed this.
> > but i'm have 1 issue and it is :
> >
> > i have very doc(post) and they have a property "like" is it  good
> > approach
> > to index like count (people can like eachother post in my social app)?
> > likecount change more and more in one day.(so as i know it must be set
> > dynamic field)
> > and if i indexed it ,i think  it costs alot , to update and index
> > likecount
> > more and more even i use bach update.
> > so is it approach to didn't index one field in solr but i could sort my
> > search result according to that unindexed field?
> >
> > tnx
>

Re: plz help me

[sara hajili]

i used pre-built persian analyzer but it isn't enough for me.
this work just for search exacly indexed work!
like search 'go' and search result is 'go'
as i said in english we have stemming,synonym,etc filters that help us to
have flexible search.
and i want some filter for persian.
that pre-built text_fa doesn't satisfied me.have you better perisan filter
than that?or a soulotion to have this filter in persian?
tnx.

On Tue, Sep 1, 2015 at 5:21 AM, Alexandre Rafalovitch <arafa...@gmail.com>
wrote:

> http://blog.griddynamics.com/2015/08/scoring-join-party-in-solr-53.html
> shows how to keep updates in a separate core. Notice that it is an
> intermediate-level article for query syntax.
>
> For persian text analysis, there is a pre-built analyser defiition in
> the techproducts example, start from that. It is in the schema.xml in
> server/solr/configsets/sample_techproducts_configs/conf and is one of
> the example configsets.
>
> Regards,
>Alex.
> 
> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
> http://www.solr-start.com/
>
>
> On 1 September 2015 at 08:07, sara hajili <hajili.s...@gmail.com> wrote:
> > and another question is:
> > my docs are persian and i use text_fa for fieldType but i wanna to have a
> > persian textfield that handle search problem such as stemming.
> > word distance,synonyms etc
> > like english types.
> > as i said i handle "میخواهم " and "خواستن"  and so on.
> > can you suggest me a fieldtype for handle this issues in persian field.
> > tnx
> >
> > On Tue, Sep 1, 2015 at 3:16 AM, sara hajili <hajili.s...@gmail.com>
> wrote:
> >
> >> i'm really confused:|
> >> i'm really anxious about cost of update like count.
> >> and as you said:
> >>  >> docValues="true"/>
> >> you indexed like_count field .and i think it cost alot to update and
> index
> >> again docs.
> >> because like count change more and more
> >> so isn't better to indede="false" that this field name??!!
> >>
> >> On Tue, Sep 1, 2015 at 3:08 AM, Upayavira <u...@odoko.co.uk> wrote:
> >>
> >>> you don't need to use a dynamic field, just a normal field will work
> for
> >>> you. But, you *will* want to index it, and you may benefit from
> >>> docValues, so:
> >>>
> >>>  >>> docValues="true"/>
> >>>
> >>> Upayavira
> >>>
> >>> On Tue, Sep 1, 2015, at 10:59 AM, sara hajili wrote:
> >>> > my solr version is 5.2.1
> >>> > i have a question.
> >>> > if i create 2 core .one for post and one for like . i must index like
> >>> > count?
> >>> > i mean in schema for like core i must write:
> >>> >  >>> > stored="true"/>
> >>> >
> >>> > am i true?
> >>> >
> >>> > On Tue, Sep 1, 2015 at 2:42 AM, Upayavira <u...@odoko.co.uk> wrote:
> >>> >
> >>> > > So you want to be able to sort by the "number of likes" value for a
> >>> > > post?
> >>> > >
> >>> > > What version of Solr are you using? How many posts do you have?
> >>> > >
> >>> > > There's a neat feature in Solr 5.2.1 (I'm pretty sure it is there,
> not
> >>> > > 5.3) called score joins. Using that you can have two cores, one
> >>> > > containing your posts, and another containing your likes.
> >>> > >
> >>> > > You cannot *sort* on these values, but you can include your likes
> into
> >>> > > the score, which might even be better.
> >>> > >
> >>> > > If this sounds good, I can dig up some syntax for such a query.
> >>> > >
> >>> > > Upayavira
> >>> > >
> >>> > > On Tue, Sep 1, 2015, at 10:36 AM, sara hajili wrote:
> >>> > > > hi.
> >>> > > > at first i.m sorry for my bad english!
> >>> > > > i have a social app.i want to use solr for searching in this app.
> >>> > > > i have many document (in my case people text that posted on my
> >>> social
> >>> > > > app).
> >>> > > > and i indexed this.
> >>> > > > but i'm have 1 issue and it is :
> >>> > > >
> >>> > > > i have very doc(post) and they have a property "like" is it  good
> >>> > > > approach
> >>> > > > to index like count (people can like eachother post in my social
> >>> app)?
> >>> > > > likecount change more and more in one day.(so as i know it must
> be
> >>> set
> >>> > > > dynamic field)
> >>> > > > and if i indexed it ,i think  it costs alot , to update and index
> >>> > > > likecount
> >>> > > > more and more even i use bach update.
> >>> > > > so is it approach to didn't index one field in solr but i could
> >>> sort my
> >>> > > > search result according to that unindexed field?
> >>> > > >
> >>> > > > tnx
> >>> > >
> >>>
> >>
> >>
>

Re: plz help me

[sara hajili]

and another question is:
my docs are persian and i use text_fa for fieldType but i wanna to have a
persian textfield that handle search problem such as stemming.
word distance,synonyms etc
like english types.
as i said i handle "میخواهم " and "خواستن"  and so on.
can you suggest me a fieldtype for handle this issues in persian field.
tnx

On Tue, Sep 1, 2015 at 3:16 AM, sara hajili <hajili.s...@gmail.com> wrote:

> i'm really confused:|
> i'm really anxious about cost of update like count.
> and as you said:
>  docValues="true"/>
> you indexed like_count field .and i think it cost alot to update and index
> again docs.
> because like count change more and more
> so isn't better to indede="false" that this field name??!!
>
> On Tue, Sep 1, 2015 at 3:08 AM, Upayavira <u...@odoko.co.uk> wrote:
>
>> you don't need to use a dynamic field, just a normal field will work for
>> you. But, you *will* want to index it, and you may benefit from
>> docValues, so:
>>
>> > docValues="true"/>
>>
>> Upayavira
>>
>> On Tue, Sep 1, 2015, at 10:59 AM, sara hajili wrote:
>> > my solr version is 5.2.1
>> > i have a question.
>> > if i create 2 core .one for post and one for like . i must index like
>> > count?
>> > i mean in schema for like core i must write:
>> > > > stored="true"/>
>> >
>> > am i true?
>> >
>> > On Tue, Sep 1, 2015 at 2:42 AM, Upayavira <u...@odoko.co.uk> wrote:
>> >
>> > > So you want to be able to sort by the "number of likes" value for a
>> > > post?
>> > >
>> > > What version of Solr are you using? How many posts do you have?
>> > >
>> > > There's a neat feature in Solr 5.2.1 (I'm pretty sure it is there, not
>> > > 5.3) called score joins. Using that you can have two cores, one
>> > > containing your posts, and another containing your likes.
>> > >
>> > > You cannot *sort* on these values, but you can include your likes into
>> > > the score, which might even be better.
>> > >
>> > > If this sounds good, I can dig up some syntax for such a query.
>> > >
>> > > Upayavira
>> > >
>> > > On Tue, Sep 1, 2015, at 10:36 AM, sara hajili wrote:
>> > > > hi.
>> > > > at first i.m sorry for my bad english!
>> > > > i have a social app.i want to use solr for searching in this app.
>> > > > i have many document (in my case people text that posted on my
>> social
>> > > > app).
>> > > > and i indexed this.
>> > > > but i'm have 1 issue and it is :
>> > > >
>> > > > i have very doc(post) and they have a property "like" is it  good
>> > > > approach
>> > > > to index like count (people can like eachother post in my social
>> app)?
>> > > > likecount change more and more in one day.(so as i know it must be
>> set
>> > > > dynamic field)
>> > > > and if i indexed it ,i think  it costs alot , to update and index
>> > > > likecount
>> > > > more and more even i use bach update.
>> > > > so is it approach to didn't index one field in solr but i could
>> sort my
>> > > > search result according to that unindexed field?
>> > > >
>> > > > tnx
>> > >
>>
>
>