date:20110901

yay i did it! i wasn't that far away from the correct implementation, it just
was a bit tricky to understand how to...
now i've got a problem with my singleton class:

i have DBConnectionManager.jar put inside a folder (lib
dir=../../../dist/custom/ regex=\*\.jar / from solrconfig.xml) but at
indextime i have this error:


Sep 1, 2011 10:21:28 AM org.apache.solr.common.SolrException log
SEVERE: java.lang.NoClassDefFoundError: db/connection/DBConnectionManager
at tothego_custom.ToTheGoCustom.init(ToTheGoCustom.java:23)
at
tothego_custom.ToTheGoCustomFactory.getInstance(ToTheGoCustomFactory.java:18)
at
org.apache.solr.update.processor.UpdateRequestProcessorChain.createProcessor(UpdateRequestProcessorChain.java:74)

Caused by: java.lang.ClassNotFoundException:
db.connection.DBConnectionManager

... 20 more

I did exactly how you told me: i created the DBConnectionManager singleton,
made the jar, put inside a folder (the one in the lib directive) and added
the lib directive in solrconfig.xml.

In ToTheGoCustomFactory i have import db.connection.*; and no errors at all,
but now solr doesn't find that class... what am i missing this time? i think
it's the last thing i need to understand now hehe :)

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-custom-plugins-is-it-possible-to-have-them-persistent-tp3292781p3300614.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr custom plugins: is it possible to have them persistent?

2011-09-01 Thread Federico Fissore


a brief question: is using an IoC framework like spring an option for you?

if so, maybe this could help
http://lucene.472066.n3.nabble.com/dependency-injection-in-solr-td3292685.html#a3295939

Re: Solr custom plugins: is it possible to have them persistent?

ok solved it by changing (lib dir=../../../dist/custom/ regex=\*\.jar
/) to 

lib path=../../../dist/custom/DBConnectionManager.jar /

ty guys for all yor help, now off to debug some java errors hehe

thanks again, for real!

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-custom-plugins-is-it-possible-to-have-them-persistent-tp3292781p3300629.html
Sent from the Solr - User mailing list archive at Nabble.com.

3.4 release date

2011-09-01 Thread Jordan Leppert

Hi,

Would anyone be able to tell me roughly when version 3.4 will be
released please?

I'm working on a project that needs the grouping functionality, and as
far as I can tell 3.4 is the version that will include the support for
this into SolrJ:
https://issues.apache.org/jira/browse/SOLR-2637

Thanks,
Jordan

index not created

2011-09-01 Thread kumar8anuj

Hi, 

I am trying to index some documents through ExtractingRequestHandler.
everything works fine with the jetty server but when i configure it with
tomcat then my documents are not getting indexed only ids are getting index.
means text field is blank but id field is having values though there is no
error in the catalina.out file. Here is the command and output of the
catalina.out

I have multicore architecture and schema.xml is default which is there for
examples in solr. Please help.

*command to index*

curl
http://localhost:8080/solr/client/update/extract?literal.id=doc1commit=true;
-F myfile=@README.txt


*output of calalina.out*


  Sep 1, 2011 3:09:53 PM org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1

commit{dir=/opt/solr/hh_scraper/client/data/index,segFN=segments_3,version=1314863371898,generation=3,filenames=[_0.tis,
_0.nrm, _0.fnm, _1.tis, _1.frq, _1_1.del, _1.fnm, _1.fdx, _1.prx, _0.tii,
_1.fdt, _0.frq, _1.tii, _0.fdx, _0.prx, _1.nrm, segments_3, _0.fdt]
Sep 1, 2011 3:09:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 1314863371898
Sep 1, 2011 3:09:53 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=false,waitSearcher=true,expungeDeletes=false)
Sep 1, 2011 3:09:53 PM org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2

commit{dir=/opt/solr/hh_scraper/client/data/index,segFN=segments_3,version=1314863371898,generation=3,filenames=[_0.tis,
_0.nrm, _0.fnm, _1.tis, _1.frq, _1_1.del, _1.fnm, _1.fdx, _1.prx, _0.tii,
_1.fdt, _0.frq, _1.tii, _0.fdx, _0.prx, _1.nrm, segments_3, _0.fdt]

commit{dir=/opt/solr/hh_scraper/client/data/index,segFN=segments_4,version=1314863371899,generation=4,filenames=[_0.tis,
_1.frq, _2.tii, _1.fnm, _1.tii, _0.prx, _0.nrm, _0.fnm, _1.tis, _2.prx,
_2.fdt, _0_1.del, _2.frq, _1_1.del, _2.fdx, _2.fnm, _1.prx, _1.fdx, _2.tis,
_0.tii, _1.fdt, _0.frq, _0.fdx, _0.fdt, _1.nrm, _2.nrm, segments_4]
Sep 1, 2011 3:09:53 PM org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 1314863371899
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher init
INFO: Opening Searcher@1510d96 main
Sep 1, 2011 3:09:53 PM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1510d96 main from Searcher@88a970 main

fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1510d96 main

fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1510d96 main from Searcher@88a970 main

filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1510d96 main

filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1510d96 main from Searcher@88a970 main

queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1510d96 main

queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1510d96 main from Searcher@88a970 main

documentCache{lookups=2,hits=1,hitratio=0.50,inserts=1,evictions=0,size=1,warmupTime=0,cumulative_lookups=10,cumulative_hits=8,cumulative_hitratio=0.80,cumulative_inserts=2,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1510d96 main

documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=10,cumulative_hits=8,cumulative_hitratio=0.80,cumulative_inserts=2,cumulative_evictions=0}
Sep 1, 2011 3:09:53 PM

Issue with Solr and copyFields

2011-09-01 Thread Julien Balmont

Hello,

I meet an issue with Solr and copyFields and other things probably. here's a
description of the issue. if anyone could help, it would be greatly
appreciated as I've searched everywhere an am not able to figure out what's
happening.

my Solr is configured as follow:
DIH request:
SELECT d.label, d.description,r.name, r.id, r.phone FROM tab_r r INNER JOIN
tab_d d ON d.rid = r.rid

schema.xml
field name=label type=text_general indexed=true stored=true
required=true /
field name=description type=text_general indexed=true stored=true
required=true /
field name=name type=text_general indexed=true stored=true
required=true /
field name=id type=int indexed=true stored=true required=true /
field name=phone type=string indexed=true stored=true
required=true /

If I leave everything as is, all is working fine.

BUT, if I add the following lines to the schema.xml
field name=search type=text_general indexed=true stored=false /

copyField source=* dest=search/

Then the import command gives:
str name=Total Documents Failed135860/str


Is there anything i'm doing wrong?

Thanks

Re: Issue with Solr and copyFields

you need to define the search field as MultiValued since you're copying
into it multiple sources

http://wiki.apache.org/solr/FAQ#How_do_I_use_copyField_with_wildcards.3F



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-with-Solr-and-copyFields-tp3300763p3300794.html
Sent from the Solr - User mailing list archive at Nabble.com.

Solritas issue in opening files

2011-09-01 Thread Jagdish Kumar


Hi 
 
I have indexed some files from a directory and I can see them in results @ 
http://localhost:8080/solr/browse
I have also added a field Location which displays the file Location as Link :
 
Following changes I have done for Links stuff:
1. Added field column=fileAbsolutePath name=links/ in data-config.xml for 
FileListEntityProcessor.
2.MetaData field field name=links type=string indexed=true stored=true 
multiValued=true/ is already there in schema.xml
3.I have updated doc.vm with : divLocation: a 
href=#field('links')mlt=true#field('links')/a/div
 
It does shows a Location field in resultsas a hyperlink, but when I try and 
open file by clicking on the link, it fails to open the file from local.
 
It would be gr8 if someone can help me out with this issue.
 
Thanks and regards
Jagdish

Re: Solritas issue in opening files

2011-09-01 Thread Balaji N.S. [via Lucene]

Hi 

To open files from local system use file protocol . (file:///followed by
absolute path)

If you are running the application in jboss instead of jetty you can deploy
the input files as a separate war which is exploded and then avoid using
file protocol.

File protocol is erroneous. It is not supported by reputed browser like
Firefox. 




__
If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/Solritas-issue-in-opening-files-tp3300836p3300867.html
This email was sent by Balaji N.S. (via Nabble)
To receive all replies by email, subscribe to this discussion: 
http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=subscribe_by_codenode=3300836code=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3JnfDMzMDA4MzZ8MTIyNjQ0MTk0Mg==

Re: Solritas issue in opening files

2011-09-01 Thread Balaji N.S. [via Lucene]

Hi

To open files from local system use file protocol . (file:///followed by
absolute path)

If you are running the application in jboss instead of jetty you can deploy
the input files as a separate war which is exploded and then avoid using
file protocol.

File protocol is erroneous. It is not supported by reputed browser like
Firefox.



__
If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/Solritas-issue-in-opening-files-tp3300836p3300872.html
This email was sent by Balaji N.S. (via Nabble)
To receive all replies by email, subscribe to this discussion: 
http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=subscribe_by_codenode=3300836code=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3JnfDMzMDA4MzZ8MTIyNjQ0MTk0Mg==

Re: Solr custom plugins: is it possible to have them persistent?

2011-09-01 Thread AbelHerrero

Solr allows you to load custom code to perform a variety of tasks within Solr
-- from custom Request Handlers to process your searches, to custom
Analyzers and Token Filters for your text field, even custom Field Types. 

-

kitchen cabinet 
--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-custom-plugins-is-it-possible-to-have-them-persistent-tp3292781p3300934.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: Solritas issue in opening files

2011-09-01 Thread Jagdish Kumar


Hi Balaji
 
Thanks for your reply, I have tried file:///Absolute path stuff as well.. it 
still fails to open the file in mozilla/ IE both
 
My files are not placed in Solr-home , can that be the issue ? please suggest.
 
Thanks 
Jagdish
 

 Date: Thu, 1 Sep 2011 03:40:00 -0700
 From: ml-node+3300872-1705263717-392...@n3.nabble.com
 To: solr-user@lucene.apache.org
 Subject: Re: Solritas issue in opening files
 
 Hi
 
 To open files from local system use file protocol . (file:///followed by
 absolute path)
 
 If you are running the application in jboss instead of jetty you can deploy
 the input files as a separate war which is exploded and then avoid using
 file protocol.
 
 File protocol is erroneous. It is not supported by reputed browser like
 Firefox.
 
 
 
 __
 If you reply to this email, your message will be added to the discussion 
 below:
 http://lucene.472066.n3.nabble.com/Solritas-issue-in-opening-files-tp3300836p3300872.html
 This email was sent by Balaji N.S. (via Nabble)
 To receive all replies by email, subscribe to this discussion: 
 http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=subscribe_by_codenode=3300836code=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3JnfDMzMDA4MzZ8MTIyNjQ0MTk0Mg==

Re: Solr custom plugins: is it possible to have them persistent?

Thanks, but this was not the point of the topic :) I'm way more further than
this :) Please, avoid random replies :)

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-custom-plugins-is-it-possible-to-have-them-persistent-tp3292781p3301057.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: NRT and commit behavior

2011-09-01 Thread Erick Erickson

Hmm, I'm guessing a bit here, but using an invalid query
doesn't sound very safe, but I suppose it *might* be OK.

What does invalid mean? Syntax error? not safe.

search that returns 0 results? I don't know, but I'd guess
that filling your caches, which is the point of warming
queries, might be short circuited if the query returns
0 results but I don't know for sure.

But the fact that invalid queries return quicker does not
inspire confidence since the *point* of warming queries
is to spend the time up front so your users don't have to
wait.

So here's a test. Comment out your warming queries.
Restart your server and fire the warming query from
the browser withdebugQuery=on and look at the
QTime parameter.

Now fire the same form of the query (as in the same
sort, facet, grouping, etc, but presumably a valid
term). See the QTime.

Now fire the same form of the query with a *different*
value in the query. That is, it should search on different
terms but with the same sort, facet, etc. to avoid
getting your data straight from the queryResultCache.

My guess is that the last query will return much more quickly
than the second query. Which would indicate that the first
form isn't doing you any good.

But a test is worth a thousand opinions.

Best
Erick

On Wed, Aug 31, 2011 at 11:04 AM, Tirthankar Chatterjee
tchatter...@commvault.com wrote:
 Also noticed that waitSearcher parameter value is not  honored inside 
 commit. It is always defaulted to true which makes it slow during indexing.

 What we are trying to do is use an invalid query (which wont return any 
 results) as a warming query. This way the commit returns faster. Are we doing 
 something wrong here?

 Thanks,
 Tirthankar

 -Original Message-
 From: Jonathan Rochkind [mailto:rochk...@jhu.edu]
 Sent: Monday, July 18, 2011 11:38 AM
 To: solr-user@lucene.apache.org; yo...@lucidimagination.com
 Subject: Re: NRT and commit behavior

 In practice, in my experience at least, a very 'expensive' commit can still 
 slow down searches significantly, I think just due to CPU (or
 i/o?) starvation. Not sure anything can be done about that.  That's my 
 experience in Solr 1.4.1, but since searches have always been async with 
 commits, it probably is the same situation even in more recent versions, I'd 
 guess.

 On 7/18/2011 11:07 AM, Yonik Seeley wrote:
 On Mon, Jul 18, 2011 at 10:53 AM, Nicholas Chasench...@earthlink.net  
 wrote:
 Very glad to hear that NRT is finally here!  But my question is this:
 will things still come to a standstill during a commit?
 New updates can now proceed in parallel with a commit, and searches
 have always been completely asynchronous w.r.t. commits.

 -Yonik
 http://www.lucidimagination.com

 **Legal Disclaimer***
 This communication may contain confidential and privileged
 material for the sole use of the intended recipient. Any
 unauthorized review, use or distribution by others is strictly
 prohibited. If you have received the message in error, please
 advise the sender by reply email and delete the message. Thank
 you.
 *

Re: [Q]Solr response passed to remote JsonStore - highlighting properties embed in the response part

2011-09-01 Thread Erick Erickson

You probably want to write your own ResponseWriter,
since you're working in JSON, maybe
JSONResponseWriter is the place to start.

Best
Erick

On Wed, Aug 31, 2011 at 4:58 AM, malic benbenfoxw...@yahoo.com.hk wrote:
 Hello, I have a very specific question about the Solr response passed to
 remote JsonStore.

 *Solr response passed to remote JsonStore*

 var myJsonStore =  new Ext.data.JsonStore({
            // store configs
                        url: myurl,
                        baseParams:
 {'wt':'json','facet':true,'facet.limit':-1,'facet.sort':'description','hl':true,'hl.fl':'*'},
          // reader configs
                        totalProperty: 'total',
                        idProperty: 'handle',
                        root:function(v){
                                return v.response.docs;
                        },
                        fields: ['handle', 'description']
 })

 *Solr standard output:*

 {
    responseHeader: {
        status: 0,
        QTime: 32
    },
    response: {
        total: 21,
        start: 0,
        docs: [
            {
                description: The matte finish waves on this wedding band
 contrast with the high polish borders. This sharp and elegant design was
 finely crafted in Japan.,
                handle: 8252,

            },
            {
                description: This elegant ring has an Akoya cultured
 pearl with a band of bezel-set round diamonds making it perfect for her to
 wear to work or the night out.,
                handle: 8142,

            },

        ]
    },
    highlighting: {
        8252: {
            description: [
                 and emelegant/em design was finely crafted in Japan.
            ]
        },
        8142: {
            description: [
                This emelegant/em ring has an Akoya cultured pearl with
 a band of bezel-set round diamonds making
            ]
        },

    }
 }


 *What I want:* to change the output by embedding the highlighting properties
 into the response properties, such that the response part looks like:

    response: {
        numFound: 21,
        start: 0,
        docs: [
            {
                description: The matte finish waves on this wedding band
 contrast with the high polish borders. This sharp and emelegant/em
 design was finely crafted in Japan.,
                UID_PK: 8252,

            },
            {
                description: This emelegant/em ring has an Akoya
 cultured pearl with a band of bezel-set round diamonds making it perfect for
 her to wear to work or the night out.,
                UID_PK: 8142,

            },

        ]
    },

 Can anyone suggest an approach to do this? Thx a lot.


 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/Q-Solr-response-passed-to-remote-JsonStore-highlighting-properties-embed-in-the-response-part-tp3297811p3297811.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr commit process and read downtime

2011-09-01 Thread Erick Erickson

See below:

On Wed, Aug 31, 2011 at 2:16 PM, Mike Austin mike.aus...@juggle.com wrote:
 I've set up a master slave configuration and it's working great!  I know
 this is the better setup but if I had just one index due to requirements,
 I'd like to know more about the performance hit of the commit. let's just
 assume I have a decent size index of a few gig normal sized documents with
 high traffic.  A few questions:

 - (main question) When you do a commit on a single index, is there anytime
 when the reads will not have an index to search on?
No. While the new searcher is warming up, all incoming searches are
handled by the old searcher. When the new searcher is warmed up,
new requests are routed to it, and when the last search is completed
in the old searcher, it's shut down

 - With the rebuilding of caches and whatever else happens, is the only
 downside the fact that the server performance will be degraded due to file
 copy, cache warming, etc.. or will the index be actually locked at some
 point?
The index will not be locked, if by locked you mean the searches will
not happen. See above. The server will certainly have more work to
do, and if you're running close to the limits you might notice some
slowdown. But often there is no noticeable pause. Note that while
all this goes on, you will have *two* copies of the caches etc. in
memory...

 - On a commit, do the files get copied so you need double the space or is
 that just the optimize?
You have to allow for the relatively rare instance when the merge
process combines all your segments into one, which will require
at least double the disk space. Optimize guarantees this
will happen, but it can (and will) happen on commit occasionally.


 I know a master/slave setup is used to reduce these issues, but if I had
 only one server I need to know the potential risks.
Well, you're just putting lots of stuff on a server. Solr will quite
happily deal
with this situation and, depending upon how much traffic you have and
your machine's size, this may be fine. Do be aware of the warmup hell
problem and don't commit too frequently or your warming searchers
may tie their knickers in a knot.

And one risk in this setup is that you have no way to quickly bring up
a server if your one machine crashes, you have to re-index *all* your data.

Best
Erick


 Thanks,
 Mike

Re: Solritas issue in opening files

2011-09-01 Thread Erik Hatcher

If your files are under Solr's conf/ directory, you can get them served up
using the ShowFileRequestHandler (see how the /browse serves up CSS and the
autocomplete JQuery library). But otherwise, Solritas doesn't give you any
capability to serve up files and the browsers aren't too happy about that
either crossing domains and all.

This is not a Solr issue, it's a browser issue. But Solr (not Solritas per se)
offers the ShowFileRequestHandler to serve up things under conf/ if you can get
your files under there.

Erik

On Sep 1, 2011, at 07:19 , jagdish2011 [via Lucene] wrote:

Hi Balaji

Thanks for your reply, I have tried file:///Absolute path stuff as well.. it
still fails to open the file in mozilla/ IE both

My files are not placed in Solr-home , can that be the issue ? please suggest.

Thanks
Jagdish

Date: Thu, 1 Sep 2011 03:40:00 -0700
From: ml-node+3300872-1705263717-392...@n3.nabble.com
To: solr-user@lucene.apache.org
Subject: Re: Solritas issue in opening files

To open files from local system use file protocol . (file:///followed by
absolute path)

If you are running the application in jboss instead of jetty you can deploy
the input files as a separate war which is exploded and then avoid using
file protocol.

File protocol is erroneous. It is not supported by reputed browser like
Firefox.

__
If you reply to this email, your message will be added to the discussion
below:
http://lucene.472066.n3.nabble.com/Solritas-issue-in-opening-files-tp3300836p3300872.html
This email was sent by Balaji N.S. (via Nabble)
To receive all replies by email, subscribe to this discussion:
http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=subscribe_by_codenode=3300836code=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3JnfDMzMDA4MzZ8MTIyNjQ0MTk0Mg==

___
If you reply to this email, your message will be added to the discussion
below:
http://lucene.472066.n3.nabble.com/Solritas-issue-in-opening-files-tp3300836p3300964.html

To unsubscribe from Solritas issue in opening files, visit
http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=3300836code=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3JnfDMzMDA4MzZ8MTIyNjQ0MTk0Mg==

Solr replication / repeater

2011-09-01 Thread Rene Lehmann



Hi there,

i´m really new in Solr and have a question about the Solr replication.
We want to use Solr in two data centers (dedicated fibre channel lane, like
intranet) behind a load balancer. Is the following infrastructure possible?

- one repeater and one slave per data center
- the repeaters used to each other for replication
- the slaves uses the local repeater for replication

Such a construction is possible? Or do I need a pure master server?

kind regards,

Rene

P.S. Sorry for the second mail. But now this is an own thread.

Re: Solr commit process and read downtime

2011-09-01 Thread Mike Austin

Wow.. thanks for the great answers Erick!  This answered my concerns
perfectly.

Mike

On Thu, Sep 1, 2011 at 7:54 AM, Erick Erickson erickerick...@gmail.comwrote:

 See below:

 On Wed, Aug 31, 2011 at 2:16 PM, Mike Austin mike.aus...@juggle.com
 wrote:
  I've set up a master slave configuration and it's working great!  I know
  this is the better setup but if I had just one index due to requirements,
  I'd like to know more about the performance hit of the commit. let's just
  assume I have a decent size index of a few gig normal sized documents
 with
  high traffic.  A few questions:
 
  - (main question) When you do a commit on a single index, is there
 anytime
  when the reads will not have an index to search on?
 No. While the new searcher is warming up, all incoming searches are
 handled by the old searcher. When the new searcher is warmed up,
 new requests are routed to it, and when the last search is completed
 in the old searcher, it's shut down

  - With the rebuilding of caches and whatever else happens, is the only
  downside the fact that the server performance will be degraded due to
 file
  copy, cache warming, etc.. or will the index be actually locked at some
  point?
 The index will not be locked, if by locked you mean the searches will
 not happen. See above. The server will certainly have more work to
 do, and if you're running close to the limits you might notice some
 slowdown. But often there is no noticeable pause. Note that while
 all this goes on, you will have *two* copies of the caches etc. in
 memory...

  - On a commit, do the files get copied so you need double the space or is
  that just the optimize?
 You have to allow for the relatively rare instance when the merge
 process combines all your segments into one, which will require
 at least double the disk space. Optimize guarantees this
 will happen, but it can (and will) happen on commit occasionally.

 
  I know a master/slave setup is used to reduce these issues, but if I had
  only one server I need to know the potential risks.
 Well, you're just putting lots of stuff on a server. Solr will quite
 happily deal
 with this situation and, depending upon how much traffic you have and
 your machine's size, this may be fine. Do be aware of the warmup hell
 problem and don't commit too frequently or your warming searchers
 may tie their knickers in a knot.

 And one risk in this setup is that you have no way to quickly bring up
 a server if your one machine crashes, you have to re-index *all* your data.

 Best
 Erick

 
  Thanks,
  Mike

MoreLikeThis assumptions

2011-09-01 Thread Herman Kiefus

Given a document id:n show me those other documents with similar values in the 
'Name' field:

http://devsolr03:8983/solr/primary/select?q=id:182652fl=id,Name,scoremlt=truemlt.fl=Name

My assumption is the above query will generate the desired outcome.  It does; 
however, given a different document (id) it does not.  Both id's identify a 
document whose name contains the term 'smith'.  Stated differently if A is like 
B, C, and D I would assume that B is like A, C, and D, but these are not the 
results that I'm seeing.

My objective is to simply seek out similar documents (based on several fields, 
I'm just using one here) for any given document; a simple 'duplicate checker' 
if you will.  Am I misguided in my assumptions?

Re: core creation and instanceDir parameter

2011-09-01 Thread Gérard Dupont

On 31 August 2011 20:27, Jaeger, Jay - DOT jay.jae...@dot.wi.gov wrote:

 Well, if it is for creating a *new* core, Solr doesn't know it is pointing
 to your shared conf directory until after you create it, does it?

 JRJ


Indeed, but the conf directory is not a problem for me. The things is I
would like to avoid to send instance path.

-- 
Gérard Dupont
Information Processing Control and Cognition (IPCC)
CASSIDIAN - an EADS company

Document  Learning team - LITIS Laboratory

Getting MoreLikeThisHandler operational.

2011-09-01 Thread Herman Kiefus

I've begun tinkering with MLT using the standard request handler.  The Wiki 
also suggests using the MoreLikeThis handler directly, but apparently, this is 
not in the default configuration (as I recall, I haven't removed anything from 
solrconfig.xml as shipped).  For example: 
http://devsolr03:8983/solr/primary/mlt?q=id:3197684fl=id,Name,Scoremlt=truemlt.fl=Name
 yields 'The requested resource is not available'.

I tried adding this to my solrconfig.xml:

searchComponent name=mlt 
class=org.apache.solr.handler.component.MoreLikeThisComponent /
requestHandler name=/mlt 
class=org.apache.solr.handler.component.MoreLikeThisComponent
   arr name=components
  strmlt/str
/arr
/requestHandler

but ends up returning a 500 error on a core reload.  What is an appropriate 
configuration entry for the MLT handler?

Best approach to Intersect results with big SetString?

2011-09-01 Thread Ryan McKinley

I have an application where I need to return all results that are not
in a SetString  (the Set is managed from hazelcast... but that is
not relevant)

As a fist approach, i have a SerachComponent that injects a BooleanQuery:

  BooleanQuery bq = new BooleanQuery(true);
  for( String id : ids) {
bq.add(new BooleanClause(new TermQuery(new
Term(id,id)),Occur.MUST_NOT));
  }

This works, but i'm concerned about how many terms we could end up
with as the size grows.

Another possibility could be a Filter that iterates though FieldCache
and checks if each value is in the SetString

Any thoughts/directions on things to look at?

thanks
ryan

Re: Getting MoreLikeThisHandler operational.

2011-09-01 Thread Koji Sekiguchi


(11/09/01 23:24), Herman Kiefus wrote:

requestHandler name=/mlt 
class=org.apache.solr.handler.component.MoreLikeThisComponent
arr name=components
   strmlt/str
/arr
/requestHandler

but ends up returning a 500 error on a core reload.  What is an appropriate 
configuration entry for the MLT handler?


Why you got 500 error because MLTComponent was set for requestHandler class.
Set class=solr.SearchHandler for it.

koji
--
Check out Query Log Visualizer for Apache Solr
http://www.rondhuit-demo.com/loganalyzer/loganalyzer.html
http://www.rondhuit.com/en/

Re: Post Processing Solr Results

Ok, so I feel like I'm 90% of the way there. For standard queries
things work fine, but for distributed queries I'm running into a bit
of an issue. Right now the queries run fine but when doing
distributed queries (using SolrCloud) the numFound is always getting
set to the number of requested rows. Can anyone shed some light on
why this might be happening?

On Tue, Aug 30, 2011 at 8:53 AM, Jamie Johnson jej2...@gmail.com wrote:
This might work in conjunction with what POST processing to help to
pair down the results, but the logic for the actual access to the data
is too complex to have entirely in solr.

On Mon, Aug 29, 2011 at 2:02 PM, Erick Erickson erickerick...@gmail.com
wrote:
It's reasonable, but post-filtering is often difficult, you have
too many documents to wade through. If you can see any way
at all to just include a clause in the query, you'll save a world
of effort...

Is there any way you can include a value in some kind of
permissions field? Let's say you have a document that
is only to be visible for tier 1 customers. If your permissions
field contained the tiers (e.g. tier0, tier1), then a simple
AND permissions:tier1 would do the trick...

I know this is a trivial example, but you see where this is headed.
The documents can contain as many of these tokens in permissions
as you want. As long as you can string together a clause
like AND permissions:(A OR B OR C) and not have the clause
get ridiculously long (as in thousands of values), that works best.

Any such scheme depends upon being able to assign the documents
some kind of code that doesn't change too often (because when it does
you have to re-index) and figure out, at query time, what permissions
a user has.

Using FieldCache or low-level Lucene routines can answer the question
Does doc X contain token Y in field Z reasonably easily. What it has
a hard time doing is answering For document X, what are all the value
in the inverted index in field Z.

If this doesn't make sense, could you explain a bit more about your
permissions model?

Hope this helps
Erick

On Mon, Aug 29, 2011 at 11:46 AM, Jamie Johnson jej2...@gmail.com wrote:
Thanks guys, perhaps I am just going about this the wrong way. So let
me explain my problem and perhaps there is a more appropriate
solution. What I need to do is basically hide certain results based
on some passed in user parameter (say their service tier for
instance). What I'd like to do is have some way to plugin my custom
logic to basically remove certain documents from the result set using
this information. Now that being said I technically don't need to
remove the documents from the full result set, I really only need to
remove them from current page (but still ensuring that a page is
filled and sorted). At present I'm trying to see if there is a way
for me to add this type of logic after the QueryComponent has
executed, perhaps by going through the DocIdandSet at this point and
then intersecting the DocIdSet with a DocIdSet which would filter out
the stuff I don't want seen. Does this sound reasonable or like a
fools errand?

On Mon, Aug 29, 2011 at 10:51 AM, Erik Hatcher erik.hatc...@gmail.com
wrote:
I haven't followed the details, but what I'm guessing you want here is
Lucene's FieldCache. Perhaps something along the lines of how faceting
uses it (in SimpleFacets.java) -

FieldCache.DocTermsIndex si =
FieldCache.DEFAULT.getTermsIndex(searcher.getIndexReader(), fieldName);

Erik

On Aug 29, 2011, at 09:58 , Erick Erickson wrote:

If you're asking whether there's a way to find, say,
all the values for the auth field associated with
a document... no. The nature of an inverted
index makes this hard (think of finding all
the definitions in a dictionary where the word
earth was in the definition).

Best
Erick

On Mon, Aug 29, 2011 at 9:21 AM, Jamie Johnson jej2...@gmail.com wrote:
Thanks Erick, if I did not know the token up front that could be in
the index is there not an efficient way to get the field for a
specific document and do some custom processing on it?

On Mon, Aug 29, 2011 at 8:34 AM, Erick Erickson
erickerick...@gmail.com wrote:
Start here I think:

http://lucene.apache.org/java/3_0_2/api/core/index.html?org/apache/lucene/index/TermDocs.html

Best
Erick

On Mon, Aug 29, 2011 at 8:24 AM, Jamie Johnson jej2...@gmail.com
wrote:
Thanks for the reply. The fields I want are indexed, but how would I
go directly at the fields I wanted?

In regards to indexing the auth tokens I've thought about this and am
trying to get confirmation if that is reasonable given our
constraints.

On Mon, Aug 29, 2011 at 8:20 AM, Erick Erickson
erickerick...@gmail.com wrote:
Yeah, loading the document inside a Collector is a
definite no-no. Have you tried going directly
at the fields you want (assuming they're
indexed)? That *should* be much faster, but
whether it'll be fast enough is a good question.

Autocomplete

2011-09-01 Thread Brian Lamb

Hi all,

I've read numerous guides on how to set up autocomplete on solr and it works
great the way I have it now. However, my only complaint is that it only
matches the beginning of the word. For example, if I try to autocomplete
dober, I would only get, Doberman, Doberman Pincher but not Pincher,
Doberman. Here is how my schema is configured:

fieldType name=edgytext class=solr.TextField
positionIncrementGap=100
   analyzer type=index
 tokenizer class=solr.KeywordTokenizerFactory/
 filter class=solr.LowerCaseFilterFactory/
 filter class=solr.EdgeNGramFilterFactory minGramSize=1
maxGramSize=25 /
   /analyzer
   analyzer type=query
 tokenizer class=solr.KeywordTokenizerFactory/
 filter class=solr.LowerCaseFilterFactory/
   /analyzer
/fieldType

field name=autocomplete_text type=edgytext indexed=true stored=true
omitNorms=true omitTermFreqAndPositions=true /

How can I update my autocomplete so that it will match the middle of a word
as well as the beginning of the word?

Thanks,

Brian Lamb

RE: is it possible to do automatic indexing in solr ?

2011-09-01 Thread Jaeger, Jay - DOT

If you are indexing data, rather than documents, another possibility is to 
use database triggers to fire off updates.

-Original Message-
From: Erik Hatcher [mailto:erik.hatc...@gmail.com] 
Sent: Wednesday, August 31, 2011 9:13 AM
To: solr-user@lucene.apache.org
Subject: Re: is it possible to do automatic indexing in solr ?

There is no scheduling built into Solr.  But many, including the search system 
deployed on our (Lucid's) website, is powered by cron jobs kicking off indexers 
of various varieties all the time.

Look into your operating systems scheduling capabilities and leverage those, is 
my advice.  Cron is your friend.

Erik

On Aug 31, 2011, at 09:59 , vighnesh wrote:

 hi all

 i am unable to do the scheduling in solr so is there any way to do automatic
 indexing in solr.Please give the solution on automatic idexing or specify
 the procedure for how to do schduling in solr.

 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/is-it-possible-to-do-automatic-indexing-in-solr-tp3298428p3298428.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Autocomplete

2011-09-01 Thread Brian Lamb

I found that if I change

filter class=solr.EdgeNGramFilterFactory minGramSize=1 maxGramSize=25
/

to

filter class=solr.NGramFilterFactory minGramSize=1 maxGramSize=25 /

I can do autocomplete in the middle of a term.

Thanks!

Brian Lamb

On Thu, Sep 1, 2011 at 11:27 AM, Brian Lamb
brian.l...@journalexperts.comwrote:

 Hi all,

 I've read numerous guides on how to set up autocomplete on solr and it
 works great the way I have it now. However, my only complaint is that it
 only matches the beginning of the word. For example, if I try to
 autocomplete dober, I would only get, Doberman, Doberman Pincher but
 not Pincher, Doberman. Here is how my schema is configured:

 fieldType name=edgytext class=solr.TextField
 positionIncrementGap=100
analyzer type=index
  tokenizer class=solr.KeywordTokenizerFactory/
  filter class=solr.LowerCaseFilterFactory/
  filter class=solr.EdgeNGramFilterFactory minGramSize=1
 maxGramSize=25 /
/analyzer
analyzer type=query
  tokenizer class=solr.KeywordTokenizerFactory/
  filter class=solr.LowerCaseFilterFactory/
/analyzer
 /fieldType

 field name=autocomplete_text type=edgytext indexed=true
 stored=true omitNorms=true omitTermFreqAndPositions=true /

 How can I update my autocomplete so that it will match the middle of a word
 as well as the beginning of the word?

 Thanks,

 Brian Lamb

Re: Post Processing Solr Results

Ok, think I got it. Basically the issue was that I can't modify the
offset and start params when the search is a distributed one,
otherwise the correct offset and max are lost, a simple check in
prepare fixed this.

On Thu, Sep 1, 2011 at 11:10 AM, Jamie Johnson jej2...@gmail.com wrote:
Ok, so I feel like I'm 90% of the way there. For standard queries
things work fine, but for distributed queries I'm running into a bit
of an issue. Right now the queries run fine but when doing
distributed queries (using SolrCloud) the numFound is always getting
set to the number of requested rows. Can anyone shed some light on
why this might be happening?

If this doesn't make sense, could you explain a bit more about your
permissions model?

Hope this helps
Erick

FieldCache.DocTermsIndex si =
FieldCache.DEFAULT.getTermsIndex(searcher.getIndexReader(), fieldName);

Erik

On Aug 29, 2011, at 09:58 , Erick Erickson wrote:

Best
Erick

On Mon, Aug 29, 2011 at 8:34 AM, Erick Erickson
erickerick...@gmail.com wrote:
Start here I think:

http://lucene.apache.org/java/3_0_2/api/core/index.html?org/apache/lucene/index/TermDocs.html

Best
Erick

On Mon, Aug 29, 2011 at 8:24 AM, Jamie Johnson jej2...@gmail.com
wrote:
Thanks for the reply. The fields I want are indexed, but how would I
go directly at the fields I wanted?

In regards to indexing the auth tokens I've thought about this and am
trying to get confirmation if that is reasonable given our
constraints.

On Mon, Aug 29, 2011

Re: A good signature class for deduplication


: I want to deduplicate documents from search results. What should be the
: parameters on which I should decide an efficient SignatureClass? Also, what
: are the SignaureClasses available?

the signature classes available are the ones mentioned on the wiki...

https://wiki.apache.org/solr/Deduplication

...which one you should choose, and which fields you feed it depend 
entirely on your goal -- if you want to deduplicate anytime both the 
user_fname and user_lname fields are exactly the same, then use those 
fields with either the MD5Signature  or the Lookup3Signature -- (lookup3 
is faster, but some people want MD5 because they want to use the computed 
MD5 for other things)

if you want to detext when some much longer body field containing a lot 
of full test is *nearly* identical, then you should consider the 
TextProfileSignature -- how exactly it works and how you tune it i 
don't know off the top of my head.



-Hoss

Re: custom filterquery


: pricing.  I have written a functionquery to get the pricing, which works
: fine as part of the search query, but doesn't seem to be doing anything when
: I try to use it in a filter query.  I wrote my pricing function query based

how are you trying to use it in a filter query?

function queries by definition match all documents -- the function value 
just determines the score.

If you want to filter on a function query you have to use something like 
the frange parser to specify that only certian function values should 
match...

https://lucene.apache.org/solr/api/org/apache/solr/search/FunctionRangeQParserPlugin.html


-Hoss

Wildcard Query

2011-09-01 Thread Aaron Bains

Hello,

The keywords field type is text_en_splitting

My query is as follows:q=keywords:(symantec AND corporation)
Result: Documents are returned as normal


My wildcard query is as follows:  q=keywords:(symante* AND corporation)
Result: Wildcard functions correctly, and documents are returned


When I try a wildcard on every term as follows:  q=keywords:(symante*
AND corporatio*)
Result: No documents are returned.

Re: Wildcard Query

2011-09-01 Thread Aaron Bains

It seems to work correctly once I remove the brackets like this:
q=keywords:symante* AND corporatio*
But I don't understand why...

On Thu, Sep 1, 2011 at 2:26 PM, Aaron Bains aaronba...@gmail.com wrote:

 Hello,

 The keywords field type is text_en_splitting

 My query is as follows:q=keywords:(symantec AND corporation)
 Result: Documents are returned as normal


 My wildcard query is as follows:  q=keywords:(symante* AND corporation)
 Result: Wildcard functions correctly, and documents are returned


 When I try a wildcard on every term as follows:  q=keywords:(symante*
 AND corporatio*)
 Result: No documents are returned.




-- 
Aaron Bains, Ivey HBA
+1 519.868.0820 (Mobile)
aar...@microcad.ca

Re: Solr Join with multiple query parameters


: 
http://localhost:8983/solr/core0/select?q={!join%20from=matchset_id_ss%20to=id}*:*fq=status_s:completed
: 
: I get filtered results of documents that are completed. The issue I am now
: trying to face is how do I filter the initial search of documents based on
: multiple conditions and then get a list of documents through the join. Here is
...
: Is there a way that I can apply multiple filters first, then complete the
: join? And if that is possible, can I then add more filters after the join?

I'm not certian, but i believe what you need to do is formulate your 
filters as part of the main sub query for the join (currently *:*).
You can make it a little cleaner by using LocalParam variable refs, but 
unfortunately LocalParams doesn't really have any syntax for refering to 
multivalued params, and i don't think there's any way to do filter 
queries. 

So i think you would have to do soemthing like...

q={!join from=matchset_id_ss to=id v=$qq}
qq=+status_i:1 +team_id_i:1223

...and make the join query into your filter constraint on the join docs

It would be nice if we had a more genearl way to support something like...

param={!foo fq=filterA fq=filterB}main query

...where the query produced was automaticly constrained by filterA and 
filterB (as if in a BooleanQuery) but those filter queries were cached 
independently -- but i'm not sure off the top of my head how to implement 
something like that.

-Hoss

RE: Getting MoreLikeThisHandler operational.

2011-09-01 Thread Herman Kiefus

Thank you very much.

requestHandler name=/mlt class=solr.MoreLikeThisHandler
lst name=defaults
str name=mlt.flName/str
/lst
arr name=components
strmlt/str
/arr
/requestHandler

-Original Message-
From: Koji Sekiguchi [mailto:k...@r.email.ne.jp] 
Sent: Thursday, September 01, 2011 11:06 AM
To: solr-user@lucene.apache.org
Subject: Re: Getting MoreLikeThisHandler operational.

(11/09/01 23:24), Herman Kiefus wrote:
 requestHandler name=/mlt 
 class=org.apache.solr.handler.component.MoreLikeThisComponent
 arr name=components
strmlt/str
 /arr
 /requestHandler

 but ends up returning a 500 error on a core reload.  What is an appropriate 
 configuration entry for the MLT handler?

Why you got 500 error because MLTComponent was set for requestHandler class.
Set class=solr.SearchHandler for it.

koji
--
Check out Query Log Visualizer for Apache Solr 
http://www.rondhuit-demo.com/loganalyzer/loganalyzer.html
http://www.rondhuit.com/en/

Re: Wildcard Query

2011-09-01 Thread Aaron Bains

I solved the problem by setting multiValued=false

On Thu, Sep 1, 2011 at 2:37 PM, Aaron  wrote:

 It seems to work correctly once I remove the brackets like this:
 q=keywords:symante* AND corporatio*
 But I don't understand why...


 On Thu, Sep 1, 2011 at 2:26 PM, Aaron wrote:

 Hello,

 The keywords field type is text_en_splitting

 My query is as follows:q=keywords:(symantec AND corporation)
 Result: Documents are returned as normal


 My wildcard query is as follows:  q=keywords:(symante* AND
 corporation)
 Result: Wildcard functions correctly, and documents are returned


 When I try a wildcard on every term as follows:  q=keywords:(symante*
 AND corporatio*)
 Result: No documents are returned.

spellchecking in nutch solr

2011-09-01 Thread alxsss



Hello,
I have tried to implement spellchecker based on index in nutch-solr by adding 
spell field to schema.xml and making it a copy from content field. However, 
this increased data folder size twice and spell filed as a copy of content 
field appears in xml feed which is not necessary. Is it possible to implement 
spellchecker without this issue?

Thanks.
Alex.

RE: Context-Sensitive Spelling Suggestions Collations

2011-09-01 Thread Dyer, James

The changes to DirectSpellChecker are included in SOLR-2585 patch, which I 
sync'ed to the current Trunk today.  So all you have to do is apply the patch, 
build and then add the 1-2 new parameters to your query:

- spellcheck.alternativeTermCount - the # of suggestions you want to generate 
on terms that were found in the index (set greater to 0).

- spellcheck.maxResultsForSuggest - the max # of query results to disable 
spellingcheck (normally you'd want 0 for regular spellchecking and something 0 
to generate did you mean suggestions) ... 0 is the default.

If you do try this patch out, please let us know how it works out and also any 
suggestions you have for improving it.  Solr's current inability to consider 
fixes for words appearing in the index is to me a significant functionality 
gap.  I would love to see something like this eventually added to the product 
so any feedback you can give is appreciated.  Thanks.

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311


-Original Message-
From: O. Klein [mailto:kl...@octoweb.nl] 
Sent: Tuesday, August 30, 2011 8:45 AM
To: solr-user@lucene.apache.org
Subject: Context-Sensitive Spelling Suggestions  Collations

Using the DirectSolrSpellChecker im very interested in this.

According to https://issues.apache.org/jira/browse/SOLR-2585 some changes
need to be made to DirectSolrSpellChecker.

Does anybody know how to get this working?

--
View this message in context: 
http://lucene.472066.n3.nabble.com/Context-Sensitive-Spelling-Suggestions-Collations-tp3295570p3295570.html
Sent from the Solr - User mailing list archive at Nabble.com.

Question on functions

2011-09-01 Thread Craig Stadler


Regarding :
http://wiki.apache.org/solr/FunctionQuery#Date_Boosting

Specifcally : recip(ms(NOW/HOUR,mydatefield),3.16e-11,1,1).

I am using dismax, and I am very unsure on where to put this or call the 
function... for example in the fq= param??, in the q= param?


Sample query :
http://localhost:8983/solr/dismax/?q=george 
clooneymm=48%25debugQuery=offindent=onstart=rows=10

If I want to factor in score/date (called creationdate)...

recip(ms(NOW/HOUR,creationdate),3.16e-11,1,1).

Help! and thanks so much for any examples or help..
-Craig

Custom response header from Solr

Is there a way to add arbitrary values into the response header?  I
have a need to insert a boolean into the header and doing something
like

SolrQueryResponse rsp = rb.rsp;
rsp.getResponseHeader().add(testValue, Boolean.TRUE);

Works so long as the query is not distributed.  When the query is
distributed what the individual shards add to the header seem to be
thrown away.  Is there a mechanism for the shard to influence the
header?

Copying a copyfield into another copyfield

2011-09-01 Thread karthik

Hi Everyone,

Sorry if the subject was too vague. What i am trying to do is this:

field name=A/
field name=B/
field name=C multiValued=true/
field name=D /
field name=E multiValued=true/


copyField source=A dest=C/
copyField source=B dest=C/
copyField source=D dest=E/
copyField source=C dest=E/

So basically i am trying to copy one of the destination fields of a copy
field to another field. The question i have is whether the field E will get
populated properly ie., by the time E is getting constructed would the field
C have all the relevant values?

Thanks

Re: Copying a copyfield into another copyfield

This won't work, according to http://wiki.apache.org/solr/SchemaXml#Copy_Fields

This is provided as a convenient way to ensure that data is put into
several fields, without needing to include the data in the update
command multiple times. The copy is done at the stream source level
and no copy feeds into another copy.

On Thu, Sep 1, 2011 at 3:26 PM, karthik kmoha...@gmail.com wrote:
 Hi Everyone,

 Sorry if the subject was too vague. What i am trying to do is this:

 field name=A/
 field name=B/
 field name=C multiValued=true/
 field name=D /
 field name=E multiValued=true/


 copyField source=A dest=C/
 copyField source=B dest=C/
 copyField source=D dest=E/
 copyField source=C dest=E/

 So basically i am trying to copy one of the destination fields of a copy
 field to another field. The question i have is whether the field E will get
 populated properly ie., by the time E is getting constructed would the field
 C have all the relevant values?

 Thanks

Re: Question on functions

2011-09-01 Thread Alexei Martchenko

We put here

requestHandler name=whatever class=solr.StandardRequestHandler
default=true
lst name=defaults
str name=defTypedismax/str
str name=qf.../str
str name=pf.../str
str name=bfrecip(ms(NOW,sear_dataupdate),3.16e-11,1,1)/str
...


2011/9/1 Craig Stadler cstadle...@hotmail.com

 Regarding :
 http://wiki.apache.org/solr/**FunctionQuery#Date_Boostinghttp://wiki.apache.org/solr/FunctionQuery#Date_Boosting

 Specifcally : recip(ms(NOW/HOUR,mydatefield)**,3.16e-11,1,1).

 I am using dismax, and I am very unsure on where to put this or call the
 function... for example in the fq= param??, in the q= param?

 Sample query :
 http://localhost:8983/solr/**dismax/?q=http://localhost:8983/solr/dismax/?q=george
 clooneymm=48%25debugQuery=**offindent=onstart=rows=10
 If I want to factor in score/date (called creationdate)...

 recip(ms(NOW/HOUR,**creationdate),3.16e-11,1,1).

 Help! and thanks so much for any examples or help..
 -Craig





-- 

*Alexei Martchenko* | *CEO* | Superdownloads
ale...@superdownloads.com.br | ale...@martchenko.com.br | (11)
5083.1018/5080.3535/5080.3533

Re: Copying a copyfield into another copyfield

2011-09-01 Thread karthik

Thanks Jamie. Missed to notice that one from the wiki.

Thanks,
Karthik

On Thu, Sep 1, 2011 at 3:38 PM, Jamie Johnson jej2...@gmail.com wrote:

 This won't work, according to
 http://wiki.apache.org/solr/SchemaXml#Copy_Fields

 This is provided as a convenient way to ensure that data is put into
 several fields, without needing to include the data in the update
 command multiple times. The copy is done at the stream source level
 and no copy feeds into another copy.

 On Thu, Sep 1, 2011 at 3:26 PM, karthik kmoha...@gmail.com wrote:
  Hi Everyone,
 
  Sorry if the subject was too vague. What i am trying to do is this:
 
  field name=A/
  field name=B/
  field name=C multiValued=true/
  field name=D /
  field name=E multiValued=true/
 
 
  copyField source=A dest=C/
  copyField source=B dest=C/
  copyField source=D dest=E/
  copyField source=C dest=E/
 
  So basically i am trying to copy one of the destination fields of a copy
  field to another field. The question i have is whether the field E will
 get
  populated properly ie., by the time E is getting constructed would the
 field
  C have all the relevant values?
 
  Thanks

Re: core creation and instanceDir parameter

2011-09-01 Thread lboutros

instanceDir=. 

does that fit your needs ?

Ludovic.

-
Jouve
France.
--
View this message in context: 
http://lucene.472066.n3.nabble.com/core-creation-and-instanceDir-parameter-tp3287124p3302496.html
Sent from the Solr - User mailing list archive at Nabble.com.

Date Faceting on Solr 3.1

2011-09-01 Thread karthik

Hi Everyone,

I am trying to configure date faceting on Solr 3.1. I browsed through the
wiki and understood how to enable and configure it.

To explain this better, lets take an example -

my index has docs with dates ranging from 01/01/1995 until NOW (ie., today)
as of now. To configure date facet I need to provide these 3 parameters -

facet.date.start - i really dont want to give 1/1/1995 as the start date ..
facet.date.end - i can specify this as NOW
facet.date.gap - i would really like to keep this as 2 years at the max

Can solr take the earliest date from the result set to be the value for
facet.date.start? I dont want to have the value 1/1/1995 hardcoded in my
application since a new data feed that gets into my index could be older
than 1995 and i might keep missing them from the facet.

However i did see that there was another field called facet.date.other
that is intended to include edge cases. I see the option before would some
what solve my problem but would like to see if there are any alternatives.

Thanks

-- karthik

Re: geodist() parameters?

2011-09-01 Thread Juan Grande

Hi Bill,

As far as I know, you can pass a completely different set of parameters to
each of the functions/filters. For example:

http://localhost:8983/solr/select?q={!func}add(geodist(field1, 10,
-10),geodist(field2, 20, -20))fq={!geofilt sfield=field3 pt=30,-30
d=100}http://localhost:8983/solr/select?q=%7B%21func%7Dadd%28geodist%28%29,geodist%28%29%29fq=%7B%21geofilt%7Dpt=39.86347,-105.04888d=100sfield=store_lat_lon

Let me know if this solved your problem!

*Juan*



On Wed, Aug 31, 2011 at 11:58 PM, William Bell billnb...@gmail.com wrote:

 I want to go a geodist() calculation on 2 different sfields. How would
 I do that?


 http://localhost:8983/solr/select?q={!func}add(geodist(),geodist())fq={!geofilt}pt=39.86347,-105.04888d=100sfield=store_lat_lon

 But I really want geodist() for one pt, and another geodist() for another
 pt.

 Can I do something like geodist(store_lat_lon,39.86347,-105.04888,100) ?



 --
 Bill Bell
 billnb...@gmail.com
 cell 720-256-8076

Re: copying one field to another using regex

2011-09-01 Thread Markus Jelsma

No, field data is copied verbatim. Copy the field and strip what you don't 
need.

 Hello,
 
 Is it possible to create a copy field from another by applying a regex or a
 function to source.
 
 Thanks.
 Alex.

Re: Getting MoreLikeThisHandler operational.

2011-09-01 Thread Koji Sekiguchi


(11/09/02 4:00), Herman Kiefus wrote:

Thank you very much.

requestHandler name=/mlt class=solr.MoreLikeThisHandler
lst name=defaults
str name=mlt.flName/str
/lst
arr name=components
strmlt/str
/arr
/requestHandler


This is not what I was suggesting. MLT*Handler* is a request handler but not 
SearchComponent-aware
handler. That said, it ignores arr name=components.../arr part. If 
MLTHandler is what you
wanted, just set requestHandler name=/mlt class=solr.MoreLikeThisHandler 
with
appropriate defaults.

koji
--
Check out Query Log Visualizer for Apache Solr
http://www.rondhuit-demo.com/loganalyzer/loganalyzer.html
http://www.rondhuit.com/en/

Re: Test failures on lucene_solr_3_3 and branch_3x


: I've got a consistent test failure on Solr source code checked out from svn.
: The same thing happens with 3.3 and branch_3x.  I have information saved from

Shawn: sorry for hte late reply.

I can't reproduce your specific problem, but the test in question is 
suspiciously hinky enough taht i took a stab at making it work a little 
cleaner...

https://issues.apache.org/jira/browse/SOLR-2739

..can you try running ant test -Dtestcase=TestSqlEntityProcessorDelta 
a few times on the updated 3x branch and comment in the issue about the 
results.

particularly: do you get any failures? if not, do you get any messages in 
the log about asumptions failing?

if you do: can you tell us more about the filesystem you are using?

-Hoss

Re: DataImportHandler using new connection on each query


: However, I tested this against a slower SQL Server and I saw 
: dramatically worse results. Instead of re-using their database, each of 
: the sub-entities is recreating a connection each time the query runs. 

are you seeing any specific errors logged before these new connections are 
created?

I don't *think* there's anything in the DIH JDBC/SQL code that causes it 
to timeout existing connections -- is it possible this is sometihng 
specific to the JDBC Driver you are using?  

Or maybe you are using the DIH threads option along with a JNDI/JDBC 
based pool of connections that is configured to create new Connections on 
demand, and with the fast DB it can reuse them but on the slow DB it does 
enough stuff in parallel to keep asking for new connections to be created?


If it's DIH creating new connections over and over then i'm pretty sure 
you should see an INFO level log message like this for each connection...

LOG.info(Creating a connection for entity 
+ context.getEntityAttribute(DataImporter.NAME) +  with URL: 
+ url);

...are those messages different against you fast DB and your slow DB?

-Hoss

Re: Sorting results by Range


: sort=map(map(myNumField,0,10,0),20,100,0) desc, score desc
: sort=map(map(myNumField,0,10,100),20,100,100) asc, score desc
...
: By doing the second one, I expected to get the same results, ordered like
: 13, 17,18, 20. But, what I got were other values as results, that are not in
: the chosen range: 9, 8,7, 5 etc.
: 
: My question previously was: How to get results with myNumFields values
: between 10 and 20 ranked above the rest, in the ascending order.

Hmmm w/o more info i'm not sure that i can explain the behavior you 
are seeing.

Using the 3x branch, with the 17 example docs indexed, i just tried these 
queries...

http://localhost:8983/solr/select?q=*:*+video+solrfl=score,popularityrows=100sort=score+desc

...returns all 17 docs, ordered by score descending, and shows that all 
the popularity values are between 0 and 10.

http://localhost:8983/solr/select?q=*:*+video+solrfl=score,popularityrows=100sort=map%28map%28popularity,0,5,0%29,8,10,0%29+desc,+score+desc

...reorders the docs so everything with a popularity value above 5 and 
less then 8 is listed first, descending (ie: pop sevens then pop sixes), 
followed by all other docs in order of score descending (regardless of the 
popularity value) 

http://localhost:8983/solr/select?q=*:*+video+solrfl=score,popularityrows=100sort=map%28map%28popularity,0,5,100%29,8,10,100%29+asc,+score+desc

...reorders the docs so everything with a popularity value above 5 and 
less then 8 is listed first, ascending (ie: pop sixes then pop sevents), 
followed by all other docs in order of score descending (regardless of the 
popularity value) 

perhaps you could post your configs and some sample data in an easy to 
consume format (csv?) demonstrating the problem you are seeing?

-Hoss

Re: can i create filters of score range


: so coming back to the issue .. even if am sorting it by _docid_ i need to do
: paging( 2 million docs in result)
: how is it internally doing it ?
: when sorted by docid, don we have deep pagin issue ? (getting all the
: previous pages into memory to get the next page)
: so whats the main difference we are gaining by sorting lucene docids and
: normal fields ?

when you sort on score (or some field) and ask for rows N - M, Lucene has 
to collect a priority queue of all the first M docs in order to give 
you back N-M.  This is where the deep paging issue can use up a lot of 
memory.

I *think* that when you sort or _docid_ the Sort code doesn't build up a 
priority queue of all M docs, it just knows it should ignore the first N 
docs it sees and start keping track of them after N until it finds a total 
of M-N.

But like i said before...

:   *sort* on _docid_ asc, which should make all inherient issues with deep
:   paging go away (as far as i know).  At no point with the internal lucene

...as far as i know this should work, but i'm not certain.  there may be 
other issues. (or i may be smocking crack that the sorting code works this 
way)


-Hoss

Re: FileDataSource baseDir to be solr.data.dir


: entity name=f processor=FileListEntityProcessor
: baseDir=/sites/ fileName=promotions.xml 
: 
: how do i set base dir to be solr.data.dir? on each server solr.data.dir is
: different. I use multi core solr instance

https://wiki.apache.org/solr/DataImportHandlerFaq#Is_it_possible_to_use_core_properties_inside_data-config_xml.3F


-Hoss

Re: getting data from only one database


: we created a solr which is connected to two databases and we created a
: jquery auto complete.in two databases we r having keywords and it is
: default search.so beside the search button we r ctearing more more
: drop down list and nmaed the two databases when the user click one one
: database and enter a search keyword we have to get the search results
: for the keyword from the selected database not fron two
: datbases..

if i understand your question, you are using DIH to index differnet types 
of documents -- some from database one, and some from database two
-- and in some cases (autocomplete?) you only want results from documents 
that were indexed from database one.

In that case -- you need to note which datbase the docs came from when you 
indexed them, which can be done by adding any field you wnat and recording 
any data you want in it about the source of the data, and then filtering 
on this at query time.

These two docs might be helpful...
https://wiki.apache.org/solr/MultipleIndexes#Flattening_Data_Into_a_Single_Index
https://wiki.apache.org/solr/DataImportHandlerFaq#How_would_I_insert_a_static_value_into_a_field_.3F


-Hoss

Re: More Like This on number fields


: For example, a document with the field numberOfParticipant at 10, i would
: like to have some similar documents with numberOfParticipant between 5 and
: 15.
: 
: Does this option exist ?

No ... MLT works purely on the basis of terms, so if you tried have 
MLT use a numeric field it would just find you docs that had the exact 
same value.

-Hoss

Re: How to send an OpenBitSet object from Solr server?


: I was not referring to Lucene's doc ids but the doc numbers (unique key)

Uh ... ok.  this is why i asked you waht you were planning on doing with 
the OpenBitSet -- it's just bits, indicating hte offsets of the documents 
in the total index.  having access to a copy of that on the lcient side 
isn't going to give you any info about the value of the uniqueKey field.

Like i said ... XY Problem ... please tell us what you overall goal is.  
why do you want the uniqueKey values for every matching doc in your remove 
client?  What would you do with the OpenBitSet from solr if you had a copy 
of it?

:  : We have a need to query and fetch millions of document ids from a Solr
:  3.3
:  : index and convert the same to a BitSet. To speed things up, we want to
:  : convert these document ids into OpenBitSet on the server side, put them
:  into
:  : the response object and read the same on the client side.
: 
:  This smells like an XY Problem ... what do you intend to do with this
:  BitSet on the client side?  the lucene doc ids are meaningless outside of
:  hte server, and for any given doc, the id could change from one request to
:  the next -- so how would having this data on the clinet be of any use to
:  you?
: 
:  https://people.apache.org/~hossman/#xyproblem
:  XY Problem
: 
:  Your question appears to be an XY Problem ... that is: you are dealing
:  with X, you are assuming Y will help you, and you are asking about Y
:  without giving more details about the X so that we can understand the
:  full issue.  Perhaps the best solution doesn't involve Y at all?
:  See Also: http://www.perlmonks.org/index.pl?node_id=542341


-Hoss

Re: [Q]Solr response passed to remote JsonStore - highlighting properties embed in the response part


: *What I want:* to change the output by embedding the highlighting properties
: into the response properties, such that the response part looks like:

Work along the lines of making this a generally available feature is 
already in progress on the trunk as part of the psuedo fields work 
(SOLR-2444) ... as i understand it from Ryan/Yonik, the idea would be to 
implement a highlight transformer...

https://issues.apache.org/jira/browse/SOLR-2444
https://wiki.apache.org/solr/CommonQueryParameters#Transformers:

...but i don't think anyone has taken a crack at it yet (i don't see any 
open issues specificly for this) so feel free to give it a shot.



-Hoss

Re: I can't pass the unit test when compile from apache-solr-3.3.0-src