Re: Measuring SOLR performance

2013-09-04 Thread Dmitry Kan
Hi Roman, Ok, I will. Thanks! Cheers, Dmitry On Tue, Sep 3, 2013 at 4:46 PM, Roman Chyla roman.ch...@gmail.com wrote: Hi Dmitry, Thanks for the feedback. Yes, it is indeed jmeter issue (or rather, the issue of the plugin we use to generate charts). You may want to use the github for

Re: Measuring SOLR performance

2013-09-03 Thread Dmitry Kan
Hi Roman, Thanks, the --additionalSolrParams was just what I wanted and works fine. BTW, if you have some special bug tracking forum for the tool, I'm happy to submit questions / bug reports there. Otherwise, this email list is ok (for me at least). One other thing I have noticed in the err

Re: Measuring SOLR performance

2013-09-03 Thread Roman Chyla
Hi Dmitry, Thanks for the feedback. Yes, it is indeed jmeter issue (or rather, the issue of the plugin we use to generate charts). You may want to use the github for whatever comes next https://github.com/romanchyla/solrjmeter/issues Cheers, roman On Tue, Sep 3, 2013 at 7:54 AM, Dmitry Kan

Re: Measuring SOLR performance

2013-09-02 Thread Dmitry Kan
Hi Roman, What's the format for running the facet+filter queries? Would something like this work: field:foo =50 fq=other_field:bar facet=true facet.field=facet_field_name Thanks, Dmitry On Fri, Aug 23, 2013 at 2:34 PM, Dmitry Kan solrexp...@gmail.com wrote: Hi Roman, With

Re: Measuring SOLR performance

2013-09-02 Thread Erick Erickson
filter and facet queries can be freely intermixed, it's not a problem. What problem are you seeing when you try this? Best, Erick On Mon, Sep 2, 2013 at 7:46 AM, Dmitry Kan solrexp...@gmail.com wrote: Hi Roman, What's the format for running the facet+filter queries? Would something like

Re: Measuring SOLR performance

2013-09-02 Thread Dmitry Kan
Hi Erick, Agree, this is perfectly fine to mix them in solr. But my question is about solrjmeter input query format. Just couldn't find a suitable example on the solrjmeter's github. Dmitry On Mon, Sep 2, 2013 at 5:40 PM, Erick Erickson erickerick...@gmail.comwrote: filter and facet queries

Re: Measuring SOLR performance

2013-09-02 Thread Roman Chyla
Hi Dmitry, If it is something you want to pass with every request (which is my use case), you can pass it as additional solr params, eg. python solrjmeter --additionalSolrParams=fq=other_field:bar+facet=true+facet.field=facet_field_name the string should be url encoded. If it is something

Re: Measuring SOLR performance

2013-08-23 Thread Dmitry Kan
Hi Roman, With adminPath=/admin or adminPath=/admin/cores, no. Interestingly enough, though, I can access http://localhost:8983/solr/statements/admin/system But I can access http://localhost:8983/solr/admin/cores, only when with adminPath=/admin/cores (which suggests that this is the right value

Re: Measuring SOLR performance

2013-08-22 Thread Roman Chyla
Hi Dmitry, So it seems solrjmeter should not assume the adminPath - and perhaps needs to be passed as an argument. When you set the adminPath, are you able to access localhost:8983/solr/statements/admin/cores ? roman On Wed, Aug 21, 2013 at 7:36 AM, Dmitry Kan solrexp...@gmail.com wrote: Hi

Re: Measuring SOLR performance

2013-08-21 Thread Dmitry Kan
Hi Roman, I have noticed a difference with different solr.xml config contents. It is probably legit, but thought to let you know (tests run on fresh checkout as of today). As mentioned before, I have two cores configured in solr.xml. If the file is: [code] solr persistent=false !--

Re: Measuring SOLR performance

2013-08-14 Thread Dmitry Kan
Hi Roman, This looks much better, thanks! The ordinary non-comarison mode works. I'll post here, if there are other findings. Thanks for quick turnarounds, Dmitry On Wed, Aug 14, 2013 at 1:32 AM, Roman Chyla roman.ch...@gmail.com wrote: Hi Dmitry, oh yes, late night fixes... :) The latest

Re: Measuring SOLR performance

2013-08-13 Thread Dmitry Kan
Hi Roman, Something bad happened in fresh checkout: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R cms -t /solr/statements -e statements -U 100 Traceback (most recent call last): File solrjmeter.py, line 1392,

Re: Measuring SOLR performance

2013-08-13 Thread Roman Chyla
Hi Dmitry, oh yes, late night fixes... :) The latest commit should make it work for you. Thanks! roman On Tue, Aug 13, 2013 at 3:37 AM, Dmitry Kan solrexp...@gmail.com wrote: Hi Roman, Something bad happened in fresh checkout: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q

Re: Measuring SOLR performance

2013-08-12 Thread Dmitry Kan
Hi Roman, Good point. I managed to run the command with -C and double quotes: python solrjmeter.py -a -C g1,cms -c hour -x ./jmx/SolrQueryTest.jmx As a result got several files (html, css, js, csv) in the running directory (any way to specify where the output should be stored in this case?)

Re: Measuring SOLR performance

2013-08-12 Thread Roman Chyla
Hi Dmitry, On Mon, Aug 12, 2013 at 9:36 AM, Dmitry Kan solrexp...@gmail.com wrote: Hi Roman, Good point. I managed to run the command with -C and double quotes: python solrjmeter.py -a -C g1,cms -c hour -x ./jmx/SolrQueryTest.jmx As a result got several files (html, css, js, csv) in the

Re: Measuring SOLR performance

2013-08-07 Thread Dmitry Kan
Hi Roman, Finally, this has worked! Thanks for quick support. The graphs look awesome. At least on the index sample :) It is quite easy to setup and run + possible to run directly on the shard server in background mode. my test run was: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q

Re: Measuring SOLR performance

2013-08-07 Thread Dmitry Kan
Hi Roman, One more question. I tried to compare different runs (g1 vs cms) using the command below, but get an error. Should I attach some other param(s)? python solrjmeter.py -C g1,foo -c hour -x ./jmx/SolrQueryTest.jmx **ERROR** File solrjmeter.py, line 1427, in module main(sys.argv)

Re: Measuring SOLR performance

2013-08-07 Thread Roman Chyla
Hi Dmitry, The command seems good. Are you sure your shell is not doing something funny with the params? You could try: python solrjmeter.py -C g1,foo -c hour -x ./jmx/SolrQueryTest.jmx -a where g1 and foo are results of the individual runs, ie. something that was started and saved with '-R g1'

Re: Measuring SOLR performance

2013-08-06 Thread Dmitry Kan
Hi Roman, With fresh checkout, the reported admin_endpoint is: http://localhost:8983/solr/admin. This url redirects to http://localhost:8983/solr/#/ . I'm using solr 4.3.1. Is your tool supporting this version? Of three URLs you asked for, only the 3rd one gave response:

Re: Measuring SOLR performance

2013-08-06 Thread Shawn Heisey
On 8/6/2013 6:17 AM, Dmitry Kan wrote: Of three URLs you asked for, only the 3rd one gave response: snip The rest report 404. On Mon, Aug 5, 2013 at 8:38 PM, Roman Chyla roman.ch...@gmail.com wrote: Hi Dmitry, So I think the admin pages are different on your version of solr, what do you

Re: Measuring SOLR performance

2013-08-06 Thread Dmitry Kan
Hi, Thanks for the clarification, Shawn! So with this in mind, the following work: http://localhost:8983/solr/statements/admin/system?wt=json http://localhost:8983/solr/statements/admin/mbeans?wt=json not copying their output to save space. Roman: is this something that should be set via -t

Re: Measuring SOLR performance

2013-08-06 Thread Roman Chyla
Hi Dmitry, I've modified the solrjmeter to retrieve data from under the core (the -t parameter) and the rest from the /solr/admin - I could test it only against 4.0, but it is there the same as 4.3 - it seems...so you can try the fresh checkout my test was: python solrjmeter.py -a -x

Re: Measuring SOLR performance

2013-08-05 Thread Dmitry Kan
Hi Roman, No problem. Still trying to launch the thing.. The query with the added -t parameter generated an error: 1. python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q ./queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R test -t /solr/statements [passed relative path

Re: Measuring SOLR performance

2013-08-05 Thread Roman Chyla
Hi Dmitry, So I think the admin pages are different on your version of solr, what do you see when you request... ? http://localhost:8983/solr/admin/system?wt=json http://localhost:8983/solr/admin/mbeans?wt=json http://localhost:8983/solr/admin/cores?wt=json If your core -t was

Re: Measuring SOLR performance

2013-08-02 Thread Erick Erickson
: On 8/1/2013 2:08 PM, Roman Chyla wrote: Hi, here is a short post describing the results of the yesterday run with added parameters as per Shawn's recommendation, have fun getting confused ;) http://29min.wordpress.com/**2013/08/01/measuring-solr-**performance-ii/ http

Re: Measuring SOLR performance

2013-08-02 Thread Dmitry Kan
Hi Roman, Sure: python solrjmeter.py -a -x ./jmx/SolrQueryTest.jmx -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries -s localhost -p 8983 -a --durationInSecs 60 -R test This is vanilla install (git clone) except for one change that I had to do related to solr cores: git diff

Re: Measuring SOLR performance

2013-08-02 Thread Roman Chyla
Hi Dmitry, Thanks, It was a toothing problem, fixed now, please try the fresh checkout AND add the following to your arguments: -t /solr/core1 that sets the path under which solr should be contacted, the handler is set in the jmeter configuration, so if you were using different query handlers

Re: Measuring SOLR performance

2013-08-01 Thread Bernd Fehling
Yes, UseNuma is only for Parallel Scavenger garbage collector and only for Solaris 9 and higher and Linux kernel 2.6.19 and glibc 2.6.1. And it performs with 64-bit better than 32-bit. So no effects for G1. With standard applications CMS is very slightly better than G1 but when it comes to huge

Re: Measuring SOLR performance

2013-08-01 Thread Dmitry Kan
Hi Roman, When I try to run with -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries here what is reported: Traceback (most recent call last): File solrjmeter.py, line 1390, in module main(sys.argv) File solrjmeter.py, line 1309, in main tests = find_tests(options)

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
Dmitry, Can you post the entire invocation line? roman On Thu, Aug 1, 2013 at 7:46 AM, Dmitry Kan solrexp...@gmail.com wrote: Hi Roman, When I try to run with -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries here what is reported: Traceback (most recent call last):

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
Hi Bernd, On Thu, Aug 1, 2013 at 4:07 AM, Bernd Fehling bernd.fehl...@uni-bielefeld.de wrote: Yes, UseNuma is only for Parallel Scavenger garbage collector and only for Solaris 9 and higher and Linux kernel 2.6.19 and glibc 2.6.1. And it performs with 64-bit better than 32-bit. So no

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
Hi, here is a short post describing the results of the yesterday run with added parameters as per Shawn's recommendation, have fun getting confused ;) http://29min.wordpress.com/2013/08/01/measuring-solr-performance-ii/ roman On Wed, Jul 31, 2013 at 12:32 PM, Roman Chyla roman.ch...@gmail.com

Re: Measuring SOLR performance

2013-08-01 Thread Shawn Heisey
On 8/1/2013 2:08 PM, Roman Chyla wrote: Hi, here is a short post describing the results of the yesterday run with added parameters as per Shawn's recommendation, have fun getting confused ;) http://29min.wordpress.com/2013/08/01/measuring-solr-performance-ii/ I am having a very difficult time

Re: Measuring SOLR performance

2013-08-01 Thread Roman Chyla
/**2013/08/01/measuring-solr-**performance-ii/http://29min.wordpress.com/2013/08/01/measuring-solr-performance-ii/ I am having a very difficult time with the graphs. I have no idea what I'm looking at. The graphs are probably self-explanatory to you, because you created them and you've been

Re: Measuring SOLR performance

2013-07-31 Thread Dmitry Kan
Hi Roman, What version and config of SOLR does the tool expect? Tried to run, but got: **ERROR** File solrjmeter.py, line 1390, in module main(sys.argv) File solrjmeter.py, line 1296, in main check_prerequisities(options) File solrjmeter.py, line 351, in check_prerequisities

Re: Measuring SOLR performance

2013-07-31 Thread Dmitry Kan
Ok, got the error fixed by modifying the base solr ulr in solrjmeter.py (added core name after /solr part). Next error is: WARNING: no test name(s) supplied nor found in: ['/home/dmitry/projects/lab/solrjmeter/demo/queries/demo.queries'] It is a 'slow start with new tool' symptom I guess.. :)

Re: Measuring SOLR performance

2013-07-31 Thread Shawn Heisey
On 7/31/2013 12:24 AM, William Bell wrote: But that link does not tell me which on you are using? You are listing like 4 versions on your site. Also, what did it fix? Pause times? Any other words of wisdom ? I'm not sure whether that was directed at me or Roman, but here's my answers:

Re: Measuring SOLR performance

2013-07-31 Thread Roman Chyla
Hi Dmitry, probably mistake in the readme, try calling it with -q /home/dmitry/projects/lab/solrjmeter/queries/demo/demo.queries as for the base_url, i was testing it on solr4.0, where it tries contactin /solr/admin/system - is it different for 4.3? I guess I should make it configurable (it

Re: Measuring SOLR performance

2013-07-31 Thread Roman Chyla
I'll try to run it with the new parameters and let you know how it goes. I've rechecked details for the G1 (default) garbage collector run and I can confirm that 2 out of 3 runs were showing high max response times, in some cases even 10secs, but the customized G1 never - so definitely the

RE: Measuring SOLR performance

2013-07-31 Thread Markus Jelsma
is obviously higher with G1GC. -Original message- From:Roman Chyla roman.ch...@gmail.com Sent: Wednesday 31st July 2013 18:32 To: solr-user@lucene.apache.org Subject: Re: Measuring SOLR performance I'll try to run it with the new parameters and let you know how it goes. I've

Re: Measuring SOLR performance

2013-07-31 Thread Roman Chyla
with G1GC. -Original message- From:Roman Chyla roman.ch...@gmail.com Sent: Wednesday 31st July 2013 18:32 To: solr-user@lucene.apache.org Subject: Re: Measuring SOLR performance I'll try to run it with the new parameters and let you know how it goes. I've rechecked details

Measuring SOLR performance

2013-07-30 Thread Roman Chyla
Hello, I have been wanting some tools for measuring performance of SOLR, similar to Mike McCandles' lucene benchmark. so yet another monitor was born, is described here: http://29min.wordpress.com/2013/07/31/measuring-solr-query-performance/ I tested it on the problem of garbage collectors (see

Re: Measuring SOLR performance

2013-07-30 Thread Shawn Heisey
On 7/30/2013 6:59 PM, Roman Chyla wrote: I have been wanting some tools for measuring performance of SOLR, similar to Mike McCandles' lucene benchmark. so yet another monitor was born, is described here: http://29min.wordpress.com/2013/07/31/measuring-solr-query-performance/ I tested it