Oh, Steve, maybe you've touched off an arms race here ;). Heck with hack-a-thon an LR, let's all come with a suitcase full of parts and build machines!
On Thu, May 28, 2015 at 1:37 AM, Michael McCandless <[email protected]> wrote: > Wow this is a delightful machine: I am jealous! > > Do you have any sense of whether the higher IOPs/throughput of NVMe > SSDs vs "mere" SATA III matters for time to run Lucene's/Solr's tests? > > Also, how did you parallelize the running of the tests? Just the > normal top-level "ant test"? Or one "ant test" under lucene and one > under solr, running at once? > > Mike McCandless > > http://blog.mikemccandless.com > > > On Wed, May 27, 2015 at 9:29 AM, Steve Rowe <[email protected]> wrote: >> SSD: http://www.newegg.com/Product/Product.aspx?Item=N82E16820167299 >> CPU: http://www.newegg.com/Product/Product.aspx?Item=N82E16819117404 >> M/B: http://www.newegg.com/Product/Product.aspx?Item=N82E16813132518 >> RAM: http://www.newegg.com/Product/Product.aspx?Item=N82E16820231820 >> >> The mem wasn’t listed as supported by the mobo manufacturer, and it isn’t >> detected at its full speed (2800MHz), so currently running at 2400 >> (“overclocked” from detected 2100 I think). CPU isn’t overclocked from >> stock 3GHz, but I got a liquid cooler, thinking I’d experiment (haven’t much >> yet). Even without overclocking the fans spin faster when all the cores are >> busy, and it’s quite the little space heater. >> >> I installed Debian 8, but had to fix the installer in a couple places >> because it didn’t know about the new NVMe device naming scheme: >> >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785147 >> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=785149 >> >> I also upgraded to the 4.0 Linux kernel, since Debian 8 ships with the 3.16 >> kernel, and 3.19 contains a bunch of NVMe improvements. >> >> And I turned “swappiness" down to zero (thanks to Mike: >> <http://blog.mikemccandless.com/2011/04/just-say-no-to-swapping.html>) after >> seeing a bunch of test stalls while running the Lucene monster tests with 4 >> JVMs (takes about 2 hours, but I can get it down to 90 minutes or so by >> splitting the two tests in Test2BSortedDocValues out into their own suites - >> I’ll make an issue). >> >> Steve >> >>> On May 27, 2015, at 5:08 AM, Anshum Gupta <[email protected]> wrote: >>> >>> 8-real-core Haswell-E with 64G DDR4 memory and a NVMe 750-series SSD. >>> >>> Can run *all* of the Lucene and Solr tests in 10 minutes by running >>> multiple ant jobs in parallel! >>> >>> On Wed, May 27, 2015 at 1:17 AM, Ramkumar R. Aiyengar >>> <[email protected]> wrote: >>> Curious.. sarowe, what's the spec? >>> >>> On 26 May 2015 20:41, "Anshum Gupta" <[email protected]> wrote: >>> The last buch of fixes seems to have fixed this. The tests passed on a >>> Jenkins that had failing tests earlier. >>> Thanks Steve Rowe for lending the super-powerful machine that runs the >>> entire suite in 8 min! >>> >>> I've seen about 20 runs on that box and also runs on Policeman Jenkins with >>> no issues related to this test since the last commit so I've also >>> back-ported this to 5x as well. >>> >>> On Tue, May 26, 2015 at 9:19 AM, Chris Hostetter <[email protected]> >>> wrote: >>> >>> : Right, as I said, we weren't hitting this issue due to our Kerberos conf. >>> : file. i.e. the only thing that was different on our machines as compared >>> to >>> : everyone else and moving that conf file got the tests to fail for me. It >>> : now fails fairly consistently without the patch (from SOLR-7468) and has >>> : been looking good with the patch. >>> >>> that smells like the kind of thing that sould have some "assume sanity >>> checks" built into it. >>> >>> Given: >>> * the test setups a special env before running the test >>> * the test assumes the specific env will exist >>> * the user's machine may already have env properties set before running ant >>> that affect the expected special env >>> >>> therefore: before the test does the setup of the special env, it should >>> sanity check that the users basic env doesn't have any properties that >>> violate the "basic" situation. >>> >>> so, hypothetical example based on what little i understand the >>> authentication stuff: if the purpose of a test is to prove that some >>> requests work with (or w/o) kerberos authentication, then before doing any >>> setup of a "mock" kerberos env (or before setting up a "mock" situation >>> where no authentication is required), the test should verify that there >>> isn't already an existing kerberos env. (or if possible: "unset" whatever >>> env/properties define that env) >>> >>> >>> trivial example of a similar situation is the script engine tests -- >>> TestBadConfigs.testBogusScriptEngine: the purpose of the test is to >>> ensure that a solrconfig.xml file that refers to a script engine (by >>> name) which is not installed on the machine will produce an expeted error >>> at Solr init. before doing the Solr init, we have an whitebox assume that >>> asks the JVM directly if a script engine with that name already exists) >>> >>> >>> >>> -Hoss >>> http://www.lucidworks.com/ >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >>> >>> >>> -- >>> Anshum Gupta >>> >>> >>> >>> -- >>> Anshum Gupta >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
