Re: Odd pattern of segment creation - branch_4x bug?

2012-12-04 Thread Shawn Heisey
On 12/3/2012 9:46 PM, Yonik Seeley wrote: On Mon, Dec 3, 2012 at 11:36 PM, Shawn Heisey s...@elyograg.org wrote: updateHandler class=solr.DirectUpdateHandler2 autoCommit maxDocs65536/maxDocs maxTime30/maxTime /autoCommit updateLog / /updateHandler Yeah, seems like that

Odd pattern of segment creation - branch_4x bug?

2012-12-03 Thread Shawn Heisey
I'm using Solr compiled from a branch_4x checkout. solr-impl4.1-SNAPSHOT 1416639M - ncindex - 2012-12-03 12:54:38 I've noticed something really odd happening during DIH full-import of millions of documents, and I'm wondering if it's a bug. Configbits that I think may be relevant are

Re: Odd pattern of segment creation - branch_4x bug?

2012-12-03 Thread Shawn Heisey
On 12/3/2012 2:39 PM, Shawn Heisey wrote: What's happening is that whenever it hits maxDocs, I get 2 segment files, one of them significantly smaller than the other. Rarely, it creates 3 segments! I know it's not a ramBuffer problem, because initially the exact same thing was happening with

Re: Odd pattern of segment creation - branch_4x bug?

2012-12-03 Thread Yonik Seeley
On Mon, Dec 3, 2012 at 9:59 PM, Shawn Heisey s...@elyograg.org wrote: As you can see, _nd is missing the fnm and nrm files and _ne is missing lots of files. When the next autocommit happened, _nf and _ng were created, and both of the segments listed above were completed. Ah, I think you're

Re: Odd pattern of segment creation - branch_4x bug?

2012-12-03 Thread Shawn Heisey
On 12/3/2012 8:44 PM, Yonik Seeley wrote: On Mon, Dec 3, 2012 at 9:59 PM, Shawn Heisey s...@elyograg.org wrote: As you can see, _nd is missing the fnm and nrm files and _ne is missing lots of files. When the next autocommit happened, _nf and _ng were created, and both of the segments listed

Re: Odd pattern of segment creation - branch_4x bug?

2012-12-03 Thread Yonik Seeley
On Mon, Dec 3, 2012 at 11:36 PM, Shawn Heisey s...@elyograg.org wrote: updateHandler class=solr.DirectUpdateHandler2 autoCommit maxDocs65536/maxDocs maxTime30/maxTime /autoCommit updateLog / /updateHandler Yeah, seems like that just should generate hard commits. Are you