Indexing performance with doc values (particularly with larger number of fields)

2014-03-23 Thread Alex at Ikanow
This might be more of a Lucene question, but a quick google didn't throw up anything. Has anyone done/seen any benchmarking on indexing performance (overhead) due to using doc values? I often index quite large JSON objects, with many fields (eg 50), I'm trying to get a feel for whether I can

Re: Indexing performance with doc values (particularly with larger number of fields)

2014-03-23 Thread Robert Muir
Would be a nice benchmark to run (and if you find hotspots/slow things to go improve in lucene...)! The data structures for docvalues are less complex than the data structures for the inverted index. I've enabled docvalues for many fields as you suggest in the past, and in my tests the time for