[ https://issues.apache.org/jira/browse/LUCENE-4620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552668#comment-13552668 ]
Shai Erera commented on LUCENE-4620: ------------------------------------ I see. I have two comments about the patch. This part is wrong: {code} + int needed = upto - buf.offset; + if (values.length < needed) { + values.grow(needed); + } {code} should be {code} + if (values.ints.length < buf.length) { + values.grow(buf.length); + } {code} Does it even run for you? because {{values.length = 0}} at start. Also, note how this way you check offset < upto on every byte read while in the current code it's checked only once per integer read. Maybe if you do a while loop inside the loop, something like {{while (b < 0)}}. > Explore IntEncoder/Decoder bulk API > ----------------------------------- > > Key: LUCENE-4620 > URL: https://issues.apache.org/jira/browse/LUCENE-4620 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/facet > Reporter: Shai Erera > Assignee: Shai Erera > Fix For: 4.1, 5.0 > > Attachments: LUCENE-4620.patch, LUCENE-4620.patch, LUCENE-4620.patch, > LUCENE-4620.patch > > > Today, IntEncoder/Decoder offer a streaming API, where you can encode(int) > and decode(int). Originally, we believed that this layer can be useful for > other scenarios, but in practice it's used only for writing/reading the > category ordinals from payload/DV. > Therefore, Mike and I would like to explore a bulk API, something like > encode(IntsRef, BytesRef) and decode(BytesRef, IntsRef). Perhaps the Encoder > can still be streaming (as we don't know in advance how many ints will be > written), dunno. Will figure this out as we go. > One thing to check is whether the bulk API can work w/ e.g. facet > associations, which can write arbitrary byte[], and so may decoding to an > IntsRef won't make sense. This too we'll figure out as we go. I don't rule > out that associations will use a different bulk API. > At the end of the day, the requirement is for someone to be able to configure > how ordinals are written (i.e. different encoding schemes: VInt, PackedInts > etc.) and later read, with as little overhead as possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org