Re: Compact Strings and APIs for fast decoding of string data

2016-02-11 Thread Paul Sandoz
Hi Peter, That’s a clever little wrapper, but it seems very particular. What if we could have (excuse the name): StringBuilder.toStringWithReset(); That would reset it’s own array to that as if by the default constructor. A copy is still required but… we could make this intrinsic (as is the

Re: Compact Strings and APIs for fast decoding of string data

2016-02-11 Thread Chris Vest
The OneShortStringBuilder looks nice. I would have to specify the Charset in its constructor, or somehow specify the capacity in bytes rather than characters, for it to optimally allocate the underlying byte array. It’s otherwise very similar, though probably more generally useful, to the builde

Re: Compact Strings and APIs for fast decoding of string data

2016-02-10 Thread Peter Levart
On 02/10/2016 02:16 PM, Chris Vest wrote: I’m slightly more worried about their array and what it implies, though. This API requires of me that I’m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks

Re: Compact Strings and APIs for fast decoding of string data

2016-02-10 Thread Peter Levart
Hi Chris and others, On 02/10/2016 02:16 PM, Chris Vest wrote: It’d be a shame to loose the no-zeroing optimisation, so if that requires a constructor, then lets find a constructor. To recap, I have possibly multiple buffers and I only want to use parts of each. So with the composed buffer I’

Re: Compact Strings and APIs for fast decoding of string data

2016-02-10 Thread Paul Sandoz
> On 10 Feb 2016, at 14:16, Chris Vest wrote: > > It’d be a shame to loose the no-zeroing optimisation, so if that requires a > constructor, then lets find a constructor. > > To recap, I have possibly multiple buffers and I only want to use parts of > each. So with the composed buffer I’d hav

Re: Compact Strings and APIs for fast decoding of string data

2016-02-10 Thread Chris Vest
It’d be a shame to loose the no-zeroing optimisation, so if that requires a constructor, then lets find a constructor. To recap, I have possibly multiple buffers and I only want to use parts of each. So with the composed buffer I’d have to first extract view buffers and then compose those, so I

Re: Compact Strings and APIs for fast decoding of string data

2016-02-10 Thread Remi Forax
openjdk.java.net > Envoyé: Mercredi 10 Février 2016 09:54:17 > Objet: Re: Compact Strings and APIs for fast decoding of string data > > Hi, > > A more functional approach would be to compose a sequence buffers into one > view, perhaps read-only. Then there would be no need to ac

Re: Compact Strings and APIs for fast decoding of string data

2016-02-10 Thread Paul Sandoz
Hi, A more functional approach would be to compose a sequence buffers into one view, perhaps read-only. Then there would be no need to accept arrays of buffers. That should work well for bulk operations. That’s a non-trivial but not very difficult amount of work, and possibly simplified if rest

Re: Compact Strings and APIs for fast decoding of string data

2016-02-09 Thread Xueming Shen
Hi Chris, I think basically you are asking a String constructor that takes a ByteBuffer. StringCoding then can take advantage of the current CompactString design to optimize the decoding operation by just a single byte[]/vectorized memory copy from the ByteBuffer to the String's internal byte

Compact Strings and APIs for fast decoding of string data

2016-02-09 Thread Chris Vest
Hi, Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string da