On Thu, Nov 1, 2012 at 6:06 PM, Christoph Engelbert <[email protected]>wrote:
> Am 01.11.2012 17:48, schrieb Roman Levenstein: > > Hi, > > > > On Thu, Nov 1, 2012 at 4:45 PM, Tatu Saloranta <[email protected]> > wrote: > >> On Thu, Nov 1, 2012 at 8:37 AM, Christoph Engelbert > >> <[email protected]> wrote: > >>> Am 01.11.2012 16:35, schrieb Tatu Saloranta: > >>>> On Thu, Nov 1, 2012 at 2:53 AM, Christoph Engelbert > >>>> <[email protected]> wrote: > >>>>> Am 30.10.2012 18:01, schrieb Tatu Saloranta: > >>>>>> You may want to see how serializer/deserializer in > >>>>>> > >>>>>> https://github.com/eishay/jvm-serializers > >>>>>> > >>>>>> is done -- Martin wrote it (or at least modified & has maintained > it), > >>>>>> so it should be along best practices. > >>>>>> > >>>>>> -+ Tatu +- > >>>>> This is only best practice for non concurrent access. The unittests > >>>>> are executed one by one so the implementation using one byte array > >>>>> is not a problem but this cannot be threadsafe. > >>>>> > >>>>> This applies to this one as well. Never use this implementation in > >>>>> concurrent environments :-) > >>>>> > https://github.com/eishay/jvm-serializers/blob/kannan/tpc/src/serializers/Kryo.java > >>>> Correct. What I meant was it was right way to do it for the use case > >>>> in question. Which granted requires one to understand the use case... > >>>> I forgot that this particular test does not do multi-threaded > >>>> processing like others do. > >>>> > >>>> It actually is one of the things for jvm-serializers to consider; > >>>> efficient reuse techniques vary a lot between codecs. > >>>> > >>>> -+ Tatu +- > >>> I guess there's still the problem about the kryo instance itself. On > >>> googlecode it is marked that the kryo instance itself is not > >>> threadsafe. Anyone has some more informations about that? Tatu? :-) > >> No, I have only tried to use it without success once or twice, outside > >> of jvm-serializers. > >> My weapon of choice if Smile+Jakcson. :-D > >> > >> But Kryo author (Nate something?) should know -- I can try to dig > >> contact info, point him to this mailing list. > >> > >> -+ Tatu +- > > I'm one of Kryo's contributors. > > Kryo instances do keep some context information during serialization > > and deserialization. Therefore, the same instance cannot be safely > > used at the same time with multiple input or output streams for > > performing simultaneous (de) serializations. And using the same > > instance with multiple threads can result in even more problems. > > > > Therefore, when I need to use Kryo in multi-threaded environment, I > > usually make sure that each thread gets its own instance of Kryo at > > least for the duration of the operation. The reduce the cost of Kryo > > instances creation I usually use a pool of such instances. Every time > > a thread needs to perform a (de) serialization operation, it would get > > an instance from the pool, perform the operation and return the > > instance back to the pool. Alternatively, one could use ThreadLocal > > Kryo instances if applicable. > > > > -Roman > > Ok that is a nice information. So I would suggest to use > concurrencylevel to instantiate a bunch of kryo instances and queue > them. If non of the preinstantiated kryo instances is available than > the (de-)serialization must wait until one's free again (but that > would mean the concurrencylevel is choosen to low ;-)). > > Or you grow the pool on demand. You may also shrink it afterwards based on the concurrency level or any other kind of metric, e.g. you could remove a Kryo instance from the pool if it is not used for a certain time period . Cheers, Roman
