Re: [akka-user] experiences with serialization: json vs protobuf vs avro vs kryo...

2017-05-02 Thread Guido Medina 
You might want to try Json Smile which is a Json binary, it is very 
effective and can favor of re-using Jackson ObjectMapper
https://github.com/FasterXML/jackson-dataformats-binary/tree/master/smile

I also use Kryo for mainly Java classes where schema evolution and 
compatibility is not important as all message classes are shared and 
cluster restarted fully.

HTH,

Guido.

On Monday, May 1, 2017 at 6:08:25 PM UTC+1, Alan Burlison wrote:
>
> On 01/05/2017 17:15, Justin du coeur wrote: 
>
> > My general impression is that protobuf is the well-established route, 
> and 
> > the one that most people are using 
>
> I wrote a wrapper around Akka-Spray to save JSON in flat files. It is 
> fairly primitive but has the (for my use case) big advantage that I can 
> examine and manipulate the persisted Actor data using other tools, which 
> is extremely useful during development. 
>
> -- 
> Alan Burlison 
> -- 
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] experiences with serialization: json vs protobuf vs avro vs kryo...

2017-05-02 Thread Joost de Vries 
Thanks for your experiences.

It sounds that protobuf is the most often used serialisation. And perhaps 
the need to write IDL is something where we'll have to bite the bullet. 
It's a bit of double book keeping but at least it should be easily 
testable

I think the standard serialisation for Lagom is json. I haven't seen 
information on the why of that decision anywhere. Would be interesting to 
know.

On Monday, May 1, 2017 at 7:08:25 PM UTC+2, Alan Burlison wrote:
>
> On 01/05/2017 17:15, Justin du coeur wrote: 
>
> > My general impression is that protobuf is the well-established route, 
> and 
> > the one that most people are using 
>
> I wrote a wrapper around Akka-Spray to save JSON in flat files. It is 
> fairly primitive but has the (for my use case) big advantage that I can 
> examine and manipulate the persisted Actor data using other tools, which 
> is extremely useful during development. 
>
> -- 
> Alan Burlison 
> -- 
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] experiences with serialization: json vs protobuf vs avro vs kryo...

2017-05-01 Thread Alan Burlison 

On 01/05/2017 17:15, Justin du coeur wrote:


My general impression is that protobuf is the well-established route, and
the one that most people are using


I wrote a wrapper around Akka-Spray to save JSON in flat files. It is 
fairly primitive but has the (for my use case) big advantage that I can 
examine and manipulate the persisted Actor data using other tools, which 
is extremely useful during development.


--
Alan Burlison
--

--

 Read the docs: http://akka.io/docs/
 Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
 Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka User List" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] experiences with serialization: json vs protobuf vs avro vs kryo...

2017-05-01 Thread Justin du coeur 
I'm using Kryo, and that's working reasonably well so far, but it's not
trivial to set up.  As I have time, I'm gradually writing that up
,
but it's only half-done.  Happy to answer questions if you want to explore
that route.

My general impression is that protobuf is the well-established route, and
the one that most people are using, but that does involve some duplication
of information.  (Or generating the case classes from the protobuf IDL.)  I
don't much care for that, which is why I went for Kryo; the end result is
broadly similar in terms of the amount of effort, I believe, but with the
serialization information encoded directly into the classes themselves.

On Mon, May 1, 2017 at 12:54 AM, Joost de Vries 
wrote:

> I'm trying to find a suitable serialization for akka persistence.
>
> There are a lot of options but not all of them are equally used and are as
> mature. F.i. we picked avro with avro4s but hit some problems that weren't
> easy to fix.
>
> Of course then it's tempting to fall back on json: it's familiar from
> rest, easy to analyse and there's stamina for schema evolution.
>
> But I wonder if there's another option. So my question is: which non json
> serialisation are you using and what are your experiences with schema
> evolution and otherwise?
> A special concern is that we'd rather not have to duplicate our case class
> structure in IDL files.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

Re: [akka-user] experiences with serialization: json vs protobuf vs avro vs kryo...

2017-05-01 Thread Richard Rodseth 
Because we're using Avro for Kafka, we also looked at avro and avro4s, but
ended up using Protobuf (with Maven plugin) for Akka Persistence. Mapping
between the IDL-generated classes and case classes is indeed unfortunate,
but no other issues so far.

On Mon, May 1, 2017 at 12:54 AM, Joost de Vries 
wrote:

> I'm trying to find a suitable serialization for akka persistence.
>
> There are a lot of options but not all of them are equally used and are as
> mature. F.i. we picked avro with avro4s but hit some problems that weren't
> easy to fix.
>
> Of course then it's tempting to fall back on json: it's familiar from
> rest, easy to analyse and there's stamina for schema evolution.
>
> But I wonder if there's another option. So my question is: which non json
> serialisation are you using and what are your experiences with schema
> evolution and otherwise?
> A special concern is that we'd rather not have to duplicate our case class
> structure in IDL files.
>
> --
> >> Read the docs: http://akka.io/docs/
> >> Check the FAQ: http://doc.akka.io/docs/akka/
> current/additional/faq.html
> >> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups
> "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akka-user+unsubscr...@googlegroups.com.
> To post to this group, send email to akka-user@googlegroups.com.
> Visit this group at https://groups.google.com/group/akka-user.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.

[akka-user] experiences with serialization: json vs protobuf vs avro vs kryo...

2017-05-01 Thread Joost de Vries 
I'm trying to find a suitable serialization for akka persistence. 

There are a lot of options but not all of them are equally used and are as 
mature. F.i. we picked avro with avro4s but hit some problems that weren't 
easy to fix.

Of course then it's tempting to fall back on json: it's familiar from rest, 
easy to analyse and there's stamina for schema evolution.

But I wonder if there's another option. So my question is: which non json 
serialisation are you using and what are your experiences with schema 
evolution and otherwise?
A special concern is that we'd rather not have to duplicate our case class 
structure in IDL files. 

-- 
>>  Read the docs: http://akka.io/docs/
>>  Check the FAQ: 
>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>  Search the archives: https://groups.google.com/group/akka-user
--- 
You received this message because you are subscribed to the Google Groups "Akka 
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.