Hello Jason! Sorry for not getting back to you earlier but I've just gotten back from vacation. I have switched projects since the versioned event data prototype was developed and in the current project we have taken a slightly different approach. Instead of saving the version number along with the event data, we store all event data in a JSON-format which allows us to evolve the event data classes by adding new data as long as it is optional (or has a default value). I still believe that saving the version number along side the event data has its merits and I will most probably make use of that method for coming projects (while still keeping the event data in JSON-format to allow out-of-band processing; I believe there is another thread on this list on that subject).
Greetings Odd On Tue, Apr 21, 2015 at 6:53 PM, Jason Law <[email protected]> wrote: > Odd, I just took a look at your code and I'm intrigued. I am very > interested in how in Akka Persistence we store events/snapshots over a long > period of time while the schema evolved, without creating a mess of code to > maintain. I've explored protobuf and Scalabuff, but it feels very > restrictive (I plan to use Maps). The versioning idea is one that interests > me. Any insights since your post last year? Did you abandon the versioning > idea in favor of a better one? If not, any chance this could make it into > the core akka codebase? > > Thanks. > Jason > > On Tuesday, May 20, 2014 at 2:06:45 AM UTC-6, Odd Möller wrote: >> >> Hi Martin! >> >> We haven't looked that close at protobuf yet (we don't use this for >> remoting purposes, only for persistence), and we needed something that was >> as simple as possible (i.e. departed from our regular "data expressed as >> case class"-approach as little as possible), and also was fully expressed >> at the class definition of our data classes (i.e. no extra definition files >> needed). It's probably mostly a mindset thing, I envision protobuf as a >> "protocol first"-kind of approach, but see this method as a more of a "code >> first"-approach. When the data to be made persistent is an instance of a >> case class, you also get the unapply implementation for your companion >> object for free, so you only need to provide the apply(Pair[Version, >> Product]) method to do the transformation from old to new. But I will >> definitely take closer look at protobuf and see what it offers (* >> reordering "read later"-list *). >> >> Greetings >> Odd >> >> >> On Tue, May 20, 2014 at 6:05 AM, Martin Krasser <[email protected]> >> wrote: >> >>> Hi Odd, >>> >>> why have you chosen this versioning approach instead of a protobuf based >>> one, for example? What are the advantages? >>> >>> >>> On 19.05.14 14:08, Odd Möller wrote: >>> >>> Hi! >>> >>> We have a versioning extension that we are experimenting with at the >>> moment (it is quite experimental at this point and completely without >>> tests): >>> https://github.com/odd/akka/tree/wip-persistence-odd/akka-contrib/src/main/scala/akka/contrib/persistence/versioning >>> >>> The basic idea is to attach a version to the actual data to be >>> persisted and then when saving the data, instead of serializing the data >>> class as is, save a tuple containing the data of the fields of the class. >>> Then at load time you can transform historic versions of the saved data to >>> the latest version (based on the attached version). >>> >>> For example: >>> >>> import akka.contrib.persistence.versioning._ >>> >>> // Version 1 -- present in the event log but no longer available at >>> runtime >>> // case class Name(first: String, last: String) extends Versioned[Name] >>> >>> // Version 2 -- current version of the class at runtime >>> case class Name(first: String, last: String, title: Option[String] = >>> None) extends Versioned[Name] >>> >>> object Name extends Versioned.Companion[Name] { >>> override def apply(data: (Version, Any)): Name = { >>> case (Version.Major(1), (first: String, last: String)) => >>> Name(first, last) >>> case (Version.Major(2), (first: String, last: String, title: >>> Option[String])) => Name(first, last, title) >>> case (v, d) => throw new IllegalStateException(s"Unknown version >>> encountered [version: $v, data: $d]") >>> } >>> } >>> >>> The versioned serializer needs to be present in the akka config: >>> >>> serializers { >>> versioned = "akka.contrib.persistence.versioning.VersionedSerializer" >>> } >>> serialization-bindings { >>> "akka.contrib.persistence.versioning.Versioned" = versioned >>> } >>> >>> >>> On Thu, May 15, 2014 at 7:41 PM, Per Johansson <[email protected]> >>> wrote: >>> >>>> What about using "versioned" objects (like Foo_v01, Foo_v02) and >>>> convert to the latest version when reading a snapshot? >>>> >>>> Anyone with experience of that approach? >>>> >>>> Was thinking it might be less work if you want to have e.g. maps after >>>> derserialization? (maps not supported in protobuf) >>>> >>>> /Per >>>> >>>> On Thursday, 8 May 2014 09:47:46 UTC+1, Martin Krasser wrote: >>>> >>>>> >>>>> On 08.05.14 10:43, Tim Pigden wrote: >>>>> >>>>> Hi >>>>> I've found a new thing to worry about :-) >>>>> If I use an eventsource model and content myself with the default >>>>> serialization and akka persistence, what's the likelihood of my serialized >>>>> objects becoming unreadable in the future? What's the long-term stability >>>>> of the default akka serialization mechanism? >>>>> >>>>> >>>>> By default, events are serialized with Akka's Java serializer. >>>>> >>>>> Is there a recommendation? >>>>> >>>>> >>>>> Use a custom >>>>> <http://doc.akka.io/docs/akka/2.3.2/scala/persistence.html#custom-serialization> >>>>> protobuf (or similar) serializer to serialize your events. This will >>>>> support backwards compatible changes to your event schema. >>>>> >>>>> Tim >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: >>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>> >>>>>>>>>> Search the archives: >>>>> https://groups.google.com/group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, >>>>> send an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> >>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>>> >>>>> -- >>>>> Martin Krasser >>>>> >>>>> blog: http://krasserm.blogspot.com >>>>> code: http://github.com/krasserm >>>>> twitter: http://twitter.com/mrt1nz >>>>> >>>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: >>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>> >>>>>>>>>> Search the archives: >>>> https://groups.google.com/group/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> -- >>> Martin Krasser >>> >>> blog: http://krasserm.blogspot.com >>> code: http://github.com/krasserm >>> twitter: http://twitter.com/mrt1nz >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
