[
https://issues.apache.org/jira/browse/OOZIE-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101864#comment-13101864
]
Hadoop QA commented on OOZIE-346:
---------------------------------
topher-zicornell remarked:
Hi Mohammad,
You'd only need distinct deserialization routines per version, not
serialization. And they can be hierarchical, like an object hierarchy (version
3.3 extends 3.2 and just does the delta work needed).
I would prefer an attribute (key-value pair) table to blobs, but even if you
did that you'd probably want to do the versioning thing just to allow for the
algorithmic changes to the interpretation of the fields.
Using a number instead of a key in the attribute table would be dangerous. I'd
recommend against that. It makes the definitions a bit magical, like runes.
Somebody trying to work with the data would just have to Know what's what.
You could have a translation table of numbers to keys, but then you'd be
constantly doing a join to get the key names out. I'm not sure which would be
better for the DB, keys right in the table, or joins on all the queries?
. Topher
> GH-558: Serialization/deserialization of WorkflowInstance
> ---------------------------------------------------------
>
> Key: OOZIE-346
> URL: https://issues.apache.org/jira/browse/OOZIE-346
> Project: Oozie
> Issue Type: Bug
> Reporter: Hadoop QA
>
> Oozie team at yahoo has recently experienced multiple production issues when
> a new oozie version is upgraded attributed to the modifications of Workflow
> tables' structure.
> More specifically, we added a new field into workflow table. Hence, for
> example, if a user submits a WF job in earlier oozie version and if the job
> is still active after the upgrade, oozie fails to de-serialize the WFInstance
> object. In other words, the object was originally serialized using the old
> structure whereas oozie tries to deserailize using the new structures after
> the upgrade. Therefore it throws exception.
> Some observations that came up from our internal discussion:
> 1. Is it required to store the blob into table? Can't we create the the
> object from the other fields of the table? I know it might not be that
> straight forward. However, other options might be worse than this.
> 2. If we want to keep the blob, the new field(s) should be added at the end
> during serialization. However if some fields are removed, how could we handle
> that? Might not be a flexible idea.
> 3. During serialization, we could use some type of version at the beginning,
> that would help to de-serailize the object. This might make the coding very
> ugly depending on how many old versions we would like to support.
> 4. Since it is a very well-known problem, there should be some standard
> procedure. However they might not be easy too.
> Anyway these are just the initial thoughts. We didn't come up in any
> conclusion yet.
> Please feel free to make comment?
> Thanks,
> Mohammad
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
