[
https://issues.apache.org/jira/browse/AVRO-23?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735992#action_12735992
]
Hong Tang commented on AVRO-23:
-------------------------------
@doug, I am concerned that the avro's capability of supporting versioned
records is diminishing if we disallow schemas of the same name in a union. Here
is my hypothetical usage case:
# I have multiple data sources, each may contain a particular version of the
same record R. To be concrete, let's say I have two sources, S1 is a container
of V1 of R, and S2 is a container of V2 of R.
# I want to sort/merge the two containers by some key (outside R), which would
lead to the interleaving of records from S1 and S2.
# I could convert all records into a common schema, but the problem is that I
do not know which version is supposed to be the "best" one.
# Particularly, the program that does the merge may only care about the keys,
and not the versions of records in the containers. What it could do is to
create a new schema as union(R_v1, R_v2), and prefix the bytes coming from S1
with index 0, and bytes coming from S2 index 1.
# The consumer of this merged container must provide a version of R (Vx), and
performs normal conversion from either V1 to Vx, or V2 to Vx for each
individual record.
Like you said earlier, a workaround is to rename the schemas to different names
(and possibly rename back in step 5). But it seems messy. And what about
records nested inside? Do we also need to rename those too? Or does/will avro
support nested (static) scoping rules?
> Schema.toString() fails for a union of versioned records.
> ---------------------------------------------------------
>
> Key: AVRO-23
> URL: https://issues.apache.org/jira/browse/AVRO-23
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Hong Tang
>
> Schema.toString() fails to create a string representation for a union that
> contains versioned records.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
