[
https://issues.apache.org/jira/browse/AVRO-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754285#comment-16754285
]
Rumeshkrishnan commented on AVRO-2299:
--------------------------------------
[~cutting] Kindly find the use case we are using avro.
Use case: We have designed data lake system using Avro Schema based structure
definition for hive, BigQuery and MDW storage. Due to some restrictions in DB
side, we limited full compatibility feature as followed.
# versioning schema allowed with adding new fields (removing not allowed,
primitive type modification not allowed) must contain default property.
# we always maintain DB with latest version of schema structure.
# we have designed the system, can accept previous version of data, so new
fields value taken from default property.
# we have separate system maintain some DB field related property or GDPR
property for that field register using field prop options, this is getting
increase day by day. the schema props and field props we are using to define
non structure related and technical information for the system.
# we are expecting the schema should be clean without user property before
sending to another system maintain structure of DB tables. since we allow
versioning default value must for new version of same structure definition.
I hope you understand the my use case, if not let me know, I can elaborate.
> Get Plain Schema
> ----------------
>
> Key: AVRO-2299
> URL: https://issues.apache.org/jira/browse/AVRO-2299
> Project: Apache Avro
> Issue Type: New Feature
> Components: java
> Affects Versions: 1.8.2
> Reporter: Rumeshkrishnan
> Priority: Minor
> Labels: features
> Fix For: 1.9.0, 1.8.2, 1.8.3, 1.8.4
>
>
> {panel:title=Avro Schema Reserved Keys:}
> "doc", "fields", "items", "name", "namespace",
> "size", "symbols", "values", "type", "aliases", "default"
> {panel}
> AVRO also supports user defined properties for both Schema and Field.
> Is there way to get the schema with reserved property (key, value)?
> Input Schema:
> {code:java}
> {
> "name": "testSchema",
> "namespace": "com.avro",
> "type": "record",
> "fields": [
> {
> "name": "email",
> "type": "string",
> "doc": "email id",
> "user_field_prop": "xxxxx"
> }
> ],
> "user_schema_prop": "xxxxxx"
> }{code}
> Expected Plain Schema:
> {code:java}
> {
> "name": "testSchema",
> "namespace": "com.avro",
> "type": "record",
> "fields": [
> {
> "name": "email",
> "type": "string",
> "doc": "email id"
> }
> ]
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
