Srikanth Venkat created ATLAS-2694:
--------------------------------------
Summary: Avro schema typedef and support for Avro schema evolution
in Atlas
Key: ATLAS-2694
URL: https://issues.apache.org/jira/browse/ATLAS-2694
Project: Atlas
Issue Type: Improvement
Components: atlas-core
Reporter: Srikanth Venkat
Currently the base types in Atlas do not include Avro schemas. It would be nice
to add typedef for Avro schema and any associated metadata to support schema
evolution.
* For example, Avro_schema type supports:
** All avro types, both primitive and complex, including union types, as
fields of schema
** All types have doc strings and defaults
** A field of a schema can be another schema
** Indefinite nesting of records, arrays.
** Associated entities array attribute contains pointers to all datasets that
reflect the avro schema
** Fully expanded avroNotation for use in serDe
** Schema evolution features such as isLatest (Boolean) and version number
* Schema evolution Process
** Input: avro schema
** Output: new version of avro schema
** Compatibility: FULL, BACKWARD, FORWARD, NONE
** IsBreakingChange (Boolean): does the change produce an incompatible schema?
(ie its compatibility is not “FULL”)
* Atlas UI enhancement for JSON-valued attributes to support avro schema and
avro schema evolution
** Currently JSON-valued attributes are fully displayed in-line with other
attributes, not pretty-printed, cluttering the display. To support a better
display, we can display JSON-valued attributes in a one-line box that can be
scrolled down, or fully expanded with a mouse click that pretty-prints the
JSON.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
