[jira] [Updated] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barbara Eckman updated ATLAS-2694: -- Description: Currently the base types in Atlas do not include Avro schemas. It would be nice to add typedef for Avro schema and any associated metadata to support schema evolution. * For example, Avro_schema type supports: ** All avro types, both primitive and complex, including union types, as fields of schema ** All types have doc strings and defaults ** A field of a schema can be another schema ** Indefinite nesting of records, arrays. ** Associated entities array attribute contains pointers to all datasets that reflect the avro schema ** Fully expanded avroNotation for use in serDe ** Schema evolution features such as isLatest (Boolean) and version number * Schema evolution Process ** Input: avro schema ** Output: new version of avro schema ** Compatibility: FULL, BACKWARD, FORWARD, NONE ** IsBreakingChange (Boolean): does the change produce an incompatible schema? (ie its compatibility is not “FULL”) * was: Currently the base types in Atlas do not include Avro schemas. It would be nice to add typedef for Avro schema and any associated metadata to support schema evolution. * For example, Avro_schema type supports: ** All avro types, both primitive and complex, including union types, as fields of schema ** All types have doc strings and defaults ** A field of a schema can be another schema ** Indefinite nesting of records, arrays. ** Associated entities array attribute contains pointers to all datasets that reflect the avro schema ** Fully expanded avroNotation for use in serDe ** Schema evolution features such as isLatest (Boolean) and version number * Schema evolution Process ** Input: avro schema ** Output: new version of avro schema ** Compatibility: FULL, BACKWARD, FORWARD, NONE ** IsBreakingChange (Boolean): does the change produce an incompatible schema? (ie its compatibility is not “FULL”) * Atlas UI enhancement for JSON-valued attributes to support avro schema and avro schema evolution ** Currently JSON-valued attributes are fully displayed in-line with other attributes, not pretty-printed, cluttering the display. To support a better display, we can display JSON-valued attributes in a one-line box that can be scrolled down, or fully expanded with a mouse click that pretty-prints the JSON. Moving UI enhancement for avro schemas to a separate Jira > Avro schema typedef and support for Avro schema evolution in Atlas > --- > > Key: ATLAS-2694 > URL: https://issues.apache.org/jira/browse/ATLAS-2694 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Reporter: Srikanth Venkat >Assignee: Barbara Eckman >Priority: Critical > Fix For: 1.0.0 > > Attachments: ATLAS-2694-2.patch, ATLAS-2694.patch, > avro_atlas_types_08.json > > > Currently the base types in Atlas do not include Avro schemas. It would be > nice to add typedef for Avro schema and any associated metadata to support > schema evolution. > * For example, Avro_schema type supports: > ** All avro types, both primitive and complex, including union types, as > fields of schema > ** All types have doc strings and defaults > ** A field of a schema can be another schema > ** Indefinite nesting of records, arrays. > ** Associated entities array attribute contains pointers to all datasets > that reflect the avro schema > ** Fully expanded avroNotation for use in serDe > ** Schema evolution features such as isLatest (Boolean) and version number > * Schema evolution Process > ** Input: avro schema > ** Output: new version of avro schema > ** Compatibility: FULL, BACKWARD, FORWARD, NONE > ** IsBreakingChange (Boolean): does the change produce an incompatible > schema? (ie its compatibility is not “FULL”) > * -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj updated ATLAS-2694: Attachment: ATLAS-2694-2.patch > Avro schema typedef and support for Avro schema evolution in Atlas > --- > > Key: ATLAS-2694 > URL: https://issues.apache.org/jira/browse/ATLAS-2694 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Reporter: Srikanth Venkat >Assignee: Barbara Eckman >Priority: Critical > Fix For: 1.0.0 > > Attachments: ATLAS-2694-2.patch, ATLAS-2694.patch, > avro_atlas_types_08.json > > > Currently the base types in Atlas do not include Avro schemas. It would be > nice to add typedef for Avro schema and any associated metadata to support > schema evolution. > * For example, Avro_schema type supports: > ** All avro types, both primitive and complex, including union types, as > fields of schema > ** All types have doc strings and defaults > ** A field of a schema can be another schema > ** Indefinite nesting of records, arrays. > ** Associated entities array attribute contains pointers to all datasets > that reflect the avro schema > ** Fully expanded avroNotation for use in serDe > ** Schema evolution features such as isLatest (Boolean) and version number > * Schema evolution Process > ** Input: avro schema > ** Output: new version of avro schema > ** Compatibility: FULL, BACKWARD, FORWARD, NONE > ** IsBreakingChange (Boolean): does the change produce an incompatible > schema? (ie its compatibility is not “FULL”) > * Atlas UI enhancement for JSON-valued attributes to support avro schema and > avro schema evolution > ** Currently JSON-valued attributes are fully displayed in-line with other > attributes, not pretty-printed, cluttering the display. To support a better > display, we can display JSON-valued attributes in a one-line box that can be > scrolled down, or fully expanded with a mouse click that pretty-prints the > JSON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Madhan Neethiraj updated ATLAS-2694: Attachment: ATLAS-2694.patch > Avro schema typedef and support for Avro schema evolution in Atlas > --- > > Key: ATLAS-2694 > URL: https://issues.apache.org/jira/browse/ATLAS-2694 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Reporter: Srikanth Venkat >Assignee: Barbara Eckman >Priority: Critical > Attachments: ATLAS-2694.patch, avro_atlas_types_08.json > > > Currently the base types in Atlas do not include Avro schemas. It would be > nice to add typedef for Avro schema and any associated metadata to support > schema evolution. > * For example, Avro_schema type supports: > ** All avro types, both primitive and complex, including union types, as > fields of schema > ** All types have doc strings and defaults > ** A field of a schema can be another schema > ** Indefinite nesting of records, arrays. > ** Associated entities array attribute contains pointers to all datasets > that reflect the avro schema > ** Fully expanded avroNotation for use in serDe > ** Schema evolution features such as isLatest (Boolean) and version number > * Schema evolution Process > ** Input: avro schema > ** Output: new version of avro schema > ** Compatibility: FULL, BACKWARD, FORWARD, NONE > ** IsBreakingChange (Boolean): does the change produce an incompatible > schema? (ie its compatibility is not “FULL”) > * Atlas UI enhancement for JSON-valued attributes to support avro schema and > avro schema evolution > ** Currently JSON-valued attributes are fully displayed in-line with other > attributes, not pretty-printed, cluttering the display. To support a better > display, we can display JSON-valued attributes in a one-line box that can be > scrolled down, or fully expanded with a mouse click that pretty-prints the > JSON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Barbara Eckman updated ATLAS-2694: -- Attachment: avro_atlas_types_08.json > Avro schema typedef and support for Avro schema evolution in Atlas > --- > > Key: ATLAS-2694 > URL: https://issues.apache.org/jira/browse/ATLAS-2694 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Reporter: Srikanth Venkat >Assignee: Barbara Eckman >Priority: Critical > Attachments: avro_atlas_types_08.json > > > Currently the base types in Atlas do not include Avro schemas. It would be > nice to add typedef for Avro schema and any associated metadata to support > schema evolution. > * For example, Avro_schema type supports: > ** All avro types, both primitive and complex, including union types, as > fields of schema > ** All types have doc strings and defaults > ** A field of a schema can be another schema > ** Indefinite nesting of records, arrays. > ** Associated entities array attribute contains pointers to all datasets > that reflect the avro schema > ** Fully expanded avroNotation for use in serDe > ** Schema evolution features such as isLatest (Boolean) and version number > * Schema evolution Process > ** Input: avro schema > ** Output: new version of avro schema > ** Compatibility: FULL, BACKWARD, FORWARD, NONE > ** IsBreakingChange (Boolean): does the change produce an incompatible > schema? (ie its compatibility is not “FULL”) > * Atlas UI enhancement for JSON-valued attributes to support avro schema and > avro schema evolution > ** Currently JSON-valued attributes are fully displayed in-line with other > attributes, not pretty-printed, cluttering the display. To support a better > display, we can display JSON-valued attributes in a one-line box that can be > scrolled down, or fully expanded with a mouse click that pretty-prints the > JSON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ATLAS-2694) Avro schema typedef and support for Avro schema evolution in Atlas
[ https://issues.apache.org/jira/browse/ATLAS-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Venkat updated ATLAS-2694: --- Issue Type: New Feature (was: Improvement) > Avro schema typedef and support for Avro schema evolution in Atlas > --- > > Key: ATLAS-2694 > URL: https://issues.apache.org/jira/browse/ATLAS-2694 > Project: Atlas > Issue Type: New Feature > Components: atlas-core >Reporter: Srikanth Venkat >Priority: Critical > > Currently the base types in Atlas do not include Avro schemas. It would be > nice to add typedef for Avro schema and any associated metadata to support > schema evolution. > * For example, Avro_schema type supports: > ** All avro types, both primitive and complex, including union types, as > fields of schema > ** All types have doc strings and defaults > ** A field of a schema can be another schema > ** Indefinite nesting of records, arrays. > ** Associated entities array attribute contains pointers to all datasets > that reflect the avro schema > ** Fully expanded avroNotation for use in serDe > ** Schema evolution features such as isLatest (Boolean) and version number > * Schema evolution Process > ** Input: avro schema > ** Output: new version of avro schema > ** Compatibility: FULL, BACKWARD, FORWARD, NONE > ** IsBreakingChange (Boolean): does the change produce an incompatible > schema? (ie its compatibility is not “FULL”) > * Atlas UI enhancement for JSON-valued attributes to support avro schema and > avro schema evolution > ** Currently JSON-valued attributes are fully displayed in-line with other > attributes, not pretty-printed, cluttering the display. To support a better > display, we can display JSON-valued attributes in a one-line box that can be > scrolled down, or fully expanded with a mouse click that pretty-prints the > JSON. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
