I need to go through approval process to release source, I wanted some feed back while that process in in progress. Let me know if you have any question or comments that I can address
- Anirudha Jadhav On Mon, Aug 31, 2015 at 4:09 PM, Prajwal Tuladhar <[email protected]> wrote: > Hi Anirudha, > > Is it possible to view the source of those projects? > > On Mon, Aug 31, 2015 at 7:46 PM, Anirudha Jadhav <[email protected]> wrote: > >> Hello All, >> >> I would like to introduce you all to a project which we have been working >> on using Avro and get some feedback. >> >> 1. AvroGraph >> ------------ >> We have created an avro to graphml serializer / deserializer. This allows >> us to visualize avro schemas in a graph to understand the relation between >> all the data points. This will later lead to creation of lineage graphs >> among other things >> - Implementation >> o similar to json serializer / deserializer >> o Apache Tinkerpop is used as a graph library and can be used to >> persist to a variety of graph stores. >> o support for scheme evolution between multiple version of the avro >> schemas >> o lot of unit tests and documentation >> >> 2. Avro Transformation Language >> ------------------------------- >> This is YAML based specification that will transform a data in a source >> schema to a target schema. For this we introduce a "transform node" to join >> the two schemas >> - The following operations can be done during the source to target data >> transformations >> o Copy source leaves to target leaves >> o Copy source parent nodes to target parent nodes, only if the sub >> graphs have the same structure. >> o Concatenate source nodes and copy to a target node >> o User-defined operations on the transforms >> o Extract certain leaves from the source and call an external end >> point for data manipulation eg. Spark / Http >> >> >> Let me know how/if these components would benefit the apache avro project >> and accordingly we would like to contribute it to the apache avro project. >> >> -Ani >> > > > > -- > -- > Cheers, > Praj > -- Anirudha P. Jadhav
