Hi Belén, You can log to a Neo4j graph database by setting the NEO4J_LOGGING_CONNECTION variable [1]. Additional metadata and lineage data can be added, but that will require additional (meta)data parsing.
This should already get you going, but there's always room for improvement. Feel free to create feature request tickets in our github repository [3] for any functionality you'd like to see in a future Apache Hop release. [1] https://hop.apache.org/manual/latest/technology/neo4j/index.html#_perspective_execution_lineage [2] https://www.know-bi.be/blog/data-lineage-with-neo4j-and-apache-hop [3] https://github.com/apache/hop/issues Regards, Bart On Thu, Jul 6, 2023 at 10:05 AM Belén Gracia <[email protected]> wrote: > Hi! > > I am just starting to use HOP although I have been using PDI (kettle) > for several years. There is something I have been missing and that is > being able to automate the lineage of tables by parsing the > jobs/transformations or transformations/pipelines. > > Is it possible to introduce customizable metadata in the steps? There > are many steps that would natively allow this automation, but getting > information about the tables read by other steps like "Table input" or > "Database join" is not always that easy. > > Do you others have this need, how have you solved it? > > I am all ears :) > > Best, > > Belén > >
