Hi, kyuubi team! I want to send SQL lineage to Atlas and refered to bellow doc, but I can't get the sql lineage on atlas, Could you guide me if I missed something? SQL Lineage Support ― Apache Kyuubi<https://kyuubi.readthedocs.io/en/v1.9.0/extensions/engines/spark/lineage.html>
here is my spark-default.conf about kyuubi lineage support. # load kyuubi transitive dependencies spark.driver.extraClassPath=/home/bigdata/apache-kyuubi-1.9.0-source/extensions/spark/kyuubi-spark-lineage/target/scala-2.12/jars/* spark.executor.extraClassPath=/home/bigdata/apache-kyuubi-1.9.0-source/extensions/spark/kyuubi-spark-lineage/target/scala-2.12/jars/* # kyuubi atlas conf spark.atlas.rest.address=http://node01:21000 spark.atlas.client.type=rest spark.atlas.client.username=admin spark.atlas.client.password=admin spark.atlas.cluster.name=primary spark.atlas.hook.spark.column.lineage.enabled=true spark.kyuubi.plugin.lineage.dispatchers=ATLAS spark.kyuubi.plugin.lineage.skip.parsing.permanent.view.enabled=true spark.sql.queryExecutionListeners=org.apache.kyuubi.plugin.lineage.SparkOperationLineageQueryExecutionListener # spark-lineage_2.12-1.9.0.jar is alredy in $SPARK_HOME/jars [bigdata@node03 jars]$ ll /opt/bigdata/spark-3.5.1-bin-hadoop3/jars/kyuubi-spark-lineage_2.12-1.9.0.jar -rw-r-----. 1 bigdata bigdata 109494 4月 9 11:01 /opt/bigdata/spark-3.5.1-bin-hadoop3/jars/kyuubi-spark-lineage_2.12-1.9.0.jar I don't find any logs when I test insert query using bin/spark-sql. So I missed something configurations? here is version info: spark: 3.5.1 kyuubi:1.9.0 Thanks, zhaoxun.
