xc-jianghan commented on issue #4814: URL: https://github.com/apache/kyuubi/issues/4814#issuecomment-3863995750
Hello, I’m a student studying big data, and recently I’ve been researching the integration of Apache Atlas and Spark lineage. I noticed that the Kyuubi extension you contributed can send Spark lineage to Atlas. However, in my usage, when I execute a single SQL statement using spark-sql -e, I encounter the following error: WARN AtlasLineageDispatcher: Send lineage to atlas failed. java.lang.IllegalStateException: Shutdown in progress, cannot add a shutdownHook As a result, the lineage data cannot be uploaded to Atlas. But if I run multiple SQL statements, it seems to “warm up” the Atlas client, and the error does not occur. Do you know how I could optimize this so that lineage can be sent successfully even when running only a single SQL statement? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
