LuciferYang commented on PR #35149: URL: https://github.com/apache/spark/pull/35149#issuecomment-1232389422
> So the problem is we have confusing error messages in the log? The query itself is fine, right? @cloud-fan I think more about this, there may another possible problems: For the above demo program, if there are other jobs to be executed after `spark.sql(sql).collect()`, the new jobs cannot be run with sufficient concurrency due to resource not released by stage 31 and 36 in time. That is to say, a completed SQL will affect the running efficiency of the next SQL. This will be displayed when using databricks spark-sql-pref to run TPCDS power test with all sqls. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
