[GitHub] [kyuubi] cfmcgrady commented on a diff in pull request #4710: [ARROW] LocalTableScanExec should not trigger job

Mon, 17 Apr 2023 23:39:01 -0700 


cfmcgrady commented on code in PR #4710:
URL: https://github.com/apache/kyuubi/pull/4710#discussion_r1169560427


##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/sql/kyuubi/SparkDatasetHelper.scala:
##########
@@ -51,6 +51,8 @@ object SparkDatasetHelper extends Logging {
       doCollectLimit(collectLimit)
     case collectLimit: CollectLimitExec if collectLimit.limit < 0 =>
       executeArrowBatchCollect(collectLimit.child)
+    case localTableScan: LocalTableScanExec =>
+      doLocalTableScan(localTableScan)

Review Comment:
   Spark also doesn't post the driver-side SQLMetrics, is that a Spark bug?
   
   the SQL UI will be changed once we post the metrics.
   
   Before:
   
   ![截屏2023-04-18 下午2 37 
50](https://user-images.githubusercontent.com/8537877/232692094-501e14e0-7797-4f81-8c82-eb091ce12c13.png)
   
   
   After:
   
   ![截屏2023-04-18 下午2 37 
04](https://user-images.githubusercontent.com/8537877/232692109-d0f8c412-9753-4b2d-a557-8b67a42e8e76.png)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to