HyukjinKwon commented on code in PR #44375:
URL: https://github.com/apache/spark/pull/44375#discussion_r1428616539
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ApplyInPandasWithStatePythonRunner.scala:
##########
@@ -61,14 +61,12 @@ class ApplyInPandasWithStatePythonRunner(
keySchema: StructType,
outputSchema: StructType,
stateValueSchema: StructType,
- pyMetrics: Map[String, SQLMetric],
+ override val pythonMetrics: Map[String, SQLMetric],
jobArtifactUUID: Option[String])
extends BasePythonRunner[InType, OutType](funcs, evalType, argOffsets,
jobArtifactUUID)
with PythonArrowInput[InType]
with PythonArrowOutput[OutType] {
- override val pythonMetrics: Option[Map[String, SQLMetric]] = Some(pyMetrics)
Review Comment:
All those changes are actually some revert of
https://github.com/apache/spark/pull/44305 because we now support metrics, and
no need to make it optional anymore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
