[ https://issues.apache.org/jira/browse/SPARK-37061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Russell Spitzer updated SPARK-37061: ------------------------------------ Description: Currently CustomMetrics uses `getCanonicalName` to get the metric type name https://github.com/apache/spark/blob/38493401d18d42a6cb176bf515536af97ba1338b/sql/core/src/main/scala/org/apache/spark/sql/execution/metric/CustomMetrics.scala#L31-L33 But when using reflection we need to use the original type name. Here is an example when working with an inner class {code:java title="CanonicalName vs Name"} Class.getName = org.apache.iceberg.spark.source.SparkBatchScan$FilesScannedMetric Class.getCanonicalName = org.apache.iceberg.spark.source.SparkBatchScan.FilesScannedMetric {code} The "$" name is required to look up this class while the "." version will fail with CNF. was: Currently CustomMetrics uses `getCanonicalName` to get the metric type name https://github.com/apache/spark/blob/38493401d18d42a6cb176bf515536af97ba1338b/sql/core/src/main/scala/org/apache/spark/sql/execution/metric/CustomMetrics.scala#L31-L33 But when using reflection we need to use the original type name. Here is an example when working with an inner class ``` Class.getName = org.apache.iceberg.spark.source.SparkBatchScan$FilesScannedMetric Class.getCanonicalName = org.apache.iceberg.spark.source.SparkBatchScan.FilesScannedMetric ``` The "$" name is required to look up this class while the "." version will fail with CNF. > Custom V2 Metrics uses wrong classname for lookup > ------------------------------------------------- > > Key: SPARK-37061 > URL: https://issues.apache.org/jira/browse/SPARK-37061 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.2.0 > Reporter: Russell Spitzer > Priority: Minor > > Currently CustomMetrics uses `getCanonicalName` to get the metric type name > https://github.com/apache/spark/blob/38493401d18d42a6cb176bf515536af97ba1338b/sql/core/src/main/scala/org/apache/spark/sql/execution/metric/CustomMetrics.scala#L31-L33 > But when using reflection we need to use the original type name. > Here is an example when working with an inner class > {code:java title="CanonicalName vs Name"} > Class.getName = > org.apache.iceberg.spark.source.SparkBatchScan$FilesScannedMetric > Class.getCanonicalName = > org.apache.iceberg.spark.source.SparkBatchScan.FilesScannedMetric > {code} > The "$" name is required to look up this class while the "." version will > fail with CNF. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org