HyukjinKwon commented on code in PR #47467:
URL: https://github.com/apache/spark/pull/47467#discussion_r1689063078
##########
mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala:
##########
@@ -581,21 +609,33 @@ private[ml] object DefaultParamsReader {
}
/**
- * Load metadata saved using [[DefaultParamsWriter.saveMetadata()]]
+ * Load metadata saved using [[DefaultParamsWriter.saveMetadataV2()]]
*
* @param expectedClassName If non empty, this is checked against the
loaded metadata.
* @throws IllegalArgumentException if expectedClassName is specified and
does not match metadata
*/
- def loadMetadata(path: String, sc: SparkContext, expectedClassName: String =
""): Metadata = {
+ @deprecated("use loadMetadataV2", "4.0.0")
+ def loadMetadata(path: String, sc: SparkContext, expectedClassName: String =
""): Metadata =
+ loadMetadataV2(path,
SparkSession.builder().sparkContext(sc).getOrCreate(), expectedClassName)
+
+ /**
+ * Load metadata saved using [[DefaultParamsWriter.saveMetadataV2()]]
+ *
+ * @param expectedClassName If non empty, this is checked against the
loaded metadata.
+ * @throws IllegalArgumentException if expectedClassName is specified and
does not match metadata
+ */
+ def loadMetadataV2(
Review Comment:
This should work:
```scala
def loadMetadata(path: String, sc: SparkContext, expectedClassName:
String): Metadata = { ... }
def loadMetadata(path: String, sc: SparkContext): Metadata =
loadMetadata(path, sc, "")
def loadMetadata(
path: String,
spark: SparkSession,
expectedClassName: String): Metadata = { ... }
def loadMetadata(
path: String,
spark: SparkSession): Metadata = { loadMetadata(path, spark, "") }
```
It's because of functions disallowed to be overwritten with the default
values. It will also not break binary compatibility too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
