panbingkun commented on code in PR #45867:
URL: https://github.com/apache/spark/pull/45867#discussion_r1550930932
##########
common/utils/src/test/scala/org/apache/spark/util/LogKeySuite.scala:
##########
@@ -25,9 +32,59 @@ class LogKeySuite
extends AnyFunSuite // scalastyle:ignore funsuite
with Logging {
- test("LogKey enumeration fields must be sorted alphabetically") {
- val keys = LogKey.values.toSeq
- assert(keys === keys.sortBy(_.toString),
- "LogKey enumeration fields must be sorted alphabetically")
+ // scalastyle:off line.size.limit
+ /* Used to regenerate the LogKey class file. Run:
+ {{{
+ SPARK_GENERATE_GOLDEN_FILES=1 build/sbt \
+ "common-utils/testOnly *LogKeySuite -- -t \"LogKey enumeration fields
are correctly sorted\""
+ }}}
+ */
+ // scalastyle:on line.size.limit
+
+ /**
+ * Get a Path relative to the root project. It is assumed that a spark home
is set.
+ */
+ protected final def getWorkspaceFilePath(first: String, more: String*): Path
= {
+ if (!(sys.props.contains("spark.test.home") ||
sys.env.contains("SPARK_HOME"))) {
+ fail("spark.test.home or SPARK_HOME is not set.")
+ }
+ val sparkHome = sys.props.getOrElse("spark.test.home",
sys.env("SPARK_HOME"))
+ java.nio.file.Paths.get(sparkHome, first +: more: _*)
+ }
+
+ private val regenerateGoldenFiles: Boolean =
System.getenv("SPARK_GENERATE_GOLDEN_FILES") == "1"
+
+ private val logKeyFilePath = getWorkspaceFilePath("common", "utils", "src",
"main", "scala",
+ "org", "apache", "spark", "internal", "LogKey.scala")
+
+ test("LogKey enumeration fields are correctly sorted") {
+ val originalKeys = LogKey.values.toSeq
+ val sortedKeys = originalKeys.sortBy(_.toString)
+ if (regenerateGoldenFiles) {
Review Comment:
Do you mean: extract the regenerated method into an `independent method` and
call it in that place?
https://github.com/apache/spark/blob/3f6ac60e9966cbe2d35fc88693fdb055ad8c63c3/core/src/test/scala/org/apache/spark/SparkThrowableSuite.scala#L90-L117
The basic idea is that when without setting environment variables
`SPARK_GENERATE_GOLDEN_FILES` (default is false), the UT will only check
whether the enumeration fields in `LogKey` have been `sorted alphabetically`.
The steps used in my process of `migrating error classes` are roughly as
follows:
- Before I submit the changes, if I `remember` the formatting step, I will
use the following command to regenerate the error classes file
`error-classes.json` and submit them.
```
SPARK_GENERATE_GOLDEN_FILES=1 build/sbt "core/testOnly *SparkThrowableSuite
-- -t \"Error classes are correctly formatted\""
```
- If I `forget` this step, if there are incorrect item in
`error-classes.json` that are not sorted alphabetically, GA will help me
discover (UT failed), then I will execute above command on local and resubmit
the changes.
I guess our scene is very similar to this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
