rmcyang commented on PR #37638: URL: https://github.com/apache/spark/pull/37638#issuecomment-1231893015
@zhouyejoe Hm not really, running those UTs locally looks to me. ``` minyang@minyang-mn3 spark % build/mvn -Dtest=none -Pyarn -Phadoop-2.10 -Dscala-2.12 -Phive -Phive-2.3 -DwildcardSuites=org.apache.spark.network.yarn.YarnShuffleServiceWithLevelDBBackendSuite test -pl 'resource-managers/yarn' Using `mvn` from path: /Users/minyang/project/OSS-linkedin/spark/build/apache-maven-3.8.6/bin/mvn ... [INFO] Skipping execution of surefire because it has already been run for this configuration [INFO] [INFO] --- scalatest-maven-plugin:2.1.0:test (test) @ spark-yarn_2.12 --- [INFO] ScalaTest report directory: /Users/minyang/project/OSS-linkedin/spark/resource-managers/yarn/target/surefire-reports Discovery starting. Discovery completed in 670 milliseconds. Run starting. Expected test count is: 17 YarnShuffleServiceWithLevelDBBackendSuite: - executor and merged shuffle state kept across NM restart - removed applications should not be in registered executor file and merged shuffle file - shuffle service should be robust to corrupt registered executor file - get correct recovery path - moving recovery file from NM local dir to recovery path - service throws error if cannot start - Consistency in AppPathInfo between in-memory hashmap and the DB - Finalized merged shuffle are written into DB and cleaned up after application stopped - Dangling finalized merged partition info in DB will be removed during restart - Dangling application path or shuffle information in DB will be removed during restart - Cleanup for former attempts local path info should be triggered in applicationRemoved - recovery db should not be created if NM recovery is not enabled - SPARK-31646: metrics should be registered into Node Manager's metrics system - SPARK-34828: metrics should be registered with configured name - create default merged shuffle file manager instance - create remote block push resolver instance - invalid class name of merge manager will use noop instance Run completed in 2 seconds, 944 milliseconds. Total number of tests run: 17 Suites: completed 2, aborted 0 Tests: succeeded 17, failed 0, canceled 0, ignored 0, pending 0 All tests passed. [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 24.019 s [INFO] Finished at: 2022-08-30T02:58:38-07:00 [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile "hadoop-2.10" could not be activated because it does not exist. [WARNING] The requested profile "hive" could not be activated because it does not exist. [WARNING] The requested profile "hive-2.3" could not be activated because it does not exist. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
