turboFei commented on code in PR #3345:
URL: https://github.com/apache/celeborn/pull/3345#discussion_r2165924357
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/profiler/JVMProfiler.scala:
##########
@@ -77,4 +82,32 @@ class JVMProfiler(conf: CelebornConf) extends Logging {
})
}
}
+
+ /** Returns the directory used for storing the extracted libraries, binaries
and JARs. */
+ private def extractionDir: Path = {
+ val extractionDirectory = applicationsDir
+ if (extractionDirectory.toFile.exists()) {
+ extractionDirectory
+ } else {
+ Utils.createTempDir(conf.workerWorkingDir, "profiler").toPath
+ }
+ }
+
+ /**
+ * Returns directory where applications places their files. Specific to
operating system.
+ * <p>Note: copied from {@code AsyncProfilerLoader#getApplicationsDir} of
ap-loader.
+ *
+ * @return Directory where applications places their files.
+ */
+ private def applicationsDir: Path = {
+ if (Utils.isLinux) {
+ val xdgDataHome = System.getenv("XDG_DATA_HOME")
+ if (xdgDataHome != null && xdgDataHome.nonEmpty) { return
Paths.get(xdgDataHome) }
+ return Paths.get(System.getProperty("user.home"), ".local", "share")
+ } else if (Utils.isMac) {
+ return Paths.get(System.getProperty("user.home"), "Library",
"Application Support")
+ }
+ throw new UnsupportedOperationException(
+ s"Unsupported os ${System.getProperty("os.name").toLowerCase()}")
Review Comment:
The logical is a little complex, maybe we can simplify it likes Spark does.
https://github.com/apache/spark/blob/1cfe07cdbeea765890cc93f8292ca0a6f13408f2/connector/profiler/src/main/scala/org/apache/spark/profiler/SparkAsyncProfiler.scala#L69-L70
##########
worker/src/main/scala/org/apache/celeborn/service/deploy/worker/profiler/JVMProfiler.scala:
##########
@@ -77,4 +82,24 @@ class JVMProfiler(conf: CelebornConf) extends Logging {
})
}
}
+
+ private def extractionDir: Path = {
+ val extractionDirectory = applicationsDir
+ if (extractionDirectory.toFile.exists()) {
+ extractionDirectory
+ } else {
+ Utils.createTempDir(conf.workerWorkingDir, "profiler").toPath
Review Comment:
how about using `asyncProfiler`? same with Spark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
