WamBamBoozle commented on a change in pull request #31162:
URL: https://github.com/apache/spark/pull/31162#discussion_r558449658
##########
File path: R/pkg/inst/worker/daemon.R
##########
@@ -32,6 +32,9 @@ inputCon <- socketConnection(
SparkR:::doServerAuth(inputCon, Sys.getenv("SPARKR_WORKER_SECRET"))
+# Application-specific daemon initialization. Typical use is loading libraries.
+eval(parse(text = Sys.getenv("SPARKR_DAEMON_INIT")))
+
Review comment:
To give you a sense of the cost of this, consider
```
> Sys.setenv(x="NULL")
> microbenchmark(eval(parse(text = Sys.getenv("x"))), times=99999)
Unit: microseconds
expr min lq mean median uq
max neval
eval(parse(text = Sys.getenv("x"))) 33.854 35.82 40.15034 37.479 39.4475
7219.072 99999
```
So, on my 2017 MacBook pro, it takes 40 microseconds.
For comparison, consider the 6 milliseconds we use loading worker.R at every
fork.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
