WamBamBoozle commented on a change in pull request #31162:
URL: https://github.com/apache/spark/pull/31162#discussion_r558449658
##########
File path: R/pkg/inst/worker/daemon.R
##########
@@ -32,6 +32,9 @@ inputCon <- socketConnection(
SparkR:::doServerAuth(inputCon, Sys.getenv("SPARKR_WORKER_SECRET"))
+# Application-specific daemon initialization. Typical use is loading libraries.
+eval(parse(text = Sys.getenv("SPARKR_DAEMON_INIT")))
+
Review comment:
To give you a sense of the cost of this, consider
```
> microbenchmark(NULL, times = 999999)
Unit: nanoseconds
expr min lq mean median uq max neval
NULL 2 4 4.328575 4 4 13885 999999
> microbenchmark(eval(parse(text = "NULL")), times=9999)
Unit: microseconds
expr min lq mean median uq max
neval
eval(parse(text = "NULL")) 31.489 34.0545 40.1457 35.938 38.496 9370.646
9999
>
```
So, on my 2017 MacBook pro, it takes 34 microseconds.
For comparison, consider the 6 milliseconds we use up loading worker.R at
every fork.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
