Github user vijoshi commented on the issue:
https://github.com/apache/spark/pull/17731
"I understand these 2 cases, can you explain how your change connect to
these two?"
Say, I do this:
```
delayAssign(delayedAssign(".sparkRsession", { sparkR.session(..) },
assign.env=SparkR:::.sparkREnv)
```
Now, when the user code such as this runs:
```
a <- createDataFrame(iris)
```
this sequence occurs:
```
createDataFrame()
> getSparkSession()
> get(".sparkRsession", envir = .sparkREnv)
> delayed evaluation of sparkR.session(...)
>
if (exists(".sparkRsession", envir = .sparkREnv))
sparkSession <- get(".sparkRsession", envir = .sparkREnv)
# error occurs here
> Error "Promise already under evaluation"
```
The change is to ignore the "Promise under evaluation" error. At the line
where error occurs, there doesn't seem to be any other possible cause for
failure since the previous line of code has already checked that the
`.sparkRsession` exists in the environment. So if we take it that this happens
only when is `.sparkRsession` bound lazily and ignore it - which is what my
change does - the code proceeds with regular computation of sparkSession.
Similar is the case with `.sparkRjsc`. The SparkR code inside
`spark.sparkContext(..)` does this:
```
if (exists(".sparkRjsc", envir = .sparkREnv)) {
sparkRjsc <- get(".sparkRjsc", envir = .sparkREnv) # "Promise under
evaluation" error occurs here
}
```
When `.sparkRjsc` is lazily bound the `exists(..)` condition succeeds, and
the ""Promise under evaluation" error occurs. If the error is ignored
considering that there can't be any other cause for failure, the lazy
initialization works.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
