[
https://issues.apache.org/jira/browse/SPARK-26522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Euijun updated SPARK-26522:
---------------------------
Affects Version/s: (was: 3.0.0)
2.3.1
Priority: Minor (was: Major)
Fix Version/s: (was: 3.0.0)
Description:
Hi expert,
I try to use livy to connect sparkR backend.
This is related to
https://stackoverflow.com/questions/53900995/livy-spark-r-issue
Error message is,
{code:java}
Error in sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap, : Auth
secret not provided in environment.{code}
caused by,
spark-2.3.1/R/pkg/R/sparkR.R
{code:java}
sparkR.sparkContext <- function(
...
authSecret <- Sys.getenv("SPARKR_BACKEND_AUTH_SECRET")
if (nchar(authSecret) == 0) {
stop("Auth secret not provided in environment.")
}
...
)
{code}
Best regard.
was:
This is a follow up to SPARK-26194, which aims to add auto-generated secrets
similar to the YARN backend.
There's a desire to support different ways to generate and propagate these auth
secrets (e.g. using things like Vault). Need to investigate:
- exposing configuration to support that
- changing SecurityManager so that it can delegate some of the secret-handling
logic to custom implementations
- figuring out whether this can also be used in client-mode, where the driver
is not created by the k8s backend in Spark.
Component/s: (was: Kubernetes)
SparkR
Issue Type: Bug (was: New Feature)
Summary: Auth secret error in RBackendAuthHandler (was: CLONE -
Add configurable auth secret source in k8s backend)
> Auth secret error in RBackendAuthHandler
> ----------------------------------------
>
> Key: SPARK-26522
> URL: https://issues.apache.org/jira/browse/SPARK-26522
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 2.3.1
> Reporter: Euijun
> Assignee: Matt Cheah
> Priority: Minor
>
> Hi expert,
> I try to use livy to connect sparkR backend.
> This is related to
> https://stackoverflow.com/questions/53900995/livy-spark-r-issue
>
> Error message is,
> {code:java}
> Error in sparkR.sparkContext(master, appName, sparkHome, sparkConfigMap, :
> Auth secret not provided in environment.{code}
> caused by,
> spark-2.3.1/R/pkg/R/sparkR.R
>
> {code:java}
> sparkR.sparkContext <- function(
> ...
> authSecret <- Sys.getenv("SPARKR_BACKEND_AUTH_SECRET")
> if (nchar(authSecret) == 0) {
> stop("Auth secret not provided in environment.")
> }
> ...
> )
> {code}
>
> Best regard.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
