[
https://issues.apache.org/jira/browse/SPARK-49104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17872880#comment-17872880
]
Jack commented on SPARK-49104:
------------------------------
Thank you for this change (*)
I have proposed some related changes in relation to the master v1 rest
submission server in 2022 that I would love to be considered for v4:
https://issues.apache.org/jira/browse/SPARK-38862 /
[https://github.com/apache/spark/pull/46851] - the primary goal of my proposal
is not to replace/change what is happening here for clarity, although it would
allow a new contact point where authorisation _could_ have been applied +
customised further - that element is now taken care of thanks to this change
which is why I'm very happy to see it implemented and delivered.
The remaining problem is a general story to holistic authentication internal to
spark being off the cards for v1 rest submission users. If we want to utilise
spark.authenticate.secret properties for internal cluster authentication, while
utilising the rest submission server we need some additional changes. The
proxy/gateway approach I've proposed allows clients communicating to have the
necessary trust to the submit server if managed properly, to then enable spark
internal authentication to be handled as a separate concern if admin user
chooses this path - spark can do that part today but not with rest submission
enabled.
Given these are somewhat related, [~gurwls223], [~viirya] would you be willing
to inspect my proposal for soundness/readiness so there may be some opportunity
for me to address this gap for spark v4.0, it feels more suited to a major
change window than minor? I am most open to changing the implementation as well
as my current approach to gain review/acceptance. If this is better suited via
dev mail list or I need to do something more, I'm relatively new to the
community so I am highly appreciative for your advice - I feel stuck on how
best to progress it currently.
> Document `JWSFilter` usage in Spark UI and REST API and rename parameter to
> `secretKey`
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-49104
> URL: https://issues.apache.org/jira/browse/SPARK-49104
> Project: Spark
> Issue Type: Sub-task
> Components: Documentation, Spark Core
> Affects Versions: 4.0.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
