[
https://issues.apache.org/jira/browse/SPARK-20772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bjorn Jonsson updated SPARK-20772:
----------------------------------
Description:
Spark uses rewrites of query parameters to paths
(http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0). This
works fine in local or standalone mode, but does not work on Yarn (with the
org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where the
query parameter is dropped.
The repro steps are:
- Start up the spark-shell and run a job
- Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
- A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
Going directly through the RM proxy works (does not cause query parameters to
be dropped).
was:
Spark uses rewrites of query parameters to paths
(http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0). This
works fine in local or standalone mode, but does not work on Yarn (with the
org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where the
query parameter is dropped.
The repro steps are:
- Start up the spark-shell in yarn client or cluster mode and run a job
- Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
- A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
Going directly through the RM proxy works (does not cause query parameters to
be dropped).
> Add support for query parameters in redirects on Yarn
> -----------------------------------------------------
>
> Key: SPARK-20772
> URL: https://issues.apache.org/jira/browse/SPARK-20772
> Project: Spark
> Issue Type: Improvement
> Components: YARN
> Affects Versions: 2.1.0
> Reporter: Bjorn Jonsson
> Priority: Minor
>
> Spark uses rewrites of query parameters to paths
> (http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0).
> This works fine in local or standalone mode, but does not work on Yarn (with
> the org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where
> the query parameter is dropped.
> The repro steps are:
> - Start up the spark-shell and run a job
> - Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
> - A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
> Going directly through the RM proxy works (does not cause query parameters to
> be dropped).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
