Bjorn Jonsson created SPARK-20772:
-------------------------------------
Summary: Add support for query parameters in redirects on Yarn
Key: SPARK-20772
URL: https://issues.apache.org/jira/browse/SPARK-20772
Project: Spark
Issue Type: Improvement
Components: YARN
Affects Versions: 2.1.0
Reporter: Bjorn Jonsson
Priority: Minor
Spark uses rewrites of query parameters to paths
(http://<fqdn>:4040/jobs/job?id=0 --> http://<fqdn>:4040/jobs/job/?id=0). This
works fine in local or standalone mode, but does not work on Yarn (with the
org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter filter), where the
query parameter is dropped.
The repro steps are:
- Start up the spark-shell in yarn client or cluster mode and run a job
- Try to access the job details through http://<fqdn>:4040/jobs/job?id=0
- A HTTP ERROR 400 is thrown (requirement failed: missing id parameter)
Going directly through the RM proxy works (does not cause query parameters to
be dropped).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
