GitHub user yoonlee95 opened a pull request: https://github.com/apache/spark/pull/18978
[SPARK-21737][YARN]Create communication channel between arbitrary clients and the Spark AM in YARN mode ## What changes were proposed in this pull request? In this JIRA, I develop code to create a communication channel between arbitrary clients and a Spark AM on YARN. This code can be utilized to send commands such as getting status command, getting history info from the CLI, killing the application and pushing new tokens. Design Doc: https://docs.google.com/document/d/1QMbWhg13ocIoADywZQBRRVj-b9Zf8CnBrruP5JhcOOY/edit?usp=sharing ## How was this patch tested? Tested by sending a Kill RPC message to an AM. Worked for both sasl/auth layer Code used to test the communication channel. https://github.com/yoonlee95/spark/tree/authkill You can merge this pull request into a Git repository by running: $ git pull https://github.com/yoonlee95/spark clientToAMRpc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/18978.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #18978 ---- commit 14c680c6635402c35e3bed4a249936f54bceee82 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-09T19:59:59Z patch applied with merge conflict fixed commit af9d7e3b48e438c06ee6e948c65792aa59c95e4a Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-09T21:28:44Z tested cluster & client mode commit 26a92dc40a846e801646ac3a934faa7fc6894e0f Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-10T16:24:38Z scala style fixed commit f53848391b0506f2f75a9912256ee4d348857c28 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-10T17:19:47Z removed out file commit 19478d10d35c727e59a8b8548e149ade7a1c9af7 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-10T21:26:37Z init striping commit f829584b4270fcadf323c387fc846e2fa3b1bee9 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-16T19:29:20Z added auth rpc layer support commit ef5aec76019c74955af5c7b3ca081c920bd47ae8 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-17T16:32:27Z auth rpc debugged commit 8c46a0400f032f4e62e28a087fa7ca84635d9837 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-17T16:44:17Z Merge branch 'master' of https://github.com/apache/spark into clientToAMRpc Merged with master before making a pr commit 96b06c3074f0758ef70b30b3f86dbae47dfc6605 Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-17T18:02:00Z made hadoopSeecurityUils Class commit 7e6b0762f37d389864a2ebe62a461bb86247c7ec Author: John Lee <jl...@yahoo-inc.com> Date: 2017-08-17T18:51:50Z style fixed ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org