[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-10 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14065 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63516/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63516/consoleFull)** for PR 14065 at commit [`bce8cd6`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63516 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63516/consoleFull)** for PR 14065 at commit [`bce8cd6`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-09 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14065 all my original comments were addressed and I won't have time to do another review until next week so I'm good with it if you are. --- If your project is set up for it, you can reply to this emai

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14065 Looks fine. There are some possible enhancements (e.g. what looks like some code repetition in the HDFS provider, neither Hive nor HBase return a token renewal time, etc) but those can be done separa

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63405/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63405/consoleFull)** for PR 14065 at commit [`71df428`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63405/consoleFull)** for PR 14065 at commit [`71df428`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63349/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63349/consoleFull)** for PR 14065 at commit [`32c16c0`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63349/consoleFull)** for PR 14065 at commit [`32c16c0`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-07 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14065 @vazin, looks like I missed that comment, I will address that today. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-05 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14065 @jerryshao do you plan to address the remaining feedback? (https://github.com/apache/spark/pull/14065#discussion_r73423485) --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63200/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63200 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63200/consoleFull)** for PR 14065 at commit [`127d85e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63200/consoleFull)** for PR 14065 at commit [`127d85e`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-03 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14065 LGTM, just pointed out one thing that I think would make the code easier to follow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-08-02 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14065 @vanzin , I did refactoring on some interfaces, especially for `obtainCredentials` method, and the implementation of `HDFSCredentialProvider` and `HiveCredential`, would you please help to review

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63068/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63068/consoleFull)** for PR 14065 at commit [`ac0062c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-31 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #63068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63068/consoleFull)** for PR 14065 at commit [`ac0062c`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14065 I still see a few, mostly minor, issues. I think the provider initialization should be cleaned up to avoid things like the hacky code in the HDFS provider. --- If your project is set up for it, you

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62915/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #62915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62915/consoleFull)** for PR 14065 at commit [`dc3be43`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #62915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62915/consoleFull)** for PR 14065 at commit [`dc3be43`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14065 I'm having trouble finding the bandwidth to look at the updated patch, but it's on my list... there were some replies to my comments that I want to take a closer look at. --- If your project is set

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/14065 couple minor nits, otherwise seems fine, but I want to run a couple tests locally. @vanzin were all your comments addressed? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62658/ Test PASSed. ---

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14065 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #62658 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62658/consoleFull)** for PR 14065 at commit [`bd04a2b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-20 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14065 Hi @tgravescs , thanks a lot for your comments. What I did to test is to run on security cluster with hive, hbase and hdfs installed and kerberized: 1. Verify credentials can be obtained i

[GitHub] spark issue #14065: [SPARK-14743][YARN] Add a configurable credential manage...

2016-07-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14065 **[Test build #62658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62658/consoleFull)** for PR 14065 at commit [`bd04a2b`](https://github.com/apache/spark/commit/b