[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-22 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 Merging to master. Thanks for your patience! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73248/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73248/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73244/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73244/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73245/ Test FAILed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73245/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73248/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Updated the PR. Thanks for the work you've done on this! Hopefully I can have a PR for the builder interface up later this week. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73245/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 Two final comments. Then I'll merge it pending tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73244/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Missed updating a test, my mistake. Fixing now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73240/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73240/ Test FAILed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz I've updated the PR per your feedback. ```BasicAWSCredentials``` will raise a ```java.lang.IllegalArgumentException``` if either keypair value is null so I elected to wrap

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73240 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73240/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz I actually think that Scaladoc may be outdated– I double checked the current master branch and it looks like ```KinesisUtils.createStream()``` will still provide

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 @budde The scaladocs mention ``` * @param awsAccessKeyId AWS AccessKeyId (if null, will use DefaultAWSCredentialsProviderChain) * @param awsSecretKey AWS SecretKey (if null, will use

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 So, if these values are ```null``` we'll still be passing them to construct a ```BasicCredentialsProvider``` to pass as ```STSCredentialsProvider.longLivedCredentialsProvider```. I could add a check

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 Can't they still use `null` to use the `DefaultProviderChain`? It's still supported, right? We're only forcing them to provide a `messageHandler`. --- If your project is set up for it, you can

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz I share your concerns around expanding this API further than necessary. I think I'm okay with this as long as we're fairly confident the builder pattern work can be merged in the same Spark

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 @budde I'm just concerned by the exponential blowoff of APIs. Here's my proposal. For both Java and Scala, let's just add the APIs with both STS token and AWS Key pair defined versions. I'm

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-21 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Anyone I can ping to help get this merged? The PR is going on a month old at this point and I know that lack of STS support is an issue that several interested parties would like to see get

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-20 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 Yes, it can be merged as-is. I would create a JIRA for the builder interface, and start working on it simultaneously. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-20 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Just for clarification, can this PR be merged as-is with a separate Jira/PR for adding a builder interface or is the builder interface a prerequisite for merging this? --- If your project

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/73090/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73090 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73090/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #73090 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/73090/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 Please do. I'd like to get this merged soon, because it's a useful PR. I'll try to take one more look today. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Fair enough. Let me know if there's anything I can do to help get this merged. I can also take a look at adding a builder class for Kinesis streams as a separate PR before the code freeze.

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-17 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 @budde Since this PR adds new functionalities, I'm inclined to not backport it since Spark Streaming APIs are considered stable, and this PR adds a lot of new APIs which is weird for a maintenance

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-16 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz, @zsxwing – Any update here? Worried that this PR is starting to languish. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-14 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Any thoughts on moving the dependency version bump to a new commit and backporting to 2.11 with the pervious versions? @zswing Any chance you could take a look at this sometime this

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-10 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Would it be possible to backport to 2.1.1 if I reverted to the old version of the KCL and made the dependency upgrade as a separate PR? We'd still be adding ```aws-java-sdk-sts``` as a

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 @budde This looks good to me, but I've asked @zsxwing to also take a look with a clean set of eyes. We can't backport this to 2.1.1, because of the dependency changes. I'm also wondering if there

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 @brkyvz Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72656/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72656 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72656/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72656 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72656/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 @budde You should be whitelisted now. you may use "retest this please" to trigger tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-09 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Pinging @brkyvz and @srowen once more for a final look and to get Jenkins to retest the latest update (not sure if this still requires Jenkins admin rights). --- If your project is set up for it,

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-08 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Looks like Jenkins is failing to build any recent PR due to the following error: ```[error] Could not find hadoop2.3 in the list. Valid options are ['hadoop2.6', 'hadoop2.7']``` I

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72593/ Test FAILed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72593/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-08 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Amending the PR again to fix new dependency conflict in spark/pom.xml. Thanks again for taking the time to review this, @brkyvz and @srowen. Please let me know if you feel any additional changes are

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72593/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72533/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72533 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72533/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72532/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72532/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72533/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Amending PR per review feedback. Issue around using optional stsExternalId argument in ```KinesisUtils.createStream()``` remains open. --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72532/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72481/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72481/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72480/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72480/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72481/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72480/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 PR has been amended to reflect feedback. Thanks for taking a look, @brkyvz. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/16744 Hi @budde, taking a look at this now. Sorry for the wait --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72465/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72465/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Amending this PR to upgrade the KCL/AWS SDK dependencies to more-current versions (1.7.3 and 1.11.76, respectively). The ```RegionUtils.getRegionByEndpoint()``` API was removed from the SDK, so I've

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72465/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-01 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Pinging @brkyvz as well, who also appears to have reviewed kinesis-asl changes in the past --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-01 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 There shouldn't be any change to behavior or compatibility when using the existing implementations of ```KinesisUtils.createStream()```. Only drawback I can think of is this is making the

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-02-01 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Pinging @zsxwing and @srowen, additional committers who have previously reviewed kinesis-asl changes. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-31 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Pinging @tdas on this-- looks like you're the committer who has contributed the most to kinesis-asl. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72175/ Test PASSed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72175/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Also, on another note, the ```SerializableKCLAuthProvider``` class that **SparkQA** is identifying as a new public class is actually package private and replaced another package private class

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 The JIRA I opended for this issue contains further details and background. Linking to it here for good measure: * https://issues.apache.org/jira/browse/SPARK-19405 --- If your project is

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72175/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread budde
Github user budde commented on the issue: https://github.com/apache/spark/pull/16744 Missed the code in python/streaming that this touches. Will update PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72165/ Test FAILed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72165/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72165/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72164/ Test FAILed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72164/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72164 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72164/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72163/ Test FAILed. ---

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72163/testReport)** for PR 16744 at commit

[GitHub] spark issue #16744: [SPARK-19405][STREAMING] Support for cross-account Kines...

2017-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16744 **[Test build #72163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72163/testReport)** for PR 16744 at commit

  1   2   >