[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-26 Thread yashs360 
Github user yashs360 commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks for your help and patience on this one @brkyvz :)


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread brkyvz 
Github user brkyvz commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged to master. Thanks!


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85373/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85373 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85373/testReport)**
 for PR 18029 at commit 
[`bdd080c`](https://github.com/apache/spark/commit/bdd080c2ea7b3c2f215e1ca6a6614ade3f074601).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public class KinesisInitialPositions `
  * `public static class Latest implements KinesisInitialPosition, 
Serializable `
  * `public static class TrimHorizon implements KinesisInitialPosition, 
Serializable `
  * `public static class AtTimestamp implements KinesisInitialPosition, 
Serializable `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85373 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85373/testReport)**
 for PR 18029 at commit 
[`bdd080c`](https://github.com/apache/spark/commit/bdd080c2ea7b3c2f215e1ca6a6614ade3f074601).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-25 Thread yashs360 
Github user yashs360 commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks @brkyvz for all your help on this one. I have pushed the changes and 
waiting for the test build.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread yashs360 
Github user yashs360 commented on the issue:

https://github.com/apache/spark/pull/18029
  
Hi @brkyvz , I've added the new changes with the java classes. Had to make 
the classes serializable for passing them to the KinesisReceiver. Please have a 
look when you get time. Thanks.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85181/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85181 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85181/testReport)**
 for PR 18029 at commit 
[`3c16c47`](https://github.com/apache/spark/commit/3c16c478257c8aed61b1cef4d75360b8bb8b166d).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public class KinesisInitialPositions `
  * `public static class Latest implements KinesisInitialPosition, 
Serializable `
  * `public static class TrimHorizon implements KinesisInitialPosition, 
Serializable `
  * `public static class AtTimestamp implements KinesisInitialPosition, 
Serializable `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85181 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85181/testReport)**
 for PR 18029 at commit 
[`3c16c47`](https://github.com/apache/spark/commit/3c16c478257c8aed61b1cef4d75360b8bb8b166d).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85171/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85171 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85171/testReport)**
 for PR 18029 at commit 
[`e18fdaa`](https://github.com/apache/spark/commit/e18fdaa2b9c70f58b57fc564c137a2dce51d2b25).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public class KinesisInitialPositions `
  * `public static class Latest implements KinesisInitialPosition `
  * `public static class TrimHorizon implements KinesisInitialPosition `
  * `public static class AtTimestamp implements KinesisInitialPosition `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85171 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85171/testReport)**
 for PR 18029 at commit 
[`e18fdaa`](https://github.com/apache/spark/commit/e18fdaa2b9c70f58b57fc564c137a2dce51d2b25).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85168/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85168 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85168/testReport)**
 for PR 18029 at commit 
[`9abf92b`](https://github.com/apache/spark/commit/9abf92b1fe871128c607089faf07c6ef807321e2).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public class KinesisInitialPositions `
  * `public static class Latest implements KinesisInitialPosition `
  * `public static class TrimHorizon implements KinesisInitialPosition `
  * `public static class AtTimestamp implements KinesisInitialPosition `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #85168 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85168/testReport)**
 for PR 18029 at commit 
[`9abf92b`](https://github.com/apache/spark/commit/9abf92b1fe871128c607089faf07c6ef807321e2).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-18 Thread brkyvz 
Github user brkyvz commented on the issue:

https://github.com/apache/spark/pull/18029
  
Making them as singletons is unnecessary. How about this:

```java
public interface InitialPosition {
public InitialPositionInStream toKinesis();
}

public class InitialPositions {
public static class Latest implements InitialPosition {
public Latest() {}

@Override
public InitialPositionInStream toKinesis() {
return InitialPositionInStream.LATEST;
}
}

public static class TrimHorizon implements InitialPosition {
public TrimHorizon() {}

@Override
public InitialPositionInStream toKinesis() {
return InitialPositionInStream.TRIM_HORIZON;
}
}

public static class AtTimestamp implements InitialPosition {
private Date timestamp;

public AtTimestamp(Date timestamp) {
this.timestamp = timestamp;
}

@Override
public InitialPositionInStream toKinesis() {
return InitialPositionInStream.AT_TIMESTAMP;
}

public Date getTimestamp() {
return timestamp;
}
}
}
```


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-15 Thread yashs360 
Github user yashs360 commented on the issue:

https://github.com/apache/spark/pull/18029
  
Hi @brkyvz Thinking on these lines, Adding them as Java objects adds more 
complexity to our design. We again have to think about making the objects 
singleton and thread safe. The Scala case class were very simple and minimal.

This is how we would have to implement the java classes for the Initial 
positions. It looks a bit unclean to me. Thoughts ?

```

abstract class InitialPosition {
public static final InitialPositionInStream initialPositionInStream = 
InitialPositionInStream.LATEST;
}

class Latest extends InitialPosition {
private static final Latest instance = new Latest();
static final InitialPositionInStream initialPositionInStream = 
InitialPositionInStream.LATEST;

private Latest(){}

public static InitialPosition getInstance() {
return instance;
}
}

class TrimHorizon extends InitialPosition {
private static final TrimHorizon instance = new TrimHorizon();
static final InitialPositionInStream initialPositionInStream = 
InitialPositionInStream.TRIM_HORIZON;

private TrimHorizon(){}

public static InitialPosition getInstance() {
return instance;
}
}

class AtTimestamp extends InitialPosition {
static final InitialPositionInStream initialPositionInStream = 
InitialPositionInStream.AT_TIMESTAMP;
Date timestamp;

private AtTimestamp(Date timestamp){
this.timestamp = timestamp;
}

public static InitialPosition getInstance(Date timestamp) {
return new AtTimestamp(timestamp);
}
}

```


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-12-14 Thread brkyvz 
Github user brkyvz commented on the issue:

https://github.com/apache/spark/pull/18029
  
Actually yeah, I like your way.

On Dec 14, 2017 3:08 PM, "yashs360"  wrote:

> *@yashs360* commented on this pull request.
> --
>
> In external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/
> InitialPosition.scala
> :
>
> > +import java.util.Date
> +
> +import 
com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream
> +
> +/**
> + * Trait for Kinesis's InitialPositionInStream.
> + * This will be overridden by more specific types.
> + */
> +sealed trait InitialPosition {
> +  val initialPositionInStream: InitialPositionInStream
> +}
> +
> +/**
> + * Case object for Kinesis's InitialPositionInStream.LATEST.
> + */
> +case object Latest extends InitialPosition {
>
> Hi @brkyvz  , Thanks for the review.
> Are you suggesting to put everything into a new object. And refer the case
> objects from the java class methods?
> In that case is it better to create the objects in Java and expose them
> directly, since we will have cases where we will need direct access to the
> case objects/classes (instead of the java methods) like one of the test
> cases:
> initialPosition.asInstanceOf[AtTimestamp].timestamp
>
> I would create a new branch with the changes and share with you if its
> fine ?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or 
mute
> the thread
> 

> .
>



---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-11-08 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks for being so helpful and patient on this one @brkyvz . I will leave 
this with you now for your final ☑️ if you're happy with it :)


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-11-03 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@brkyvz could you please have a look if it looks good. Would be great if 
you're happy with the changes and we could merge it.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-27 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@brkyvz Squashed multiple commits into one for better readability. Please 
have a look when you get time. Thanks.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-27 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-27 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #83144 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83144/testReport)**
 for PR 18029 at commit 
[`2c2a56a`](https://github.com/apache/spark/commit/2c2a56a5c8c7dc4660af77757d26c627ca639e8a).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public class KinesisInitialPosition `
  * `sealed trait InitialPosition `
  * `case class AtTimestamp(timestamp: Date) extends InitialPosition `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-27 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83144/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-27 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #83144 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83144/testReport)**
 for PR 18029 at commit 
[`2c2a56a`](https://github.com/apache/spark/commit/2c2a56a5c8c7dc4660af77757d26c627ca639e8a).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-19 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@brkyvz Please have a look once you have time. Thanks.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-12 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks for the review @brkyvz . Updated the pull request with version 2.3.0.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-12 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82708/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-12 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82708 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82708/testReport)**
 for PR 18029 at commit 
[`ac0ae53`](https://github.com/apache/spark/commit/ac0ae53935510221ebb6a44f6ec30b68fca34343).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-12 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-12 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82708 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82708/testReport)**
 for PR 18029 at commit 
[`ac0ae53`](https://github.com/apache/spark/commit/ac0ae53935510221ebb6a44f6ec30b68fca34343).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks for the review @brkyvz .
Please have a look at the new PR. I have implemented the review comments.
For the API hygiene, I have added both the new and old API, and have marked 
the old API as deprecated. The rational behind keeping the new API is to 
support any future changes to the Kinesis consumer lib, and we could just add 
new case classes in one place to handle new params. Otherwise we would have to 
keep adding new arguments to the kinesis receiver function. The New api also 
makes it clear to understand what exactly is the purpose of the arguments.
I have also added new test cases to test the old api with both +ve and -ve 
scenarios.

Please have a look and share your thoughts.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82635/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82635 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82635/testReport)**
 for PR 18029 at commit 
[`72703a0`](https://github.com/apache/spark/commit/72703a072e34407b52d555bab98a435414d2ed25).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82635 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82635/testReport)**
 for PR 18029 at commit 
[`72703a0`](https://github.com/apache/spark/commit/72703a072e34407b52d555bab98a435414d2ed25).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks @brkyvz . Appreciate the review comments.
I am facing some issues with the scala packaging. I will keep working on it 
and update the patch soon.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82631/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82631 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82631/testReport)**
 for PR 18029 at commit 
[`ed9cab4`](https://github.com/apache/spark/commit/ed9cab48c61627f1b4e1a6f1afc16d96ae24207e).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-11 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82631 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82631/testReport)**
 for PR 18029 at commit 
[`ed9cab4`](https://github.com/apache/spark/commit/ed9cab48c61627f1b4e1a6f1afc16d96ae24207e).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82626/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82626 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82626/testReport)**
 for PR 18029 at commit 
[`483a697`](https://github.com/apache/spark/commit/483a6979d8e06ed99cd1d2deadefdd980bf1ffdf).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82626 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82626/testReport)**
 for PR 18029 at commit 
[`483a697`](https://github.com/apache/spark/commit/483a6979d8e06ed99cd1d2deadefdd980bf1ffdf).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82625/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82625 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82625/testReport)**
 for PR 18029 at commit 
[`06e3969`](https://github.com/apache/spark/commit/06e3969d0d4e19fb2fb989973a7836ec80f27d82).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82625 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82625/testReport)**
 for PR 18029 at commit 
[`06e3969`](https://github.com/apache/spark/commit/06e3969d0d4e19fb2fb989973a7836ec80f27d82).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82624/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82624 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82624/testReport)**
 for PR 18029 at commit 
[`05cad04`](https://github.com/apache/spark/commit/05cad04d742811689f2f5da7685244bf5c5937a5).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82624 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82624/testReport)**
 for PR 18029 at commit 
[`05cad04`](https://github.com/apache/spark/commit/05cad04d742811689f2f5da7685244bf5c5937a5).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82623 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82623/testReport)**
 for PR 18029 at commit 
[`107c1ab`](https://github.com/apache/spark/commit/107c1aba8b029cac933ab726e45e9f13fabcfccb).
 * This patch **fails Scala style tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82623/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-10 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #82623 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82623/testReport)**
 for PR 18029 at commit 
[`107c1ab`](https://github.com/apache/spark/commit/107c1aba8b029cac933ab726e45e9f13fabcfccb).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-03 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Hi @brkyvz can I get some love here please.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-24 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@brkyvz Could you please check this for the last suggestions ?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-20 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@budde could you please do one last review of this one.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-07 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81530/
Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-07 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #81530 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81530/testReport)**
 for PR 18029 at commit 
[`cef5cde`](https://github.com/apache/spark/commit/cef5cdece2bd2a7c95e19493c511d602c1b46461).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `public class KinesisInitialPosition `
  * `sealed trait InitialPosition `
  * `case class AtTimestamp(timestamp: Date) extends InitialPosition `


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-07 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-07 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #81530 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81530/testReport)**
 for PR 18029 at commit 
[`cef5cde`](https://github.com/apache/spark/commit/cef5cdece2bd2a7c95e19493c511d602c1b46461).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-06 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks for the comments @brkyvz . I will be working on the changes and 
update the PR very soon.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/81358/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #81358 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81358/testReport)**
 for PR 18029 at commit 
[`0c46008`](https://github.com/apache/spark/commit/0c4600818b6e248741621d23a748c37c7b7e8c68).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `sealed trait InitialPosition `
  * `case class AtTimestamp(timestamp: Date) extends InitialPosition `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #81358 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/81358/testReport)**
 for PR 18029 at commit 
[`0c46008`](https://github.com/apache/spark/commit/0c4600818b6e248741621d23a748c37c7b7e8c68).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-09-03 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Resolved conflict introduced by other code commits.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-30 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Could I get some love here from the committers please @brkyvz @HyukjinKwon 
@srowen . Would love to work on any changes if required.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-23 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Hi @srowen  we've iterated this patch to bring it in a good state. Need a 
committer ✔️ before we can go ahead merging it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #80913 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80913/testReport)**
 for PR 18029 at commit 
[`eb7ad56`](https://github.com/apache/spark/commit/eb7ad56b598af5e537e5fa1808dc93b692a14f6f).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `sealed trait InitialPosition `
  * `case class AtTimestamp(timestamp: Date) extends InitialPosition `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80913/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Will wait for @brkyvz , @HyukjinKwon for final ☑️ 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #80913 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80913/testReport)**
 for PR 18029 at commit 
[`eb7ad56`](https://github.com/apache/spark/commit/eb7ad56b598af5e537e5fa1808dc93b692a14f6f).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Added review suggestions @budde !


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-20 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Will update and post another request seen. Thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80876/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #80876 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80876/testReport)**
 for PR 18029 at commit 
[`7d9a08a`](https://github.com/apache/spark/commit/7d9a08aa4471e2b76bb15e72325ef717ab469ee2).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds the following public classes _(experimental)_:
  * `sealed trait InitialPosition `
  * `case class AtTimestamp(timestamp: Date) extends InitialPosition `


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #80876 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80876/testReport)**
 for PR 18029 at commit 
[`7d9a08a`](https://github.com/apache/spark/commit/7d9a08aa4471e2b76bb15e72325ef717ab469ee2).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #80871 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80871/testReport)**
 for PR 18029 at commit 
[`c3622b9`](https://github.com/apache/spark/commit/c3622b91332775a363c68d9571f4f5dbc3af84fc).
 * This patch **fails to build**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80871/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/18029
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/18029
  
**[Test build #80871 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80871/testReport)**
 for PR 18029 at commit 
[`c3622b9`](https://github.com/apache/spark/commit/c3622b91332775a363c68d9571f4f5dbc3af84fc).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@brkyvz I have made the suggested modifications to the code. Please have a 
look when you get time. Thanks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-19 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
Thanks @budde for the review . Love the suggestions. 👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-18 Thread budde 
Github user budde commented on the issue:

https://github.com/apache/spark/pull/18029
  
@yssharma Let me know what you think of my review suggestions. I should be 
able to review any updates from here on in a timely manner but you will still 
need @brkyvz or another Spark committer to do the final review and approve the 
PR.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-08-05 Thread yssharma 
Github user yssharma commented on the issue:

https://github.com/apache/spark/pull/18029
  
@brkyvz , @budde, @HyukjinKwon  could you please review this sometime. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org