[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-07-13 Thread Mike Percy (Code Review) 
Mike Percy has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/10725 )

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..

spark: Add support for scanRequestTimeout to DefaultSource

The timeout was manually tested via spark-shell and when set to a very
low value (1ms) a combination or error messages and timeout warnings
were printed due to Spark task retries.

Added a simple verification test to ensure that parameter parsing works.

Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Reviewed-on: http://gerrit.cloudera.org:8080/10725
Tested-by: Kudu Jenkins
Reviewed-by: Dan Burkert 
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
4 files changed, 50 insertions(+), 7 deletions(-)

Approvals:
  Kudu Jenkins: Verified
  Dan Burkert: Looks good to me, approved

--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 10
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-07-12 Thread Dan Burkert (Code Review) 
Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10725 )

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..


Patch Set 9: Code-Review+2


--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 9
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Comment-Date: Fri, 13 Jul 2018 00:41:38 +
Gerrit-HasComments: No

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-07-12 Thread Mike Percy (Code Review) 
Hello Dan Burkert, Kudu Jenkins, Grant Henke,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10725

to look at the new patch set (#9).

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..

spark: Add support for scanRequestTimeout to DefaultSource

The timeout was manually tested via spark-shell and when set to a very
low value (1ms) a combination or error messages and timeout warnings
were printed due to Spark task retries.

Added a simple verification test to ensure that parameter parsing works.

Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
4 files changed, 50 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/10725/9
--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 9
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-07-11 Thread Mike Percy (Code Review) 
Mike Percy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10725 )

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10725/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
File 
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala:

http://gerrit.cloudera.org:8080/#/c/10725/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@24
PS8, Line 24: import com.google.common.annotations.VisibleForTesting
> guava isn't a dependency of kudu-spark, so this could be problematic.  It's
Ah, good point. I'll just remove the annotation.



--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 8
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Comment-Date: Wed, 11 Jul 2018 19:16:06 +
Gerrit-HasComments: Yes

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-07-11 Thread Dan Burkert (Code Review) 
Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10725 )

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..


Patch Set 8:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10725/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
File 
java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala:

http://gerrit.cloudera.org:8080/#/c/10725/8/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala@24
PS8, Line 24: import com.google.common.annotations.VisibleForTesting
guava isn't a dependency of kudu-spark, so this could be problematic.  It's 
also arguably not needed, since the symbol you are exposing isn't actually 
public, it's only visible within this project.



--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 8
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Comment-Date: Wed, 11 Jul 2018 18:53:38 +
Gerrit-HasComments: Yes

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-07-10 Thread Mike Percy (Code Review) 
Hello Dan Burkert, Kudu Jenkins, Grant Henke,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/10725

to look at the new patch set (#8).

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..

spark: Add support for scanRequestTimeout to DefaultSource

The timeout was manually tested via spark-shell and when set to a very
low value (1ms) a combination or error messages and timeout warnings
were printed due to Spark task retries.

Added a simple verification test to ensure that parameter parsing works.

Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala
M 
java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
4 files changed, 53 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/10725/8
--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 8
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Grant Henke 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-06-15 Thread Mike Percy (Code Review) 
Mike Percy has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10725 )

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/10725/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10725/2//COMMIT_MSG@10
PS2, Line 10: low value (1ms) timeout warnings were printed.
> But the job didn't fail?
Yeah, it apparently succeeded on retry. I'll dig deeper to try to understand 
exactly what happened.



--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 4
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy 
Gerrit-Comment-Date: Fri, 15 Jun 2018 17:01:08 +
Gerrit-HasComments: Yes

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-06-14 Thread Dan Burkert (Code Review) 
Dan Burkert has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/10725 )

Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..


Patch Set 2:

(1 comment)

Code LGTM

http://gerrit.cloudera.org:8080/#/c/10725/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/10725/2//COMMIT_MSG@10
PS2, Line 10: low value (1ms) timeout warnings were printed.
But the job didn't fail?



--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 2
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert 
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Comment-Date: Fri, 15 Jun 2018 00:40:35 +
Gerrit-HasComments: Yes

[kudu-CR] spark: Add support for scanRequestTimeout to DefaultSource

2018-06-14 Thread Mike Percy (Code Review) 
Hello Dan Burkert,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10725

to review the following change.


Change subject: spark: Add support for scanRequestTimeout to DefaultSource
..

spark: Add support for scanRequestTimeout to DefaultSource

The timeout was manually tested via spark-shell and when set to a very
low value (1ms) timeout warnings were printed.

Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
---
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/DefaultSource.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala
M java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala
3 files changed, 20 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/10725/1
--
To view, visit http://gerrit.cloudera.org:8080/10725
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I62b01af9d0532448fc3a6cf9328b3912a7865ebb
Gerrit-Change-Number: 10725
Gerrit-PatchSet: 1
Gerrit-Owner: Mike Percy 
Gerrit-Reviewer: Dan Burkert