[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172613993 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamingDataWriterFactory.java --- @@ -0,0 +1,51 @@ +/* + * Licen

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #88020 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88020/testReport)** for PR 20295 at commit [`c74ed05`](https://github.com/apache/spark/commit/c7

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1328/ Tes

[GitHub] spark pull request #20742: [SPARK-23572][docs] Bring "security.md" up to dat...

2018-03-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20742#discussion_r172616332 --- Diff: R/pkg/DESCRIPTION --- @@ -57,6 +57,6 @@ Collate: 'types.R' 'utils.R' 'window.R' -RoxygenNote: 5.0.1 +RoxygenNote:

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172616601 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriter.java --- @@ -27,6 +28,9 @@ * * Streaming queries

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172616831 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -54,7 +55,14 @@ case class WriteToDataSourc

[GitHub] spark pull request #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml...

2018-03-06 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/20686#discussion_r172616958 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -324,19 +352,46 @@ class QuantileDiscretizerSuite

[GitHub] spark issue #20742: [SPARK-23572][docs] Bring "security.md" up to date.

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20742 **[Test build #88021 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88021/testReport)** for PR 20742 at commit [`63abd8a`](https://github.com/apache/spark/commit/63

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172617421 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamingDataWriterFactory.java --- @@ -0,0 +1,51 @@ +/* + *

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20751 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172617442 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriter.java --- @@ -27,6 +28,9 @@ * * Streaming que

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20750 **[Test build #88009 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88009/testReport)** for PR 20750 at commit [`8ed0695`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20750 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88009/ Test PASSed. ---

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1329/ Tes

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20751 **[Test build #88022 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88022/testReport)** for PR 20751 at commit [`6843098`](https://github.com/apache/spark/commit/68

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20686 **[Test build #88023 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88023/testReport)** for PR 20686 at commit [`7a14154`](https://github.com/apache/spark/commit/7a

[GitHub] spark issue #20742: [SPARK-23572][docs] Bring "security.md" up to date.

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20742 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172618681 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamingDataWriterFactory.java --- @@ -0,0 +1,51 @@ +/* + *

[GitHub] spark issue #20742: [SPARK-23572][docs] Bring "security.md" up to date.

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1330/ Tes

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20752 I'm not very familiar with the streaming side, but here is my 2 cents: I agree with @rdblue that it's unnecessary to introduce the epoch id to data sources that don't care about streaming. However

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172620679 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriter.java --- @@ -27,6 +28,9 @@ * * Streaming queries

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172621472 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamingDataWriterFactory.java --- @@ -0,0 +1,51 @@ +/* + * Licen

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172624233 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamingDataWriterFactory.java --- @@ -0,0 +1,51 @@ +/* + *

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20752 Thanks for the clear summary, @cloud-fan. I think we want to make it easy to support batch, and then easy to reuse those internals to support streaming by adding new mix-in interfaces. Streaming is m

[GitHub] spark pull request #20752: [SPARK-23559][SS] Create StreamingDataWriterFacto...

2018-03-06 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20752#discussion_r172627199 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriter.java --- @@ -27,6 +28,9 @@ * * Streaming que

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20751 **[Test build #88010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88010/testReport)** for PR 20751 at commit [`131ce84`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88010/ Test PASSed. ---

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20752 I agree that we should put a design for the streaming side on the dev list, and I intend to do so. The streaming interfaces will remain evolving until a design discussion about them has happened

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20752 > Right now, we're still at the point where we aren't quite sure what a streaming API needs to look like. We're starting from basically ground zero; the V1 streaming API just throws a DataFrame at th

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/20687 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20751 **[Test build #88011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88011/testReport)** for PR 20751 at commit [`f598486`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88011/ Test PASSed. ---

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1331/ Tes

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20687 **[Test build #88024 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88024/testReport)** for PR 20687 at commit [`63c7098`](https://github.com/apache/spark/commit/63

[GitHub] spark pull request #20750: [SPARK-23581][SQL] Add interpreted unsafe project...

2018-03-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20750#discussion_r172634434 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala --- @@ -146,6 +145,18 @@ object UnsafeProjection {

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20686 **[Test build #88023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88023/testReport)** for PR 20686 at commit [`7a14154`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20686 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88023/ Test PASSed. ---

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20752 Sounds fair to me. I'll continue iterating on the read side, and send out a design proposal for the write side incorporating this discussion in the next few days. ---

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #88020 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88020/testReport)** for PR 20295 at commit [`c74ed05`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88020/ Test FAILed. ---

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-06 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r172656702 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -42,7 +52,12 @@ class HadoopFileLinesReader

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-06 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r172657412 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextOptions.scala --- @@ -39,9 +39,12 @@ private[text] class TextOptions(

[GitHub] spark issue #20699: [SPARK-23544][SQL]Remove redundancy ShuffleExchange in t...

2018-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20699 I think they are caused by the same issue: an operator(the repartition operator) can produce exchange operator directly. If `EnsureRequirement` is the only place to produce/add exchange operator,

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20678 **[Test build #88016 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88016/testReport)** for PR 20678 at commit [`b5bea82`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88016/ Test PASSed. ---

[GitHub] spark issue #20678: [SPARK-23380][PYTHON] Adds a conf for Arrow fallback in ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20678 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20295 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20685: [SPARK-23524] Big local shuffle blocks should not be che...

2018-03-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20685 From what I've seen in https://github.com/apache/spark/pull/15923 , this corruption check is mostly to detect network failures, so seems like we don't need this check for local blocks at all.

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1332/ Tes

[GitHub] spark issue #20295: [SPARK-23011][SQL][PYTHON] Support alternative function ...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #88025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88025/testReport)** for PR 20295 at commit [`c74ed05`](https://github.com/apache/spark/commit/c7

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20750 **[Test build #88017 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88017/testReport)** for PR 20750 at commit [`207933b`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88017/ Test PASSed. ---

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20750 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r172661317 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java --- @@ -57,20 +59,20 @@ // The data stor

[GitHub] spark pull request #20735: [MINOR][YARN] Add disable yarn.nodemanager.vmem-c...

2018-03-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20735#discussion_r172661657 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -736,7 +736,8 @@ private object YarnAllocator

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20752 **[Test build #88019 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88019/testReport)** for PR 20752 at commit [`0c68fd1`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20752 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20752: [SPARK-23559][SS] Create StreamingDataWriterFactory for ...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20752 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88019/ Test PASSed. ---

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20687 **[Test build #88024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88024/testReport)** for PR 20687 at commit [`63c7098`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88024/ Test FAILed. ---

[GitHub] spark issue #20687: [SPARK-23500][SQL] Fix complex type simplification rules...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20687 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20751 **[Test build #88022 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88022/testReport)** for PR 20751 at commit [`6843098`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20751: [SPARK-23591][SQL] Add interpreted execution to EncodeUs...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20751 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88022/ Test PASSed. ---

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r172677514 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverConfigPropertiesStep.scala --- @@ -0,0 +1,85 @@

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r172678246 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverConfigPropertiesStep.scala --- @@ -0,0 +1,85 @@

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-03-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r172679601 --- Diff: resource-managers/kubernetes/integration-tests/include/util.sh --- @@ -0,0 +1,43 @@ +#!/usr/bin/env bash + +# Licensed to the Apache So

[GitHub] spark issue #20742: [SPARK-23572][docs] Bring "security.md" up to date.

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20742 **[Test build #88021 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88021/testReport)** for PR 20742 at commit [`63abd8a`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #20742: [SPARK-23572][docs] Bring "security.md" up to date.

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20742 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20742: [SPARK-23572][docs] Bring "security.md" up to date.

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88021/ Test PASSed. ---

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-03-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r172680200 --- Diff: resource-managers/kubernetes/integration-tests/e2e/e2e-prow.sh --- @@ -0,0 +1,69 @@ +#!/bin/bash + +# Licensed to the Apache Software F

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-03-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r172680337 --- Diff: resource-managers/kubernetes/integration-tests/build/mvn --- @@ -0,0 +1,29 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache S

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r172682591 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/HadoopFileLinesReader.scala --- @@ -42,7 +52,12 @@ class HadoopFileLinesRe

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-03-06 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/20697 In much of the old integration test setup, we were cloning and building Spark into a TGZ. I wonder if we can skip both of these steps. We shouldn't ever have to clone Spark, as we have the repositor

[GitHub] spark issue #10949: [SPARK-12832][MESOS] mesos scheduler respect agent attri...

2018-03-06 Thread atongen
Github user atongen commented on the issue: https://github.com/apache/spark/pull/10949 I am going to close this PR because there doesn't appear to be any interest in getting it merged. It's unfortunate, because it was a nice feature. --- -

[GitHub] spark pull request #10949: [SPARK-12832][MESOS] mesos scheduler respect agen...

2018-03-06 Thread atongen
Github user atongen closed the pull request at: https://github.com/apache/spark/pull/10949 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-03-06 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r172685889 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/cloud/CloudTestBackend.scala ---

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-03-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r172685274 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala --- @@ -120,14 +121,30 @@ abstract class JdbcDialect extends Serializab

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-03-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r172685440 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -95,4 +95,21 @@ private case object OracleDialect extends JdbcD

[GitHub] spark pull request #20697: [SPARK-23010][k8s] Initial checkin of k8s integra...

2018-03-06 Thread ssuchter
Github user ssuchter commented on a diff in the pull request: https://github.com/apache/spark/pull/20697#discussion_r172691439 --- Diff: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/backend/cloud/CloudTestBackend.scala ---

[GitHub] spark issue #20697: [SPARK-23010][k8s] Initial checkin of k8s integration te...

2018-03-06 Thread ssuchter
Github user ssuchter commented on the issue: https://github.com/apache/spark/pull/20697 I agree with your point about using a non-tgz version, but I'd prefer to do that as a separate change. (However, I reserve my right to change this stance if it turns out to be easier to make the ot

[GitHub] spark pull request #20464: [SPARK-23291][SQL][R] R's substr should not reduc...

2018-03-06 Thread viirya
GitHub user viirya reopened a pull request: https://github.com/apache/spark/pull/20464 [SPARK-23291][SQL][R] R's substr should not reduce starting position by 1 when calling Scala API ## What changes were proposed in this pull request? Seems R's substr API treats Scala subs

[GitHub] spark pull request #20464: [SPARK-23291][SQL][R] R's substr should not reduc...

2018-03-06 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/20464 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r172694097 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextOptions.scala --- @@ -39,9 +39,12 @@ private[text] class TextOpti

[GitHub] spark pull request #20727: [SPARK-23577][SQL] Supports custom line separator...

2018-03-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20727#discussion_r172694770 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/text/TextOptions.scala --- @@ -39,9 +39,12 @@ private[text] class TextOpti

[GitHub] spark issue #20748: [SPARK-23611][SQL] Add a helper function to check except...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20748 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20748: [SPARK-23611][SQL] Add a helper function to check except...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20748 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1333/ Tes

[GitHub] spark issue #20748: [SPARK-23611][SQL] Add a helper function to check except...

2018-03-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20748 **[Test build #88026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88026/testReport)** for PR 20748 at commit [`be88721`](https://github.com/apache/spark/commit/be

[GitHub] spark issue #20748: [SPARK-23611][SQL] Add a helper function to check except...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20748 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20748: [SPARK-23611][SQL] Add a helper function to check except...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20748 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1334/ Tes

[GitHub] spark issue #20748: [SPARK-23611][SQL] Add a helper function to check except...

2018-03-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20748 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

<    1   2   3   4   5   >