[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23254: [SPARK-26304][SS] Add default value to spark.kafka.sasl....

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23254 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99830/ Test PASSed. ---

[GitHub] spark issue #23254: [SPARK-26304][SS] Add default value to spark.kafka.sasl....

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23254 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23254: [SPARK-26304][SS] Add default value to spark.kafka.sasl....

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23254 **[Test build #99830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99830/testReport)** for PR 23254 at commit

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99834/ Test FAILed. ---

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99834/testReport)** for PR 22957 at commit

[GitHub] spark issue #22612: [SPARK-24958] Add executors' process tree total memory i...

2018-12-07 Thread rezasafi
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/22612 Thank you @squito for the reviews. I hope we can merge this soon if there is no concern from @tgravescs @dhruve @edwinalu @LantaoJin. Thanks every one in advance. ---

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21465 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23221: [SPARK-24243][CORE] Expose exceptions from InProc...

2018-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23221 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21849: [SPARK-24243][CORE] Expose exceptions from InProc...

2018-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21849 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21465: [SPARK-24333][ML][PYTHON]Add fit with validation set to ...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21465 **[Test build #99838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99838/testReport)** for PR 21465 at commit

[GitHub] spark pull request #21465: [SPARK-24333][ML][PYTHON]Add fit with validation ...

2018-12-07 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21465#discussion_r239904265 --- Diff: python/pyspark/ml/param/shared.py --- @@ -814,3 +814,25 @@ def getDistanceMeasure(self): """ return

[GitHub] spark issue #23252: [SPARK-26239] File-based secret key loading for SASL.

2018-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23252 (Also, another nit, Spark authentication is not necessarily SASL-based.) --- - To unsubscribe, e-mail:

[GitHub] spark issue #23221: [SPARK-24243][CORE] Expose exceptions from InProcessAppH...

2018-12-07 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23221 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22904: [SPARK-25887][K8S] Configurable K8S context suppo...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22904#discussion_r239903144 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -67,8 +66,16 @@

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239898432 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -380,6 +400,12 @@ private[spark] class SecurityManager( } }

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239899856 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -112,12 +112,14 @@

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r23991 --- Diff: core/src/test/scala/org/apache/spark/SecurityManagerSuite.scala --- @@ -440,12 +473,27 @@ class SecurityManagerSuite extends SparkFunSuite with

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239900712 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -24,7 +24,7 @@ import

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239897958 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -367,11 +372,26 @@ private[spark] class SecurityManager( case _

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239900460 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStepSuite.scala --- @@ -158,6 +161,25

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239900826 --- Diff: core/src/test/scala/org/apache/spark/SecurityManagerSuite.scala --- @@ -18,8 +18,11 @@ package org.apache.spark import

[GitHub] spark issue #23252: [SPARK-26239] File-based secret key loading for SASL.

2018-12-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/23252 How is this file protected in kubernetes? I actually don't like this idea at least for yarn and other deployments, I see people abusing it (accidentally) and using it in non-secure manner.

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/23255 For the reviewers, we need to merge this to 2.3, 2.4 and master. Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23255 **[Test build #99836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99836/testReport)** for PR 23255 at commit

[GitHub] spark issue #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23072 **[Test build #99837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99837/testReport)** for PR 23072 at commit

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23255 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23072 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned t...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23255 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23072: [SPARK-19827][R]spark.ml R API for PIC

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23072 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23174: [SPARK-26194][k8s] Auto generate auth secret for ...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23174#discussion_r239895212 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -87,44 +88,61 @@

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-12-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r239895028 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -95,9 +77,116 @@ case class

[GitHub] spark pull request #23255: [SPARK-26307] [SQL] Fix CTAS when INSERT a partit...

2018-12-07 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/23255 [SPARK-26307] [SQL] Fix CTAS when INSERT a partitioned table using Hive serde ## What changes were proposed in this pull request? This was a Spark 2.3 regression introduced in

[GitHub] spark pull request #23174: [SPARK-26194][k8s] Auto generate auth secret for ...

2018-12-07 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/23174#discussion_r239894356 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -87,44 +88,61 @@

[GitHub] spark pull request #23174: [SPARK-26194][k8s] Auto generate auth secret for ...

2018-12-07 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23174#discussion_r239892984 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala --- @@ -87,44 +88,61 @@

[GitHub] spark issue #17673: [SPARK-20372] [ML] Word2Vec Continuous Bag of Words mode...

2018-12-07 Thread ngopal
Github user ngopal commented on the issue: https://github.com/apache/spark/pull/17673 When can we anticipate this branch being merged? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #23253: [SPARK-26303][SQL] Return partial results for bad...

2018-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23253#discussion_r239890397 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2563,4 +2563,18 @@ class JsonSuite extends

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239889212 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -177,32 +245,47 @@ object StringIndexer extends

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239887472 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +159,60 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239889869 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -310,11 +439,23 @@ object StringIndexerModel extends

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239885097 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -79,26 +81,53 @@ private[feature] trait StringIndexerBase extends

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239888373 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +159,60 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239887259 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +159,60 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239888591 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +159,60 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark pull request #20146: [SPARK-11215][ML] Add multiple columns support to...

2018-12-07 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/20146#discussion_r239885459 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -130,21 +159,60 @@ class StringIndexer @Since("1.4.0") (

[GitHub] spark issue #23208: [SPARK-25530][SQL] data source v2 API refactor (batch wr...

2018-12-07 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/23208 @cloud-fan, what are you suggesting to use as a design? If you think this shouldn't mirror the read side, then let's be clear on what it should look like. Maybe that's a design doc, or maybe that's

[GitHub] spark pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (b...

2018-12-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23208#discussion_r239889152 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala --- @@ -17,52 +17,49 @@ package

[GitHub] spark pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (b...

2018-12-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23208#discussion_r239888975 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/SupportsBatchWrite.java --- @@ -25,14 +25,14 @@ import

[GitHub] spark pull request #23208: [SPARK-25530][SQL] data source v2 API refactor (b...

2018-12-07 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23208#discussion_r239888795 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/TableProvider.java --- @@ -25,7 +25,10 @@ * The base interface for v2 data sources

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-12-07 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r239887300 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -95,9 +77,116 @@ case class

[GitHub] spark issue #23252: [SPARK-26239] File-based secret key loading for SASL.

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23252 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23252: [SPARK-26239] File-based secret key loading for SASL.

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23252 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23252: [SPARK-26239] File-based secret key loading for SASL.

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23252 **[Test build #99835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99835/testReport)** for PR 23252 at commit

[GitHub] spark pull request #23252: [SPARK-26239] File-based secret key loading for S...

2018-12-07 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/23252#discussion_r239884995 --- Diff: core/src/test/scala/org/apache/spark/SecurityManagerSuite.scala --- @@ -440,12 +473,27 @@ class SecurityManagerSuite extends SparkFunSuite with

[GitHub] spark issue #23252: [SPARK-26239] File-based secret key loading for SASL.

2018-12-07 Thread mccheah
Github user mccheah commented on the issue: https://github.com/apache/spark/pull/23252 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23226: [SPARK-26286][TEST] Add MAXIMUM_PAGE_SIZE_BYTES exceptio...

2018-12-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23226 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17654: [SPARK-20351] [ML] Add trait hasTrainingSummary to repla...

2018-12-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17654 Gentle ping here, it's out of sync with master if you've got the time to bring it up to date that would be great. --- - To

[GitHub] spark issue #22273: [SPARK-25272][PYTHON][TEST] Add test to better indicate ...

2018-12-07 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22273 How about if we checked an env variable like EXPECT_ARROW (and set it when tests are running in Jenkins) and if it's set but we don't have the required versions installed fail? That way we know if

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #99834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99834/testReport)** for PR 22957 at commit

[GitHub] spark issue #23239: [SPARK-26021][SQL][followup] only deal with NaN and -0.0...

2018-12-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23239 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22957: [SPARK-25951][SQL] Ignore aliases for distributions and ...

2018-12-07 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22957 @cloud-fan @gatorsmile I updated the PR according to the previous suggestions and added a new dedicated test suite. May you please review this again? Thanks. ---

[GitHub] spark issue #23224: [SPARK-26277][SQL][TEST] WholeStageCodegen metrics shoul...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23224 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99829/ Test PASSed. ---

[GitHub] spark issue #23224: [SPARK-26277][SQL][TEST] WholeStageCodegen metrics shoul...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23224 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23224: [SPARK-26277][SQL][TEST] WholeStageCodegen metrics shoul...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23224 **[Test build #99829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99829/testReport)** for PR 23224 at commit

[GitHub] spark pull request #23225: [SPARK-26287][CORE]Don't need to create an empty ...

2018-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23225#discussion_r239872296 --- Diff: core/src/test/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriterSuite.java --- @@ -235,6 +235,7 @@ public void writeEmptyIterator()

[GitHub] spark pull request #23225: [SPARK-26287][CORE]Don't need to create an empty ...

2018-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23225#discussion_r239860885 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -161,6 +161,10 @@ private void writeSortedFile(boolean

[GitHub] spark pull request #23225: [SPARK-26287][CORE]Don't need to create an empty ...

2018-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23225#discussion_r239859088 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -161,6 +161,10 @@ private void writeSortedFile(boolean

[GitHub] spark issue #23253: [SPARK-26303][SQL] Return partial results for bad JSON r...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23253 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99828/ Test PASSed. ---

[GitHub] spark issue #23253: [SPARK-26303][SQL] Return partial results for bad JSON r...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23253 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23253: [SPARK-26303][SQL] Return partial results for bad JSON r...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23253 **[Test build #99828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99828/testReport)** for PR 23253 at commit

[GitHub] spark pull request #23245: [SPARK-26060][SQL][FOLLOW-UP] Rename the config n...

2018-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23245 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22331: [SPARK-25331][SS] Make FileStreamSink ignore part...

2018-12-07 Thread misutoth
Github user misutoth closed the pull request at: https://github.com/apache/spark/pull/22331 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22331: [SPARK-25331][SS] Make FileStreamSink ignore partitions ...

2018-12-07 Thread misutoth
Github user misutoth commented on the issue: https://github.com/apache/spark/pull/22331 So I am considering this as the recommended way to read a file sink's output. If there is a need to include the protocol in this PR as an alternative we can still reopen it. ---

[GitHub] spark issue #23245: [SPARK-26060][SQL][FOLLOW-UP] Rename the config name.

2018-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23245 Thank you all. Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23253: [SPARK-26303][SQL] Return partial results for bad...

2018-12-07 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/23253#discussion_r239853384 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -37,6 +37,8 @@ displayTitle: Spark SQL Upgrading Guide - In Spark version 2.4 and earlier,

[GitHub] spark issue #22952: [SPARK-20568][SS] Provide option to clean up completed f...

2018-12-07 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22952 @gaborgsomogyi @steveloughran Please take a look at 17b9b9a043ead0d448048c88b30f544228bd230b which just leverages GlobFilter. You may find that when the depth of archive path is more than

[GitHub] spark issue #22952: [SPARK-20568][SS] Provide option to clean up completed f...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22952 **[Test build #99833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99833/testReport)** for PR 22952 at commit

[GitHub] spark pull request #23253: [SPARK-26303][SQL] Return partial results for bad...

2018-12-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23253#discussion_r239846601 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -37,6 +37,8 @@ displayTitle: Spark SQL Upgrading Guide - In Spark version 2.4 and earlier,

[GitHub] spark pull request #23253: [SPARK-26303][SQL] Return partial results for bad...

2018-12-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23253#discussion_r239846300 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -37,6 +37,8 @@ displayTitle: Spark SQL Upgrading Guide - In Spark version 2.4 and earlier,

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22683: [SPARK-25696] The storage memory displayed on spark Appl...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22683 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99827/ Test PASSed. ---

[GitHub] spark issue #22683: [SPARK-25696] The storage memory displayed on spark Appl...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22683 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #99832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99832/testReport)** for PR 20146 at commit

[GitHub] spark issue #22683: [SPARK-25696] The storage memory displayed on spark Appl...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22683 **[Test build #99827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99827/testReport)** for PR 22683 at commit

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-12-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20146 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23224: [SPARK-26277][SQL][TEST] WholeStageCodegen metric...

2018-12-07 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23224#discussion_r239837818 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -80,8 +80,10 @@ class SQLMetricsSuite extends

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23242 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23242 **[Test build #99831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99831/testReport)** for PR 23242 at commit

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23242 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99831/ Test FAILed. ---

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23242 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23242 **[Test build #99831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99831/testReport)** for PR 23242 at commit

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23242 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23207 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99825/ Test PASSed. ---

[GitHub] spark issue #23207: [SPARK-26193][SQL] Implement shuffle write metrics in SQ...

2018-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23207 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23242: [SPARK-26285][CORE] accumulator metrics sources for Long...

2018-12-07 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/23242 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

<    1   2   3   4   5   >