[GitHub] spark issue #21052: [SPARK-23799][SQL] FilterEstimation.evaluateInSet produc...

2018-04-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21052 Let me revert it from Spark 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21132: [SPARK-24029][core] Follow up: set SO_REUSEADDR on the s...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21132 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21049: [SPARK-23957][SQL] Remove redundant sort operators from ...

2018-04-23 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/21049 @dilipbiswal Thanks! Although Spark doesn't necessarily parse the query in the `from` clause as a subquery, is it fair to say it plans it as one? (Since the planner puts the alias under a

[GitHub] spark issue #21052: [SPARK-23799][SQL] FilterEstimation.evaluateInSet produc...

2018-04-23 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/21052 @gatorsmile this broke 2.3 compilation. https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/job/spark-branch-2.3-compile-maven-hadoop-2.6/638/ ---

[GitHub] spark issue #21132: [SPARK-24029][core] Follow up: set SO_REUSEADDR on the s...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21132 **[Test build #89740 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89740/testReport)** for PR 21132 at commit

[GitHub] spark issue #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Support cus...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20937 **[Test build #89741 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89741/testReport)** for PR 20937 at commit

[GitHub] spark issue #21132: [SPARK-24029][core] Follow up: set SO_REUSEADDR on the s...

2018-04-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21132 @HyukjinKwon @jerryshao since you looked at the other patch. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21132: [SPARK-24029][core] Follow up: set SO_REUSEADDR o...

2018-04-23 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/21132 [SPARK-24029][core] Follow up: set SO_REUSEADDR on the server socket. "childOption" is for the remote connections, not for the server socket that actually listens for incoming connections. You

[GitHub] spark pull request #20998: [SPARK-23888][CORE] correct the comment of hasAtt...

2018-04-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20998 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21131 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21131 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2599/

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20998 merged to master, thanks @Ngone51 . I also updated the commit msg some before committing, I thought it best to focus on the eventual change, figured it wasn't worth bugging you for another update

[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21131 **[Test build #89739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89739/testReport)** for PR 21131 at commit

[GitHub] spark issue #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API for PIC

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21119 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2598/

[GitHub] spark issue #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API for PIC

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21119 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21131: [SPARK-23433][CORE] Late zombie task completions ...

2018-04-23 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/21131 [SPARK-23433][CORE] Late zombie task completions update all tasksets Fetch failure lead to multiple tasksets which are active for a given stage. A late completion from an earlier attempt of the

[GitHub] spark issue #21081: [SPARK-23975][ML]Allow Clustering to take Arrays of Doub...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21081 **[Test build #89738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89738/testReport)** for PR 21081 at commit

[GitHub] spark issue #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API for PIC

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21119 **[Test build #89737 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89737/testReport)** for PR 21119 at commit

[GitHub] spark issue #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API for PIC

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21119 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2597/

[GitHub] spark issue #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API for PIC

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21119 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21119: [SPARK-19826][ML][PYTHON]add spark.ml Python API for PIC

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21119 **[Test build #89735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89735/testReport)** for PR 21119 at commit

[GitHub] spark issue #21081: [SPARK-23975][ML]Allow Clustering to take Arrays of Doub...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21081 **[Test build #89736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89736/testReport)** for PR 21081 at commit

[GitHub] spark pull request #20907: [SPARK-11237][ML] Add pmml export for k-means in ...

2018-04-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20907 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20907: [SPARK-11237][ML] Add pmml export for k-means in Spark M...

2018-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/20907 LGTM too! +1 on the documentation which can be a followup PR. Merged into master, and thanks. DB Tsai | Siri Open Source Technologies |  Apple, Inc ---

[GitHub] spark pull request #21124: [SPARK-23004][SS] Ensure StateStore.commit is cal...

2018-04-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21124 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Supp...

2018-04-23 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r183524310 --- Diff: python/pyspark/sql/readwriter.py --- @@ -773,6 +776,8 @@ def json(self, path, mode=None, compression=None, dateFormat=None, timestampForm

[GitHub] spark pull request #21078: [SPARK-23990][ML] Instruments logging improvement...

2018-04-23 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/21078#discussion_r183514966 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/Instrumentation.scala --- @@ -157,34 +161,55 @@ private[spark] object Instrumentation {

[GitHub] spark pull request #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Supp...

2018-04-23 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r183512588 --- Diff: python/pyspark/sql/readwriter.py --- @@ -773,6 +776,8 @@ def json(self, path, mode=None, compression=None, dateFormat=None, timestampForm

[GitHub] spark pull request #20937: [SPARK-23094][SPARK-23723][SPARK-23724][SQL] Supp...

2018-04-23 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/20937#discussion_r183511452 --- Diff: python/pyspark/sql/readwriter.py --- @@ -237,6 +237,9 @@ def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-04-23 Thread henryr
Github user henryr commented on the issue: https://github.com/apache/spark/pull/21070 This looks pretty good to me - are there any committers that can give it a (hopefully) final review? --- - To unsubscribe,

[GitHub] spark pull request #19887: [SPARK-21168] KafkaRDD should always set kafka cl...

2018-04-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19887 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21128: [SPARK-24053][CORE] Support add subdirectory named as us...

2018-04-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21128 Moreover, you can set `spark.yarn.stagingDir` to whatever you want, and may even reference env variables or system properties. e.g. ``` spark.yarn.stagingDir=/tmp/${system:user.name}

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19887 Merging to master, thanks @liu-zhaokun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19887 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89733/ Test PASSed. ---

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19887 **[Test build #89733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89733/testReport)** for PR 19887 at commit

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2596/

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20998 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89728/ Test PASSed. ---

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20998 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #89734 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89734/testReport)** for PR 21123 at commit

[GitHub] spark issue #20998: [SPARK-23888][CORE] correct the comment of hasAttemptOnH...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20998 **[Test build #89728 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89728/testReport)** for PR 20998 at commit

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21123 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21113 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89730/ Test FAILed. ---

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21113 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21113: [MINOR][DOCS] Fix comments of SQLExecution#withExecution...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21113 **[Test build #89730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89730/testReport)** for PR 21113 at commit

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19887 **[Test build #89733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89733/testReport)** for PR 19887 at commit

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19887 Seems ok to me, long as it passes retest --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19887: [SPARK-21168] KafkaRDD should always set kafka clientId.

2018-04-23 Thread koeninger
Github user koeninger commented on the issue: https://github.com/apache/spark/pull/19887 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20280: [SPARK-22232][PYTHON][SQL] Fixed Row pickling to include...

2018-04-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20280 Thanks @HyukjinKwon and @felixcheung , I'm a bit worried too that this might break someones code, but it doesn't affect `createDataFrame` from `Row`s, it's only when the Row is serialized like

[GitHub] spark pull request #21116: [SPARK-24038][SS] Refactor continuous writing to ...

2018-04-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21116#discussion_r183489854 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/WriteToContinuousDataSourceExec.scala --- @@ -0,0 +1,126 @@ +/*

[GitHub] spark pull request #21116: [SPARK-24038][SS] Refactor continuous writing to ...

2018-04-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21116#discussion_r183489611 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/WriteToContinuousDataSourceExec.scala --- @@ -0,0 +1,126 @@ +/*

[GitHub] spark pull request #21116: [SPARK-24038][SS] Refactor continuous writing to ...

2018-04-23 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/21116#discussion_r183488923 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/WriteToContinuousDataSourceExec.scala --- @@ -0,0 +1,126 @@ +/*

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89732/ Test FAILed. ---

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #89732 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89732/testReport)** for PR 21123 at commit

[GitHub] spark issue #21116: [SPARK-24038][SS] Refactor continuous writing to its own...

2018-04-23 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/21116 LGTM! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21126: [SPARK-24050][SS] Calculate input / processing rates cor...

2018-04-23 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/21126 @brkyvz @jose-torres --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89727/ Test FAILed. ---

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #89727 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89727/testReport)** for PR 20146 at commit

[GitHub] spark issue #19404: [SPARK-21760] [Streaming] Fix for Structured streaming t...

2018-04-23 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/19404 BTW, perf wise: hflush() is required to block until the flush has got to the store (visible to others), and with hsync actually saved to the durable store. So it will take time, but if you

[GitHub] spark issue #21125: [Spark-24024][ML] Fix poisson deviance calculations in G...

2018-04-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21125 LGTM, merged into master. Thanks. DB Tsai | Siri Open Source Technologies |  Apple, Inc --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19404: [SPARK-21760] [Streaming] Fix for Structured stre...

2018-04-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19404#discussion_r183482802 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/CompactibleFileStreamLog.scala --- @@ -139,6 +139,9 @@ abstract class

[GitHub] spark pull request #19404: [SPARK-21760] [Streaming] Fix for Structured stre...

2018-04-23 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19404#discussion_r183480609 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala --- @@ -123,6 +123,7 @@ class HDFSMetadataLog[T <:

[GitHub] spark issue #19404: [SPARK-21760] [Streaming] Fix for Structured streaming t...

2018-04-23 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/19404 Problem here is that a stream which doesn't implement hflush/hsync is required to throw an exception; it's a way of guaranteeing that if hsync/hflush does complete, the action has done what

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89725/ Test PASSed. ---

[GitHub] spark pull request #21125: [Spark-24024][ML] Fix poisson deviance calculatio...

2018-04-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21125 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #89725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89725/testReport)** for PR 21123 at commit

[GitHub] spark issue #21072: [SPARK-23973][SQL] Remove consecutive Sorts

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21072 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89726/ Test PASSed. ---

[GitHub] spark issue #21072: [SPARK-23973][SQL] Remove consecutive Sorts

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21072 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21072: [SPARK-23973][SQL] Remove consecutive Sorts

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21072 **[Test build #89726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89726/testReport)** for PR 21072 at commit

[GitHub] spark pull request #21125: [Spark-24024][ML] Fix poisson deviance calculatio...

2018-04-23 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/21125#discussion_r183475337 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -782,8 +782,12 @@ object GeneralizedLinearRegression

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20146 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89713/ Test PASSed. ---

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20146 **[Test build #89713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89713/testReport)** for PR 20146 at commit

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89724/ Test PASSed. ---

[GitHub] spark pull request #20535: [SPARK-23341][SQL] define some standard options f...

2018-04-23 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r183470867 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -193,10 +196,13 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18903 **[Test build #4154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4154/testReport)** for PR 18903 at commit

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #89724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89724/testReport)** for PR 21123 at commit

[GitHub] spark pull request #20923: [SPARK-23807][BUILD] Add Hadoop 3.1 profile with ...

2018-04-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20923#discussion_r183470258 --- Diff: pom.xml --- @@ -2671,6 +2671,15 @@ + + hadoop-3.1 --- End diff -- +1 for skipping

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add a executorAllocationRatio parame...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19881 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add a executorAllocationRatio parame...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19881 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89712/ Test PASSed. ---

[GitHub] spark issue #19881: [SPARK-22683][CORE] Add a executorAllocationRatio parame...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19881 **[Test build #89712 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89712/testReport)** for PR 19881 at commit

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89722/ Test PASSed. ---

[GitHub] spark issue #21127: [SPARK-24052][CORE][UI] Add spark version information on...

2018-04-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21127 The Spark version already shows next to the Spark logo on every page in the UI? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20701: [SPARK-23528][ML] Add numIter to ClusteringSummary

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20701 **[Test build #89722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89722/testReport)** for PR 20701 at commit

[GitHub] spark issue #21128: [SPARK-24053][CORE] Support add subdirectory named as us...

2018-04-23 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21128 > When we have multiple users on the same cluster Then the staging directory would be created under the respective user already (it's created under the user's home directory). I have no idea

[GitHub] spark issue #21018: [SPARK-23880][SQL] Do not trigger any jobs for caching d...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21018 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21018: [SPARK-23880][SQL] Do not trigger any jobs for caching d...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21018 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89718/ Test PASSed. ---

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21018: [SPARK-23880][SQL] Do not trigger any jobs for caching d...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21018 **[Test build #89718 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89718/testReport)** for PR 21018 at commit

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21123 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89717/ Test PASSed. ---

[GitHub] spark issue #21123: [SPARK-24045][SQL]Create base class for file data source...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21123 **[Test build #89717 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89717/testReport)** for PR 21123 at commit

[GitHub] spark issue #21128: [SPARK-24053][CORE] Support add subdirectory named as us...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21128 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89715/ Test PASSed. ---

[GitHub] spark issue #21128: [SPARK-24053][CORE] Support add subdirectory named as us...

2018-04-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21128 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21128: [SPARK-24053][CORE] Support add subdirectory named as us...

2018-04-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21128 **[Test build #89715 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89715/testReport)** for PR 21128 at commit

<    1   2   3   4   5   6   >