[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19689 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83589 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83589/testReport)** for PR 19689 at commit

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19690 **[Test build #83590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83590/testReport)** for PR 19690 at commit

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19607 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83587/ Test FAILed. ---

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19690 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83588/ Test FAILed. ---

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19607 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83586/ Test FAILed. ---

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83585/ Test FAILed. ---

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19607 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19683 **[Test build #83587 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83587/testReport)** for PR 19683 at commit

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83584/testReport)** for PR 19689 at commit

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19607 **[Test build #83586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83586/testReport)** for PR 19607 at commit

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19681 **[Test build #83585 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83585/testReport)** for PR 19681 at commit

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19690 **[Test build #83588 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83588/testReport)** for PR 19690 at commit

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19683 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19681 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83584/ Test FAILed. ---

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19690 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/19690 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83591 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83591/testReport)** for PR 19689 at commit

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19607 **[Test build #83592 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83592/testReport)** for PR 19607 at commit

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-08 Thread mpetruska
Github user mpetruska commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149600715 --- Diff: mllib/src/main/scala/org/apache/spark/ml/extensions/seq/package.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19683: [SPARK-21657][SQL] optimize explode quadratic memory con...

2017-11-08 Thread uzadude
Github user uzadude commented on the issue: https://github.com/apache/spark/pull/19683 Do you understand this failure? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19694: [SPARK-22470][DOC][SQL] functions.hash is also used inte...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19694 **[Test build #83595 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83595/testReport)** for PR 19694 at commit

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r149655172 --- Diff: python/pyspark/sql/session.py --- @@ -557,7 +577,13 @@ def createDataFrame(self, data, schema=None, samplingRatio=None, verifySchema=Tr

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19607 **[Test build #83593 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83593/testReport)** for PR 19607 at commit

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19607 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19640: [SPARK-16986][CORE][WEB-UI] Support configure his...

2017-11-08 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19640#discussion_r149661121 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala --- @@ -426,6 +426,8 @@ class SparkHadoopUtil extends Logging {

[GitHub] spark issue #19573: [SPARK-22350][SQL] select grouping__id from subquery

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19573 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19573: [SPARK-22350][SQL] select grouping__id from subquery

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19573 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83594/ Test PASSed. ---

[GitHub] spark issue #19685: [SPARK-19759][ML] not using blas in ALSModel.predict for...

2017-11-08 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19685 @srowen I tried enabling native BLAS, but native BLAS implementation is still much slower: average on 10 runs is 2529,922753 ms against 515,510185 ms of the for loop. As a reference, I am using a

[GitHub] spark issue #19694: [SPARK-22470][DOC][SQL] functions.hash is also used inte...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19694 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19694: [SPARK-22470][DOC][SQL] functions.hash is also used inte...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19694 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83595/ Test PASSed. ---

[GitHub] spark issue #19688: [SPARK-22466][Spark Submit]export SPARK_CONF_DIR while c...

2017-11-08 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19688 @jerryshao PR description added. I notice that this is small, but they seem to be different issues. --- - To unsubscribe,

[GitHub] spark issue #19675: [SPARK-14540][BUILD] Support Scala 2.12 closures and Jav...

2017-11-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19675 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83591 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83591/testReport)** for PR 19689 at commit

[GitHub] spark issue #19662: [SPARK-22446][SQL][ML] Declare StringIndexerModel indexe...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19662 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19607 **[Test build #83592 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83592/testReport)** for PR 19607 at commit

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-08 Thread zivanfi
Github user zivanfi commented on the issue: https://github.com/apache/spark/pull/19250 Hive and Impala introduced the following workaround for timestamp interoperability a long ago: The footer of the Parquet file contains metadata about the library that wrote the file. For Hive and

[GitHub] spark pull request #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof d...

2017-11-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19695#discussion_r149649386 --- Diff: dev/create-release/release-build.sh --- @@ -130,6 +130,10 @@ else fi fi +LSOF=lsof +if ! hash $LSOF 2>/dev/null; then

[GitHub] spark issue #19694: [SPARK-22470][DOC][SQL] functions.hash is also used inte...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19694 **[Test build #83595 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83595/testReport)** for PR 19694 at commit

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-11-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19459#discussion_r149626358 --- Diff: python/pyspark/serializers.py --- @@ -213,7 +213,15 @@ def __repr__(self): return "ArrowSerializer" -def

[GitHub] spark pull request #19687: [SPARK-19644][SQL]Clean up Scala reflection garba...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19687#discussion_r149632280 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -370,7 +372,7 @@ class

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83591/ Test PASSed. ---

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof d...

2017-11-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19695#discussion_r149651615 --- Diff: dev/create-release/release-build.sh --- @@ -130,6 +130,10 @@ else fi fi +LSOF=lsof +if ! hash $LSOF 2>/dev/null;

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19250 IIUC, using the `parquet.timezone-adjustment` table property requires changing the writer. e.g. Impala creates a table and Hive wants to write data to it, then Hive needs to write

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19607 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83593/ Test PASSed. ---

[GitHub] spark issue #19693: [MINOR][CORE] Improved statistical shuffle write time

2017-11-08 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19693 Personally I don't think we should take the file open/close time into account, cc @cloud-fan . --- - To unsubscribe,

[GitHub] spark issue #19685: [SPARK-19759][ML] not using blas in ALSModel.predict for...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19685 **[Test build #83598 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83598/testReport)** for PR 19685 at commit

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19607 **[Test build #83593 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83593/testReport)** for PR 19607 at commit

[GitHub] spark pull request #19664: [SPARK-22442][SQL] ScalaReflection should produce...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19664#discussion_r149633100 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala --- @@ -335,4 +338,17 @@ class ScalaReflectionSuite

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19690 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19690 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83590/ Test FAILed. ---

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19689 **[Test build #83589 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83589/testReport)** for PR 19689 at commit

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19689: [SPARK-22462][SQL] Make rdd-based actions in Dataset tra...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83589/ Test PASSed. ---

[GitHub] spark pull request #19156: [SPARK-19634][SQL][ML][FOLLOW-UP] Improve interfa...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19156#discussion_r149641764 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -527,27 +570,28 @@ private[ml] object SummaryBuilderImpl extends Logging

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19695 cc @holdenk and @xynny, mind taking a look please? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19573: [SPARK-22350][SQL] select grouping__id from subquery

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19573 **[Test build #83594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83594/testReport)** for PR 19573 at commit

[GitHub] spark pull request #19687: [SPARK-19644][SQL]Clean up Scala reflection garba...

2017-11-08 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19687#discussion_r149627183 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -441,4 +443,28 @@ class

[GitHub] spark issue #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for addJar ...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19643 **[Test build #83596 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83596/testReport)** for PR 19643 at commit

[GitHub] spark pull request #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for ...

2017-11-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19643#discussion_r149641601 --- Diff: python/pyspark/context.py --- @@ -860,6 +860,23 @@ def addPyFile(self, path): import importlib

[GitHub] spark pull request #19156: [SPARK-19634][SQL][ML][FOLLOW-UP] Improve interfa...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19156#discussion_r149641398 --- Diff: mllib/src/main/scala/org/apache/spark/ml/stat/Summarizer.scala --- @@ -94,46 +97,86 @@ object Summarizer extends Logging { * - min: the

[GitHub] spark pull request #19692: [SPARK-22469][SQL] Accuracy problem in comparison...

2017-11-08 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19692#discussion_r149659840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -137,6 +137,8 @@ object TypeCoercion {

[GitHub] spark issue #19649: [SPARK-22405][SQL] Add more ExternalCatalogEvent

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19649 not sure, but maybe do it in a new PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19692: [SPARK-22469][SQL] Accuracy problem in comparison...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19692#discussion_r149639379 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -137,6 +137,8 @@ object TypeCoercion {

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19607 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior of time...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19607 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83592/ Test PASSed. ---

[GitHub] spark pull request #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof d...

2017-11-08 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19695 [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not exists in release-build.sh ## What changes were proposed in this pull request? This PR proposes to use `/usr/sbin/lsof` if

[GitHub] spark pull request #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof d...

2017-11-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19695#discussion_r149652048 --- Diff: dev/create-release/release-build.sh --- @@ -130,6 +130,10 @@ else fi fi +LSOF=lsof +if ! hash $LSOF 2>/dev/null;

[GitHub] spark pull request #19607: [WIP][SPARK-22395][SQL][PYTHON] Fix the behavior ...

2017-11-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/19607#discussion_r149661483 --- Diff: python/pyspark/sql/session.py --- @@ -557,7 +577,13 @@ def createDataFrame(self, data, schema=None, samplingRatio=None, verifySchema=Tr

[GitHub] spark issue #19688: [SPARK-22466][Spark Submit]export SPARK_CONF_DIR while c...

2017-11-08 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19688 not familiar with dos cmd, plz review again @jiangxb1987 @srowen , thanks --- - To unsubscribe, e-mail:

[GitHub] spark issue #19687: [SPARK-19644][SQL]Clean up Scala reflection garbage afte...

2017-11-08 Thread ManchesterUnited16
Github user ManchesterUnited16 commented on the issue: https://github.com/apache/spark/pull/19687 java.io.NotSerializableException: scala.reflect.api.TypeTags$PredefTypeCreator --- - To unsubscribe, e-mail:

[GitHub] spark issue #19687: [SPARK-19644][SQL]Clean up Scala reflection garbage afte...

2017-11-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19687 LGTM, is it targeted for branch 2.2 too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19687: [SPARK-19644][SQL]Clean up Scala reflection garba...

2017-11-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19687#discussion_r149630182 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -441,4 +443,28 @@ class

[GitHub] spark issue #19690: [SPARK-22467]Added a switch to support whether `stdout_s...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19690 **[Test build #83590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83590/testReport)** for PR 19690 at commit

[GitHub] spark issue #19573: [SPARK-22350][SQL] select grouping__id from subquery

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19573 **[Test build #83594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83594/testReport)** for PR 19573 at commit

[GitHub] spark pull request #19479: [SPARK-17074] [SQL] Generate equi-height histogra...

2017-11-08 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/19479#discussion_r149678221 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala --- @@ -275,6 +317,98 @@ object ColumnStat extends

[GitHub] spark issue #19696: [SPARK-22473][TEST] Replace deprecated AsyncAssertions.W...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19696 **[Test build #83600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83600/testReport)** for PR 19696 at commit

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19695 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83597/ Test PASSed. ---

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19695 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-11-08 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19479 > We should also consider how to show the histogram in ANALYZE COLUMN, for debug purpose. Do you mean show the histogram in DESC COLUMN command? ---

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-11-08 Thread zivanfi
Github user zivanfi commented on the issue: https://github.com/apache/spark/pull/19250 Yes, you understand correctly, the table property affects both the read path and the write path, while the current workaround used by Hive and Impala only affects the read path. (Both are

[GitHub] spark issue #19688: [SPARK-22466][Spark Submit]export SPARK_CONF_DIR while c...

2017-11-08 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19688 cc @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19685: [SPARK-19759][ML] not using blas in ALSModel.predict for...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19685 **[Test build #83598 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83598/testReport)** for PR 19685 at commit

[GitHub] spark pull request #19696: [SPARK-22473][TEST] Replace deprecated AsyncAsser...

2017-11-08 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/19696 [SPARK-22473][TEST] Replace deprecated AsyncAssertions.Waiter and methods of java.sql.Date ## What changes were proposed in this pull request? In `spark-sql` module tests there are

[GitHub] spark issue #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for addJar ...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19643 **[Test build #83596 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83596/testReport)** for PR 19643 at commit

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-08 Thread mpetruska
Github user mpetruska commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149690477 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/NGram.scala --- @@ -42,11 +42,22 @@ class NGram @Since("1.5.0") (@Since("1.5.0") override val

[GitHub] spark pull request #19659: [SPARK-19668][ML] Multiple NGram sizes

2017-11-08 Thread mpetruska
Github user mpetruska commented on a diff in the pull request: https://github.com/apache/spark/pull/19659#discussion_r149696337 --- Diff: mllib/src/main/scala/org/apache/spark/ml/extensions/seq/package.scala --- @@ -0,0 +1,121 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #19685: [SPARK-19759][ML] not using blas in ALSModel.predict for...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19685 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19685: [SPARK-19759][ML] not using blas in ALSModel.predict for...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19685 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83598/ Test PASSed. ---

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19479 **[Test build #83599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83599/testReport)** for PR 19479 at commit

[GitHub] spark issue #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for addJar ...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19643 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19643: [SPARK-11421][CORE][PYTHON][R] Added ability for addJar ...

2017-11-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19643 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83596/ Test PASSed. ---

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19695 **[Test build #83597 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83597/testReport)** for PR 19695 at commit

[GitHub] spark issue #19695: [SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not...

2017-11-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19695 I realized that some OSs puts `lsof` into /usr/bin and other OSs put `lsof` into `/usr/sbin`. Could you apply this to `dev/run-tests.py`, too? ---

[GitHub] spark issue #19555: [SPARK-22133][DOCS] Documentation for Mesos Reject Offer...

2017-11-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19555 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19681: [SPARK-20652][sql] Store SQL UI data in the new app stat...

2017-11-08 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19681 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

  1   2   3   4   5   >