[GitHub] spark issue #19486: [SPARK-22268][BUILD] Fix lint-java

2017-10-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19486 Ping @ash211 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19511: [SPARK-22293][SQL] Avoid unnecessary traversal in...

2017-10-17 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/19511#discussion_r145041400 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -890,32 +890,39 @@ class Analyzer( /**

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19505 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18664 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19511: [SPARK-22293][SQL] Avoid unnecessary traversal in Resolv...

2017-10-17 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/19511 Hi, @cloud-fan @gatorsmile. Would you mind take a look? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19511: [SPARK-22293][SQL] Avoid unnecessary traversal in Resolv...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19511 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19480: [SPARK-22226][SQL] splitExpression can create too many m...

2017-10-17 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19480 @bdrillard thanks but we removed the "guilty" test case for many reasons. Thank you anyway. --- - To unsubscribe, e-mail:

[GitHub] spark issue #17290: [SPARK-16599][CORE] java.util.NoSuchElementException: No...

2017-10-17 Thread aphasingnirvana
Github user aphasingnirvana commented on the issue: https://github.com/apache/spark/pull/17290 @srowen So could I open another issue altogether? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19494: [SPARK-22249][SQL] isin with empty list throws exception...

2017-10-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19494 Merged to master/2.2 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19485 Less duplication is good but could we similar contents with http://spark.apache.org/docs/latest/sql-programming-guide.html#json-datasets? It looks the examples are quite different.

[GitHub] spark issue #19363: [SPARK-22224][SQL]Override toString of KeyValue/Relation...

2017-10-17 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19363 hi @cloud-fan jenkins ok --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19374: [SPARK-22145][MESOS] fix supervise with checkpoin...

2017-10-17 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19374#discussion_r145066047 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -276,8 +276,8 @@

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19476 **[Test build #82833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82833/testReport)** for PR 19476 at commit

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19476 **[Test build #82833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82833/testReport)** for PR 19476 at commit

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19505 **[Test build #82831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82831/testReport)** for PR 19505 at commit

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19505 **[Test build #82835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82835/testReport)** for PR 19505 at commit

[GitHub] spark pull request #19354: [SPARK-20992][Scheduler] Add links in documentati...

2017-10-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19354 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19469: [SPARK-22243][DStreams]spark.yarn.jars reload from confi...

2017-10-17 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19469 @ChenjunZou did you get a chance to look at my left comment? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19459#discussion_r145039356 --- Diff: python/pyspark/sql/session.py --- @@ -414,6 +415,43 @@ def _createFromLocal(self, data, schema): data = [schema.toInternal(row)

[GitHub] spark pull request #19511: [SPARK-22293][SQL] Avoid unnecessary traversal in...

2017-10-17 Thread ConeyLiu
GitHub user ConeyLiu opened a pull request: https://github.com/apache/spark/pull/19511 [SPARK-22293][SQL] Avoid unnecessary traversal in ResolveReferences ## What changes were proposed in this pull request? We don't need traverse the children expression to determine whether

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82832/ Test FAILed. ---

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19476 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82833/ Test FAILed. ---

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #82832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82832/testReport)** for PR 18664 at commit

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19505 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19505 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82831/ Test FAILed. ---

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19476 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19505 Change itself LGTM if we are okay to go separating this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82837/testReport)** for PR 18747 at commit

[GitHub] spark issue #19263: [SPARK-22050][CORE] Allow BlockUpdated events to be opti...

2017-10-17 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19263 @vanzin, do you have other comments? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19485: [SPARK-20055] [Docs] Added documentation for load...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19485#discussion_r145055873 --- Diff: docs/sql-programming-guide.md --- @@ -462,7 +462,6 @@ names (`json`, `parquet`, `jdbc`, `orc`, `libsvm`, `csv`, `text`). DataFrames lo

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19485 @gatorsmile WDYT? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Tim...

2017-10-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r145037361 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala --- @@ -55,6 +55,12 @@ object ArrowWriter { case

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Tim...

2017-10-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r145036404 --- Diff: python/pyspark/sql/types.py --- @@ -1619,11 +1619,38 @@ def to_arrow_type(dt): arrow_type = pa.decimal(dt.precision, dt.scale)

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19476 **[Test build #82834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82834/testReport)** for PR 19476 at commit

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19476 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19509: [SPARK-22290][core] Avoid creating Hive delegation token...

2017-10-17 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19509 >The effect of this change is that now it's possible to initialize multiple, non-concurrent SparkContext instances in the same JVM. @vanzin , do we support in now? As I remembered it

[GitHub] spark issue #19419: [SPARK-22188] [CORE] Adding security headers for prevent...

2017-10-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19419 For the content type header, does the UI seem to work normally with it on? I don't think we should have any problem with that restriction or else need to fix it . You could move the defaults to on

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-17 Thread jomach
Github user jomach commented on the issue: https://github.com/apache/spark/pull/19485 Yes I'm viewing the docs with Jekyll. I addressed that on my previous comment. I really don't think we should make a huge example as the json does. It's a csv ... What do you think ?

[GitHub] spark pull request #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on...

2017-10-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19370#discussion_r145039851 --- Diff: bin/find-spark-home.cmd --- @@ -0,0 +1,44 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one

[GitHub] spark issue #19354: [SPARK-20992][Scheduler] Add links in documentation to N...

2017-10-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19354 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #82836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82836/testReport)** for PR 18664 at commit

[GitHub] spark pull request #19494: [SPARK-22249][SQL] isin with empty list throws ex...

2017-10-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19494 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #17290: [SPARK-16599][CORE] java.util.NoSuchElementException: No...

2017-10-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/17290 No, https://issues.apache.org/jira/browse/SPARK-16599 is still open --- - To unsubscribe, e-mail:

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #82838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82838/testReport)** for PR 14151 at commit

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-17 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r145079551 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackendUtil.scala --- @@ -17,10 +17,14 @@

[GitHub] spark issue #19363: [SPARK-22224][SQL]Override toString of KeyValue/Relation...

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19363 merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19505 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19512: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19512 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19514 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #82839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82839/testReport)** for PR 14151 at commit

[GitHub] spark pull request #19514: [SPARK-21551][Python] Increase timeout for Python...

2017-10-17 Thread FRosner
GitHub user FRosner opened a pull request: https://github.com/apache/spark/pull/19514 [SPARK-21551][Python] Increase timeout for PythonRDD.serveIterator Backport of https://github.com/apache/spark/pull/18752 (https://issues.apache.org/jira/browse/SPARK-21551) (cherry

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82838/ Test PASSed. ---

[GitHub] spark issue #19513: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19513 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not hono...

2017-10-17 Thread pmackles
GitHub user pmackles opened a pull request: https://github.com/apache/spark/pull/19515 [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by MesosClusterD… …ispatcher ## What changes were proposed in this pull request? Allow JVM max heap size to be controlled

[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19515 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #82839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82839/testReport)** for PR 14151 at commit

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19476 **[Test build #82834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82834/testReport)** for PR 19476 at commit

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82837/ Test PASSed. ---

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82837/testReport)** for PR 18747 at commit

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-17 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19437#discussion_r145079127 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosFineGrainedSchedulerBackend.scala --- @@ -159,7 +160,8 @@

[GitHub] spark issue #19488: [SPARK-22266][SQL] The same aggregate function was evalu...

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19488 +1, before we figure out the semantic of nondeterministic aggregate functions, let's be careful here. --- - To unsubscribe,

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82836/ Test PASSed. ---

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19505 **[Test build #82835 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82835/testReport)** for PR 19505 at commit

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19505 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82835/ Test PASSed. ---

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL][WIP] Add Date and Timestamp ...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #82836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82836/testReport)** for PR 18664 at commit

[GitHub] spark pull request #19269: [SPARK-22026][SQL] data source v2 write path

2017-10-17 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19269#discussion_r145096772 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/v2/SimpleWritableDataSource.scala --- @@ -0,0 +1,252 @@ +/* + * Licensed to

[GitHub] spark issue #19497: [SPARK-21549][CORE] Respect OutputFormats with no/invali...

2017-10-17 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/19497 I guess one aspect of `saveAsNewAPIHadoopFile` is that it calls ` jobConfiguration.set("mapreduce.output.fileoutputformat.outputdir", path)`, and `Configuration.set(String key, String value)`

[GitHub] spark pull request #19513: [SPARK-21551][Python] Increase timeout for Python...

2017-10-17 Thread FRosner
GitHub user FRosner opened a pull request: https://github.com/apache/spark/pull/19513 [SPARK-21551][Python] Increase timeout for PythonRDD.serveIterator Backport of https://github.com/apache/spark/pull/18752 (https://issues.apache.org/jira/browse/SPARK-21551) (cherry

[GitHub] spark pull request #19512: [SPARK-21551][Python] Increase timeout for Python...

2017-10-17 Thread FRosner
GitHub user FRosner opened a pull request: https://github.com/apache/spark/pull/19512 [SPARK-21551][Python] Increase timeout for PythonRDD.serveIterator Backport of https://github.com/apache/spark/pull/18752 (https://issues.apache.org/jira/browse/SPARK-21551) (cherry

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14151 **[Test build #82838 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82838/testReport)** for PR 14151 at commit

[GitHub] spark pull request #19363: [SPARK-22224][SQL]Override toString of KeyValue/R...

2017-10-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19363 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19476 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19476 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82834/ Test PASSed. ---

[GitHub] spark issue #19469: [SPARK-22243][DStreams]spark.yarn.jars reload from confi...

2017-10-17 Thread ssaavedra
Github user ssaavedra commented on the issue: https://github.com/apache/spark/pull/19469 I think that may be a good idea. I'd say this can depend on the scheduler. Should that be discussed under a different JIRA number? ---

[GitHub] spark issue #14151: [SPARK-16496][SQL] Add wholetext as option for reading t...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82839/ Test PASSed. ---

[GitHub] spark issue #19513: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19513 **[Test build #3950 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3950/testReport)** for PR 19513 at commit

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19514 **[Test build #3949 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3949/consoleFull)** for PR 19514 at commit

[GitHub] spark issue #19512: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19512 **[Test build #3948 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3948/testReport)** for PR 19512 at commit

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-17 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19433 @smurching I found some issues and have some thoughts on the columnar features format: - In your doc, you said "Specifically, we only need to store sufficient stats for each bin of a

[GitHub] spark pull request #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby()....

2017-10-17 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/19517 [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply() with pandas udf ## What changes were proposed in this pull request? This is a follow-up of #18732. This pr modifies

[GitHub] spark issue #19516: [SPARK-22277][ML]fix the bug of ChiSqSelector on prepari...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19516 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82840/ Test PASSed. ---

[GitHub] spark issue #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19517 **[Test build #82841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82841/testReport)** for PR 19517 at commit

[GitHub] spark pull request #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby()....

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19517#discussion_r145149081 --- Diff: python/pyspark/sql/functions.py --- @@ -2038,13 +2038,22 @@ def _wrap_function(sc, func, returnType):

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r145146799 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -23,21 +23,58 @@ import

[GitHub] spark issue #19516: [SPARK-22277][ML]fix the bug of ChiSqSelector on prepari...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19516 **[Test build #82840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82840/testReport)** for PR 19516 at commit

[GitHub] spark issue #19516: [SPARK-22277][ML]fix the bug of ChiSqSelector on prepari...

2017-10-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19516 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r145148278 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala --- @@ -63,6 +64,23 @@ case class InMemoryRelation(

[GitHub] spark issue #19267: [WIP][SPARK-20628][CORE] Blacklist nodes when they trans...

2017-10-17 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19267 just reading through your description here all the yarn pieces aren't in place so you have an admin type command to signal spark that a node is being decommissioned. But that means someone has to

[GitHub] spark issue #19476: [SPARK-22062][CORE] Spill large block to disk in BlockMa...

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19476 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19516: [SPARK-22277][ML]fix the bug of ChiSqSelector on prepari...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19516 **[Test build #82840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82840/testReport)** for PR 19516 at commit

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r145137543 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala --- @@ -63,6 +64,23 @@ case class InMemoryRelation(

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r145139806 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -23,21 +23,58 @@ import

[GitHub] spark pull request #18747: [SPARK-20822][SQL] Generate code to directly get ...

2017-10-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18747#discussion_r145139673 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala --- @@ -23,21 +23,58 @@ import

[GitHub] spark pull request #19516: [SPARK-22277][ML]fix the bug of ChiSqSelector on ...

2017-10-17 Thread mpjlu
GitHub user mpjlu opened a pull request: https://github.com/apache/spark/pull/19516 [SPARK-22277][ML]fix the bug of ChiSqSelector on preparing the output column ## What changes were proposed in this pull request? To prepare the output columns when use ChiSqSelector, the

[GitHub] spark issue #19505: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19505 **[Test build #82843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82843/testReport)** for PR 19505 at commit

  1   2   3   4   5   >