[GitHub] spark issue #16739: [SPARK-19399][SPARKR] Add R coalesce API for DataFrame a...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16739 **[Test build #72925 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72925/testReport)** for PR 16739 at commit

[GitHub] spark pull request #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to...

2017-02-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r101218456 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2729,6 +2749,18 @@ class Dataset[T] private[sql]( } }

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-02-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15125 @mallman Looks good to me. Wait for any committers to check this further. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16761: [BackPort-2.1][SPARK-19319][SparkR]:SparkR Kmeans summar...

2017-02-14 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16761 ping @wangmiao1981 please close this PR - backport PR merged to non-master branch isn't closed automatically. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #15125: [SPARK-5484][GraphX] Periodically do checkpoint i...

2017-02-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15125#discussion_r101216945 --- Diff: graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala --- @@ -155,6 +169,8 @@ object Pregel extends Logging { i += 1 }

[GitHub] spark pull request #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsup...

2017-02-14 Thread windpiger
Github user windpiger commented on a diff in the pull request: https://github.com/apache/spark/pull/15918#discussion_r101216681 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -708,6 +715,13 @@ object ScalaReflection extends

[GitHub] spark issue #16715: [Spark-18080][ML][PYTHON] Python API & Examples for Loca...

2017-02-14 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/16715 @sethah Really appreciate your detailed code review and comments. :) @MLnick @yanboliang Thank you for the help as well. Please let me know if you guys have any other comments. --- If your

[GitHub] spark issue #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsupported ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15918 **[Test build #72924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72924/testReport)** for PR 15918 at commit

[GitHub] spark issue #16715: [Spark-18080][ML][PYTHON] Python API & Examples for Loca...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16715 **[Test build #72923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72923/testReport)** for PR 16715 at commit

[GitHub] spark issue #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsupported ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15918 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsupported ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15918 **[Test build #72922 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72922/testReport)** for PR 15918 at commit

[GitHub] spark issue #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsupported ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15918 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72921/ Test FAILed. ---

[GitHub] spark issue #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsupported ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15918 **[Test build #72921 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72921/testReport)** for PR 15918 at commit

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101214678 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed to

[GitHub] spark pull request #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to...

2017-02-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r101214398 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ArrowConverters.scala --- @@ -0,0 +1,360 @@ +/* +* Licensed to the Apache Software Foundation

[GitHub] spark pull request #15821: [SPARK-13534][WIP][PySpark] Using Apache Arrow to...

2017-02-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r101214262 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ArrowConverters.scala --- @@ -0,0 +1,360 @@ +/* +* Licensed to the Apache Software Foundation

[GitHub] spark issue #15918: [SPARK-18122][SQL][WIP]Fallback to Kryo for unsupported ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15918 **[Test build #72921 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72921/testReport)** for PR 15918 at commit

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/16909 @hvanhovell : Re performance: I ran a SMB join query over two real world tables (2 trillion rows (40 TB) and 6 million rows (120 GB)). Note that this dataset does not have skew so no

[GitHub] spark issue #16935: [SPARK-19604] [TESTS] Log the start of every Python test

2017-02-14 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/16935 Let's not merge it right now. I may need to log more. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16931: [SPARK-19587][SQL] bucket sorting columns should ...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16931#discussion_r101207253 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -270,52 +269,17 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark pull request #16931: [SPARK-19587][SQL] bucket sorting columns should ...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16931#discussion_r101206619 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -270,52 +269,17 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72918/ Test FAILed. ---

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16909 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16909 **[Test build #72918 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72918/testReport)** for PR 16909 at commit

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16228 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16228 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72917/ Test PASSed. ---

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16228 **[Test build #72917 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72917/testReport)** for PR 16228 at commit

[GitHub] spark issue #16896: [SPARK-19561][Python] cast TimestampType.toInternal outp...

2017-02-14 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/16896 Just one minor comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #16896: [SPARK-19561][Python] cast TimestampType.toIntern...

2017-02-14 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/16896#discussion_r101204910 --- Diff: python/pyspark/sql/types.py --- @@ -189,7 +189,7 @@ def toInternal(self, dt): if dt is not None: seconds =

[GitHub] spark issue #16928: [SPARK-18699][SQL] Fill NULL in a field when detecting a...

2017-02-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16928 Aha, looks good to me. Just a sec, and I'll modify the code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16928: [SPARK-18699][SQL] Fill NULL in a field when detecting a...

2017-02-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16928 @maropu, I just ran some similar tests with JSON datasource. What do you think about matching it to JSON's behaviour by introducing `columnNameOfCorruptRecord`? I ran with the data and

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16620 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72913/ Test PASSed. ---

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16620 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72913/testReport)** for PR 16620 at commit

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16620 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72912/ Test PASSed. ---

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16620 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16674 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16674 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72914/ Test PASSed. ---

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72912/testReport)** for PR 16620 at commit

[GitHub] spark issue #16935: [SPARK-19604] [TESTS] Log the start of every Python test

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16935 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72919/ Test PASSed. ---

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16674 **[Test build #72914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72914/testReport)** for PR 16674 at commit

[GitHub] spark issue #16935: [SPARK-19604] [TESTS] Log the start of every Python test

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16935 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16935: [SPARK-19604] [TESTS] Log the start of every Python test

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16935 **[Test build #72919 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72919/testReport)** for PR 16935 at commit

[GitHub] spark issue #16672: [SPARK-19329][SQL]Reading from or writing to a datasourc...

2017-02-14 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16672 @gatorsmile I have fixed some review issues. Could you help to continue to review this ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16932 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16932 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72906/ Test FAILed. ---

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16932 **[Test build #72906 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72906/testReport)** for PR 16932 at commit

[GitHub] spark pull request #16715: [Spark-18080][ML][PYTHON] Python API & Examples f...

2017-02-14 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16715#discussion_r101201608 --- Diff: examples/src/main/java/org/apache/spark/examples/ml/JavaMinHashLSHExample.java --- @@ -17,6 +17,7 @@ package

[GitHub] spark issue #16935: [SPARK-19604] [TESTS] Log the start of every Python test

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16935 **[Test build #72919 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72919/testReport)** for PR 16935 at commit

[GitHub] spark pull request #16935: [SPARK-19604] [TESTS] Log the start of every Pyth...

2017-02-14 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/16935 [SPARK-19604] [TESTS] Log the start of every Python test ## What changes were proposed in this pull request? Right now, we only have info level log after we finish the tests of a Python

[GitHub] spark issue #16907: [SPARK-19572][SPARKR] Allow to disable hive in sparkR sh...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16907 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16907: [SPARK-19572][SPARKR] Allow to disable hive in sparkR sh...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16907 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72915/ Test FAILed. ---

[GitHub] spark issue #16907: [SPARK-19572][SPARKR] Allow to disable hive in sparkR sh...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16907 **[Test build #72915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72915/testReport)** for PR 16907 at commit

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101201086 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,218 @@ +/* + * Licensed

[GitHub] spark issue #16818: [SPARK-19451][SQL][Core] Underlying integer overflow in ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16818 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72911/ Test PASSed. ---

[GitHub] spark issue #16818: [SPARK-19451][SQL][Core] Underlying integer overflow in ...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16818 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16818: [SPARK-19451][SQL][Core] Underlying integer overflow in ...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16818 **[Test build #72911 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72911/testReport)** for PR 16818 at commit

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101199851 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100926134 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -376,8 +386,15 @@ case class SortMergeJoinExec(

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100899895 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100900763 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100899694 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArraySuite.scala --- @@ -0,0 +1,300 @@ +/* + *

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101150812 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100925208 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala --- @@ -285,6 +283,9 @@ case class WindowExec( val

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100807225 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala --- @@ -310,10 +311,15 @@ case class WindowExec(

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100818273 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100899498 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101150862 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/window/RowBuffer.scala --- @@ -17,99 +17,33 @@ package

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r101150777 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala --- @@ -97,6 +98,11 @@ case class SortMergeJoinExec(

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100899601 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala --- @@ -285,6 +283,9 @@ case class WindowExec( val

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100821291 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark pull request #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeR...

2017-02-14 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/16909#discussion_r100818119 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ExternalAppendOnlyUnsafeRowArray.scala --- @@ -0,0 +1,179 @@ +/* + * Licensed

[GitHub] spark issue #16920: [MINOR][DOCS] Add jira url in pull request description

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16920 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16920: [MINOR][DOCS] Add jira url in pull request description

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16920 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72910/ Test PASSed. ---

[GitHub] spark issue #16920: [MINOR][DOCS] Add jira url in pull request description

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16920 **[Test build #72910 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72910/testReport)** for PR 16920 at commit

[GitHub] spark issue #16909: [SPARK-13450] Introduce ExternalAppendOnlyUnsafeRowArray...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16909 **[Test build #72918 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72918/testReport)** for PR 16909 at commit

[GitHub] spark pull request #16776: [SPARK-19436][SQL] Add missing tests for approxQu...

2017-02-14 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/16776#discussion_r101199261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala --- @@ -58,49 +58,52 @@ final class DataFrameStatFunctions

[GitHub] spark issue #16934: [SPARK-19603][SS]Fix StreamingQuery explain command

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16934 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16934: [SPARK-19603][SS]Fix StreamingQuery explain command

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16934 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72909/ Test PASSed. ---

[GitHub] spark issue #16934: [SPARK-19603][SS]Fix StreamingQuery explain command

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16934 **[Test build #72909 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72909/testReport)** for PR 16934 at commit

[GitHub] spark pull request #16918: [SPARK-19584] [SS] [DOCS] update structured strea...

2017-02-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16918 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16918: [SPARK-19584] [SS] [DOCS] update structured streaming do...

2017-02-14 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16918 LGTM. merging now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16228 **[Test build #72917 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72917/testReport)** for PR 16228 at commit

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16228 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72916/ Test FAILed. ---

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16228 **[Test build #72916 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72916/testReport)** for PR 16228 at commit

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16228 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16228 **[Test build #72916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72916/testReport)** for PR 16228 at commit

[GitHub] spark issue #16907: [SPARK-19572][SPARKR] Allow to disable hive in sparkR sh...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16907 **[Test build #72915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72915/testReport)** for PR 16907 at commit

[GitHub] spark issue #16674: [SPARK-19331][SQL][TESTS] Improve the test coverage of S...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16674 **[Test build #72914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72914/testReport)** for PR 16674 at commit

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72913/testReport)** for PR 16620 at commit

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16620 @squito Thanks a lot. I've refined the comment, please take another look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16932 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16932 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72898/ Test FAILed. ---

[GitHub] spark issue #16932: [SPARK-19599][SS]Clean up HDFSMetadataLog for Hadoop 2.6...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16932 **[Test build #72898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72898/testReport)** for PR 16932 at commit

[GitHub] spark issue #16620: [SPARK-19263] DAGScheduler should avoid sending conflict...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16620 **[Test build #72912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72912/testReport)** for PR 16620 at commit

[GitHub] spark issue #16146: [SPARK-18091] [SQL] [BACKPORT-1.6] Deep if expressions c...

2017-02-14 Thread zzcclp
Github user zzcclp commented on the issue: https://github.com/apache/spark/pull/16146 ping @cloud-fan , will this pr be merged into branch-1.6? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #16901: [SPARK-19565] Improve DAGScheduler tests.

2017-02-14 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16901 @squito Thanks a lot for your comments. I've refined the comment. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16228 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72903/ Test PASSed. ---

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16228 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16228: [SPARK-17076] [SQL] Cardinality estimation for join base...

2017-02-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16228 **[Test build #72903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72903/testReport)** for PR 16228 at commit

  1   2   3   4   5   6   7   >