[GitHub] spark issue #20709: [SPARK-18844][MLLIB] Adding more binary classification e...

2018-03-02 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/20709 You don't need to (and should not) open a new PR to fix merge conflicts. Just fix them through git, on the same branch. --- - To

[GitHub] spark issue #20664: [SPARK-23496][CORE] Locality of coalesced partitions can...

2018-03-02 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/20664 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20706#discussion_r171881215 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1876,17 +1822,6 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request #20706: [SPARK-23550][core] Cleanup `Utils`.

2018-03-02 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/20706#discussion_r171879418 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -1018,70 +1016,18 @@ private[spark] object Utils extends Logging { " " +

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20618 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87883/ Test PASSed. ---

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20618 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87883 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87883/testReport)** for PR 20618 at commit

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20716 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87888/ Test PASSed. ---

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20716 **[Test build #87888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87888/testReport)** for PR 20716 at commit

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20716 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20704: [SPARK-23551][BUILD] Exclude `hadoop-mapreduce-client-co...

2018-03-02 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/20704 kicks in downstream depending on the order of imports; maven is closest-first in the graph. If you explicitly add hadoop-client in your deps at the top then everything gets reconciled

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87887/ Test FAILed. ---

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20700 **[Test build #87887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87887/testReport)** for PR 20700 at commit

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20716 **[Test build #87888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87888/testReport)** for PR 20716 at commit

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20700 **[Test build #87886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87886/testReport)** for PR 20700 at commit

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87886/ Test FAILed. ---

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20716 It's fine. Can you double check if there are same instances in this file or other files? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20716 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20700 **[Test build #87887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87887/testReport)** for PR 20700 at commit

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1228/

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20639: [SPARK-23288][SS] Fix output metrics with parquet sink

2018-03-02 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20639 Executed these tests manually again but working fine. Seems like flaky. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20700 **[Test build #87886 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87886/testReport)** for PR 20700 at commit

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1227/

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20700 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20716 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20716: [SPARK-23566][Minor][Doc] Argument name mismatch fixed

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20716 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20716: [SPARK-23566][Minor][Doc] Argument name mismatch ...

2018-03-02 Thread animenon
GitHub user animenon opened a pull request: https://github.com/apache/spark/pull/20716 [SPARK-23566][Minor][Doc] Argument name mismatch fixed Argument name mismatch fixed. ## What changes were proposed in this pull request? `col` changed to `new` in doc string to

[GitHub] spark issue #20700: [SPARK-23546][SQL] Refactor stateless methods/values in ...

2018-03-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20700 @rednaxelafx Oh, very interesting since we are doing the similar thing in West coast and Japan! I just say `not refactored YET`. Yeah, I absolutely love to delete old instance method if we

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20682 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20682 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87882/ Test PASSed. ---

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20682 **[Test build #87882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87882/testReport)** for PR 20682 at commit

[GitHub] spark issue #20712: [SPARK-23563] config codegen compile cache size

2018-03-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20712 +1, I would like to know the case where we increased cache hit ratio. Do we reuse the same plan beyond more than 100 plans? ---

[GitHub] spark issue #20712: [SPARK-23563] config codegen compile cache size

2018-03-02 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20712 @mgaido91 Yeah, you are right. This is pretty pointless unless you either set this as a static spark conf or pass it to the executor using local properties. Both aren't very attractive IMO.

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171842603 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #87885 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87885/testReport)** for PR 19222 at commit

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1226/

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19222 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87884/ Test PASSed. ---

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20121 **[Test build #87884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87884/testReport)** for PR 20121 at commit

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171839479 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171839162 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OffHeapMemoryBlock.java --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20709: [SPARK-18844][MLLIB] Adding more binary classification e...

2018-03-02 Thread sandecho
Github user sandecho commented on the issue: https://github.com/apache/spark/pull/20709 Can you please test it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r171836519 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1313,131 +1313,165 @@ object functions {

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r171834617 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1500,31 +1534,35 @@ object functions { } /** -

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171834562 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OffHeapMemoryBlock.java --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171834517 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -50,12 +52,12 @@ // These are only updated by

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171834629 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -52,11 +53,16 @@ public static int

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r171829946 --- Diff: python/pyspark/sql/functions.py --- @@ -173,16 +172,26 @@ def _(): _functions_2_1 = { # unary math functions -

[GitHub] spark pull request #20618: [SPARK-23329][SQL] Fix documentation of trigonome...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20618#discussion_r171829796 --- Diff: python/pyspark/sql/functions.py --- @@ -173,16 +172,26 @@ def _(): _functions_2_1 = { # unary math functions -

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20121 **[Test build #87884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87884/testReport)** for PR 20121 at commit

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20618 **[Test build #87883 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87883/testReport)** for PR 20618 at commit

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20121 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20121 @smurakozi Address your comments. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20121: [SPARK-22882][ML][TESTS] ML test for structured streamin...

2018-03-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20121 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1225/

[GitHub] spark issue #20618: [SPARK-23329][SQL] Fix documentation of trigonometric fu...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20618 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171822744 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -36,22 +42,35 @@ public MemoryBlock allocate(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171822360 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171821972 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -87,6 +106,35 @@ public static int hashUnsafeBytes2(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171821706 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -87,6 +106,35 @@ public static int hashUnsafeBytes2(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171820928 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171821064 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -87,6 +106,35 @@ public static int hashUnsafeBytes2(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171820330 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171815998 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OffHeapMemoryBlock.java --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20692: [SPARK-23531][SQL] Show attribute type in explain

2018-03-02 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20692 thanks @gatorsmile and @rdblue for your discussion and thanks @gatorsmile for trusting me. I hope I will be worthy of your trust. I agree that since we are not close to a new release,

[GitHub] spark issue #20433: [SPARK-23264][SQL] Support interval values without INTER...

2018-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20433 @gatorsmile ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-03-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20343 We need to review #20433 first --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20682 lgtm otherwise --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171805189 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/bitset/BitSetMethods.java --- @@ -45,6 +46,14 @@ public static void set(Object baseObject,

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20682 **[Test build #87882 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87882/testReport)** for PR 20682 at commit

[GitHub] spark pull request #20682: [SPARK-23522][Python] always use sys.exit over bu...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20682#discussion_r171803941 --- Diff: python/pyspark/ml/evaluation.py --- @@ -15,6 +15,8 @@ # limitations under the License. # +import sys + --- End

[GitHub] spark issue #20682: [SPARK-23522][Python] always use sys.exit over builtin e...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20682 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171801856 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,136 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171801720 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,136 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171801055 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,136 @@ */ public static final int

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-03-02 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r171796232 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/GBTClassifierSuite.scala --- @@ -169,59 +171,28 @@ class GBTClassifierSuite extends

[GitHub] spark pull request #20121: [SPARK-22882][ML][TESTS] ML test for structured s...

2018-03-02 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20121#discussion_r171796773 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -75,11 +71,9 @@ class

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20705 @dongjoon-hyun, I agree with the idea in general but just to be clear do you target to improve the test coverage for both when `orc` and `parquet` are set to `spark.sql.sources.default` here?

[GitHub] spark pull request #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml...

2018-03-02 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20686#discussion_r171790543 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/RFormulaSuite.scala --- @@ -313,13 +306,14 @@ class RFormulaSuite extends MLTest with

[GitHub] spark pull request #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml...

2018-03-02 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20686#discussion_r171790083 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTest.scala --- @@ -108,5 +111,29 @@ trait MLTest extends StreamTest with TempDirectory {

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171787057 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OffHeapMemoryBlock.java --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171785582 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -87,6 +106,35 @@ public static int hashUnsafeBytes2(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171784371 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/bitset/BitSetMethods.java --- @@ -45,6 +46,14 @@ public static void set(Object baseObject,

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171783726 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,136 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171787508 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/UTF8String.java --- @@ -50,12 +52,12 @@ // These are only updated by

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171784878 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/bitset/BitSetMethods.java --- @@ -80,6 +105,16 @@ public static boolean anySet(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171784254 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -52,11 +53,16 @@ public static int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171785720 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -87,6 +106,35 @@ public static int hashUnsafeBytes2(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171787117 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OffHeapMemoryBlock.java --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171784866 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/bitset/BitSetMethods.java --- @@ -67,6 +84,14 @@ public static boolean isSet(Object

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171784381 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/bitset/BitSetMethods.java --- @@ -56,6 +65,14 @@ public static void unset(Object baseObject,

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171785381 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171787333 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/UnsafeMemoryAllocator.java --- @@ -36,22 +42,35 @@ public MemoryBlock allocate(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171785802 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -52,13 +53,31 @@ public int hashUnsafeWords(Object base, long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171782719 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -17,47 +17,168 @@ package

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171783898 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,136 @@ */ public static final int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-02 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r171783854 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/MemoryBlock.java --- @@ -45,38 +45,136 @@ */ public static final int

<    1   2   3   4   5   >