[GitHub] spark issue #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle partition...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19816 **[Test build #84171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84171/testReport)** for PR 19816 at commit

[GitHub] spark issue #19818: [SPARK-22604][SQL] remove the get address methods from C...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19818 **[Test build #84174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84174/testReport)** for PR 19818 at commit

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19808 FYI, I believe the test failures are from SPARK-22595, which is fixed now. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19818: [SPARK-22604][SQL] remove the get address methods from C...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19818 cc @kiszk @viirya @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19818: [SPARK-22604][SQL] remove the get address methods...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19818#discussion_r153004995 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala --- @@ -413,16 +399,12 @@ class ColumnarBatchSuite

[GitHub] spark pull request #19818: [SPARK-22604][SQL] remove the get address methods...

2017-11-24 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/19818 [SPARK-22604][SQL] remove the get address methods from ColumnVector ## What changes were proposed in this pull request? `nullsNativeAddress` and `valuesNativeAddress` are only used in

[GitHub] spark issue #19814: [SPARK-22484][DOC] Document PySpark DataFrame csv writer...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19814 I think we should document this in read side too if we are going to do and actually I think it's not intended as the thirdparty one does not allow empty string IIRC. Will take a closer look

[GitHub] spark issue #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle partition...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19816 I expect 10 ~ 15 minutes decrease. Let me wait the results before cc'ing someone to review. --- - To unsubscribe,

[GitHub] spark issue #19817: [SPARK-22603][SQL] Fix 64KB JVM bytecode limit problem w...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19817 **[Test build #84173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84173/testReport)** for PR 19817 at commit

[GitHub] spark issue #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle partition...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19816 Build started: [SparkR] `ALL`

[GitHub] spark pull request #19817: [SPARK-22603][SQL] Fix 64KB JVM bytecode limit pr...

2017-11-24 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/19817 [SPARK-22603][SQL] Fix 64KB JVM bytecode limit problem with FormatString ## What changes were proposed in this pull request? This PR changes `FormatString` code generation to place generated

[GitHub] spark issue #19815: [SPARK-22602][SQL] remove ColumnVector#loadBytes

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19815 **[Test build #84172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84172/testReport)** for PR 19815 at commit

[GitHub] spark pull request #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle pa...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19816#discussion_r153001382 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3078,101 +3091,117 @@ test_that("coalesce, repartition, numPartitions", { })

[GitHub] spark pull request #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle pa...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19816#discussion_r153001109 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3021,41 +3021,54 @@ test_that("dapplyCollect() on DataFrame with a binary column", { })

[GitHub] spark pull request #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle pa...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19816#discussion_r153001070 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3078,101 +3091,117 @@ test_that("coalesce, repartition, numPartitions", { })

[GitHub] spark pull request #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle pa...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19816#discussion_r153000843 --- Diff: R/pkg/tests/fulltests/test_sparkSQL.R --- @@ -3021,41 +3021,54 @@ test_that("dapplyCollect() on DataFrame with a binary column", { })

[GitHub] spark issue #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle partition...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19816 This one is the last where I am seeing kind of hanging in few specific tests. I think I need AppVeyor tests to see the actual elapsed time diff. ---

[GitHub] spark issue #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle partition...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19816 **[Test build #84171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84171/testReport)** for PR 19816 at commit

[GitHub] spark pull request #19816: [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle pa...

2017-11-24 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19816 [WIP][SPARK-21693][FOLLOWUP][R] Reduce shuffle partitions running R worker in few tests to speed up ## What changes were proposed in this pull request? This is a followup to reduce

[GitHub] spark issue #19815: [SPARK-22602][SQL] remove ColumnVector#loadBytes

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19815 **[Test build #84170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84170/testReport)** for PR 19815 at commit

[GitHub] spark issue #19815: [SPARK-22602][SQL] remove ColumnVector#loadBytes

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19815 cc @michal-databricks @kiszk @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19815: [SPARK-22602][SQL] remove ColumnVector#loadBytes

2017-11-24 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/19815 [SPARK-22602][SQL] remove ColumnVector#loadBytes ## What changes were proposed in this pull request? `ColumnVector#loadBytes` is only used as an optimization for reading UTF8String in

[GitHub] spark issue #19814: [SPARK-22484][DOC] Document PySpark DataFrame csv writer...

2017-11-24 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/19814 As I've seen the univocity library does not support explicit quote turn off feature on the write side. --- - To

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 In int case, the followings are the number of java byte code length for a value 1 byte: -1, 0, 1, 2, 3, 4, 5 (iconst_?) 2 byte: -128 ~ -2, 6 ~ 127 (bipush) 3 bytes: -32768 ~ -129, 128

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84169 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84169/testReport)** for PR 19811 at commit

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19518 The PR looks good. Although it can solve the constant pool pressure, however, I have a question. Does it mean every time a constant falling in short range is used in codes, we increase 1

[GitHub] spark issue #19752: [SPARK-22520][SQL] Support code generation for large Cas...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19752 **[Test build #84168 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84168/testReport)** for PR 19752 at commit

[GitHub] spark issue #19752: [SPARK-22520][SQL] Support code generation for large Cas...

2017-11-24 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19752 @gatorsmile I added a test case to check that the execution plan is `WholeStageCodegenExec` as expected. I also made some performance test using almost the same code, ie.: ``` val N = 30

[GitHub] spark pull request #19756: [SPARK-22527][SQL] Reuse coordinated exchanges if...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19756#discussion_r152992616 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala --- @@ -108,6 +108,7 @@ class QueryExecution(val sparkSession:

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84167/ Test FAILed. ---

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84167/testReport)** for PR 19811 at commit

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84167/testReport)** for PR 19811 at commit

[GitHub] spark issue #19814: [SPARK-22484][DOC] Document PySpark DataFrame csv writer...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19814 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19814: [SPARK-22484][DOC] Document PySpark DataFrame csv...

2017-11-24 Thread gaborgsomogyi
GitHub user gaborgsomogyi opened a pull request: https://github.com/apache/spark/pull/19814 [SPARK-22484][DOC] Document PySpark DataFrame csv writer behavior whe… ## What changes were proposed in this pull request? In PySpark API Document, DataFrame.write.csv() says that

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84165/ Test FAILed. ---

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19788 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #84165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84165/testReport)** for PR 19788 at commit

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19773 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84164/ Test PASSed. ---

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19773 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19773 **[Test build #84164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84164/testReport)** for PR 19773 at commit

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 Here is [a PR](https://github.com/janino-compiler/janino/pull/34) to fix a problem regarding `sipush`. --- - To unsubscribe,

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19811 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84163/ Test FAILed. ---

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84163/testReport)** for PR 19811 at commit

[GitHub] spark issue #19756: [SPARK-22527][SQL] Reuse coordinated exchanges if possib...

2017-11-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19756 Do you mean we manually run `ReuseExchange` after the registration of exchanges? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19788 think about shuffle as a server-client framework, does your change need to update the server side? i.e. do users need to upgrade their external shuffle service for new spark version with this

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152980151 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockId.scala --- @@ -52,8 +52,9 @@ case class RDDBlockId(rddId: Int, splitIndex: Int) extends

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-24 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19788 Also cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152976816 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java --- @@ -110,6 +110,13 @@ public

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152976135 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java --- @@ -165,13 +165,23 @@ public

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152976754 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java --- @@ -110,6 +110,13 @@ public

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152978408 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockId.scala --- @@ -52,8 +52,9 @@ case class RDDBlockId(rddId: Int, splitIndex: Int) extends

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18424 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19763: [SPARK-22537][core] Aggregation of map output sta...

2017-11-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19763 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19802: [WIP][SPARK-22594][CORE] Handling spark-submit and maste...

2017-11-24 Thread Jiri-Kremser
Github user Jiri-Kremser commented on the issue: https://github.com/apache/spark/pull/19802 To explain better the intentions, it doesn't try to solve of somehow provide a compatibility layer between old and new versions of Spark, all it does is slightly improving the UX, because

[GitHub] spark pull request #19786: [SPARK-22559][CORE]history server: handle excepti...

2017-11-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19786 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19763 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19805: [SQL] Adding localCheckpoint to Dataset API

2017-11-24 Thread ferdonline
Github user ferdonline commented on the issue: https://github.com/apache/spark/pull/19805 cc @andrewor14 since I believe you know this part of spark pretty well, maybe you could help me integrating this. Any idea why Jenkins didn't start the testing? ---

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19763 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19786: [SPARK-22559][CORE]history server: handle exception on o...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19786 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19763 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84161/ Test PASSed. ---

[GitHub] spark issue #19763: [SPARK-22537][core] Aggregation of map output statistics...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19763 **[Test build #84161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84161/testReport)** for PR 19763 at commit

[GitHub] spark issue #19756: [SPARK-22527][SQL] Reuse coordinated exchanges if possib...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19756 can we run `ReuseExchange` twice? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19786: [SPARK-22559][CORE]history server: handle exception on o...

2017-11-24 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19786 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 Next, I analyzed usage of constant pool entries and java bytecode ops using `janinoc`. The summary is as follows: ``` array[4] : 6 + 0 * n entries, 6-8 java bytecode ops / read access

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84166/testReport)** for PR 19808 at commit

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84159/ Test PASSed. ---

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84159 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84159/testReport)** for PR 19808 at commit

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19808 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84160/ Test FAILed. ---

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84160 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84160/testReport)** for PR 19808 at commit

[GitHub] spark issue #19518: [SPARK-18016][SQL][CATALYST] Code Generation: Constant P...

2017-11-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19518 First of all, I have to share sad news with you. `janino` does not use `sipush` for values from 128 to 32767. Current `janino` 3.0.7 uses `iconst...`, `bipush`, or `ldc`. `javac` uses `sipush` for

[GitHub] spark issue #19756: [SPARK-22527][SQL] Reuse coordinated exchanges if possib...

2017-11-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19756 The exchanges only register with `ExchangeCoordinator` after query transformation. So during transformation, we don't know if two `ExchangeCoordinator` have the same exchanges. ---

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152962372 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,12 +196,14 @@ private[spark] class

[GitHub] spark pull request #19788: [SPARK-9853][Core] Optimize shuffle fetch of cont...

2017-11-24 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/19788#discussion_r152961094 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,12 +196,14 @@ private[spark] class

[GitHub] spark pull request #19802: [WIP][SPARK-22594][CORE] Handling spark-submit an...

2017-11-24 Thread Jiri-Kremser
Github user Jiri-Kremser commented on a diff in the pull request: https://github.com/apache/spark/pull/19802#discussion_r152960380 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/TransportRequestHandler.java --- @@ -206,6 +207,11 @@ public void

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19788 **[Test build #84165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84165/testReport)** for PR 19788 at commit

[GitHub] spark pull request #19802: [WIP][SPARK-22594][CORE] Handling spark-submit an...

2017-11-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19802#discussion_r152959040 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/TransportRequestHandler.java --- @@ -206,6 +207,11 @@ public void

[GitHub] spark issue #19811: [WIP][SPARK-18016][SQL] Code Generation: Constant Pool L...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19811 **[Test build #84163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84163/testReport)** for PR 19811 at commit

[GitHub] spark issue #19773: [SPARK-22546][SQL] Supporting for changing column dataTy...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19773 **[Test build #84164 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84164/testReport)** for PR 19773 at commit

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84157/ Test PASSed. ---

[GitHub] spark issue #19802: [WIP][SPARK-22594][CORE] Handling spark-submit and maste...

2017-11-24 Thread Jiri-Kremser
Github user Jiri-Kremser commented on the issue: https://github.com/apache/spark/pull/19802 > Can you please explain more, and how to reproduce this issue? Sure, 1. Start master and worker (in version 2.2.0 for instance): ```bash cat ./python/pyspark/version.py

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84157 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84157/testReport)** for PR 19808 at commit

[GitHub] spark pull request #19773: [SPARK-22546][SQL] Supporting for changing column...

2017-11-24 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19773#discussion_r152957444 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -318,16 +318,26 @@ case class AlterTableChangeColumnCommand(

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84162/ Test PASSed. ---

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19789 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #84162 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84162/testReport)** for PR 19789 at commit

[GitHub] spark pull request #19808: [SPARK-22597][SQL] Add spark-sql cmd script for W...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19808#discussion_r152953457 --- Diff: bin/spark-sql.cmd --- @@ -0,0 +1,25 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19789 **[Test build #84162 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84162/testReport)** for PR 19789 at commit

[GitHub] spark issue #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsafe evic...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19789 cc @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19789: [SPARK-22562][Streaming] CachedKafkaConsumer unsa...

2017-11-24 Thread daroo
Github user daroo commented on a diff in the pull request: https://github.com/apache/spark/pull/19789#discussion_r152951885 --- Diff: external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaRDD.scala --- @@ -211,8 +211,8 @@ private[spark] class KafkaRDD[K, V](

[GitHub] spark pull request #19808: [SPARK-22597][SQL] Add spark-sql cmd script for W...

2017-11-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19808#discussion_r152951398 --- Diff: bin/spark-sql.cmd --- @@ -0,0 +1,25 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or

[GitHub] spark pull request #19808: [SPARK-22597][SQL] Add spark-sql cmd script for W...

2017-11-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19808#discussion_r152947466 --- Diff: bin/spark-sql.cmd --- @@ -0,0 +1,25 @@ +@echo off + +rem +rem Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19808 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84156/ Test FAILed. ---

[GitHub] spark issue #19808: [SPARK-22597][SQL] Add spark-sql cmd script for Windows ...

2017-11-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19808 **[Test build #84156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84156/testReport)** for PR 19808 at commit

<    1   2   3   >