[GitHub] spark issue #18350: [MINOR] Fix some typo of the document

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18350 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18343#discussion_r122625325 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -175,6 +175,7 @@ class KryoSerializer(conf: SparkConf)

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18343#discussion_r122625202 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -141,7 +141,7 @@ private[spark] class HighlyCompressedMapStatus private (

[GitHub] spark issue #18350: [MINOR] Fix some typo of the document

2017-06-18 Thread ConeyLiu
Github user ConeyLiu commented on the issue: https://github.com/apache/spark/pull/18350 Hi @srowen, would you mind take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18343 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #18350: [MINOR] Fix some typo of the document

2017-06-18 Thread ConeyLiu
GitHub user ConeyLiu opened a pull request: https://github.com/apache/spark/pull/18350 [MINOR] Fix some typo of the document ## What changes were proposed in this pull request? Fix some typo of the document. ## How was this patch tested? Existing tests.

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18343 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78239/ Test PASSed. ---

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18343 **[Test build #78239 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78239/testReport)** for PR 18343 at commit

[GitHub] spark issue #17328: [SPARK-19975][Python][SQL] Add map_keys and map_values f...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17328 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17328: [SPARK-19975][Python][SQL] Add map_keys and map_values f...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17328 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78242/ Test PASSed. ---

[GitHub] spark issue #17328: [SPARK-19975][Python][SQL] Add map_keys and map_values f...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17328 **[Test build #78242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78242/testReport)** for PR 17328 at commit

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18343#discussion_r122623575 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -141,7 +141,7 @@ private[spark] class HighlyCompressedMapStatus private (

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18343#discussion_r122623298 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -141,7 +141,7 @@ private[spark] class HighlyCompressedMapStatus private (

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18346 cc @cloud-fan @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18269: [SPARK-21056][SQL] Use at most one spark job to list fil...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18269 let's wait @mallman 's response to make sure this patch does fix the problem --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #18269: [SPARK-21056][SQL] Use at most one spark job to l...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18269#discussion_r122622206 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala --- @@ -248,60 +245,94 @@ object InMemoryFileIndex

[GitHub] spark pull request #18269: [SPARK-21056][SQL] Use at most one spark job to l...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18269#discussion_r122622157 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala --- @@ -248,60 +245,94 @@ object InMemoryFileIndex

[GitHub] spark pull request #18269: [SPARK-21056][SQL] Use at most one spark job to l...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18269#discussion_r122622031 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala --- @@ -248,60 +245,94 @@ object InMemoryFileIndex

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17471 I was just looking though PRs for my curiosity. Please let me leave a gentle ping @sitalkedia. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #18269: [SPARK-21056][SQL] Use at most one spark job to l...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18269#discussion_r122621941 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala --- @@ -248,60 +245,94 @@ object InMemoryFileIndex

[GitHub] spark pull request #18303: [SPARK-19824][Core] Update JsonProtocol to keep c...

2017-06-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18303 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18092: [SPARK-20640][CORE]Make rpc timeout and retry for shuffl...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18092 **[Test build #78247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78247/testReport)** for PR 18092 at commit

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-06-18 Thread liyichao
Github user liyichao commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r122621671 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1281,6 +1286,61 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18343#discussion_r122621682 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -141,7 +141,7 @@ private[spark] class HighlyCompressedMapStatus private (

[GitHub] spark issue #18303: [SPARK-19824][Core] Update JsonProtocol to keep consiste...

2017-06-18 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18303 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18303: [SPARK-19824][Core] Update JsonProtocol to keep consiste...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18303 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78235/ Test PASSed. ---

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18303: [SPARK-19824][Core] Update JsonProtocol to keep consiste...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18303 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78238/ Test PASSed. ---

[GitHub] spark issue #18290: [SPARK-20989][Core] Fail to start multiple workers on on...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18290 LGTM, only one question: are we going to support it? To reuse the same shuffle service across workers or to allow multiple shuffle services on one host? --- If your project is set up for it,

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78237/ Test PASSed. ---

[GitHub] spark issue #18303: [SPARK-19824][Core] Update JsonProtocol to keep consiste...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18303 **[Test build #78235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78235/testReport)** for PR 18303 at commit

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18346 **[Test build #78238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78238/testReport)** for PR 18346 at commit

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18346 **[Test build #78237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78237/testReport)** for PR 18346 at commit

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78236/ Test PASSed. ---

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18346: [SPARK-21134][SQL] Don't collapse codegen-only expressio...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18346 **[Test build #78236 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78236/testReport)** for PR 18346 at commit

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r122620995 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1281,6 +1286,61 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark pull request #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18343#discussion_r122620862 --- Diff: core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala --- @@ -141,7 +141,7 @@ private[spark] class HighlyCompressedMapStatus private (

[GitHub] spark issue #17401: [SPARK-18364][YARN] Expose metrics for YarnShuffleServic...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17401 gentle ping @ash211. I just wonder if it is active now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #18324: [SPARK-21045][PYSPARK]Fixed executor blocked beca...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18324#discussion_r122620720 --- Diff: python/pyspark/worker.py --- @@ -177,8 +180,11 @@ def process(): process() except Exception: try: +

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15417 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15417 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78243/ Test FAILed. ---

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15417 **[Test build #78243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78243/testReport)** for PR 15417 at commit

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-06-18 Thread liyichao
Github user liyichao commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r122620600 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1281,6 +1286,59 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-18 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17758 ok, I'll recheck. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18343 @wangyum Can you also add a test for this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18323: [SPARK-21117][SQL] Built-in SQL Function Support - WIDTH...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18323 **[Test build #78245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78245/testReport)** for PR 18323 at commit

[GitHub] spark pull request #14085: [SPARK-16408][SQL] SparkSQL Added file get Except...

2017-06-18 Thread zenglinxi0615
Github user zenglinxi0615 commented on a diff in the pull request: https://github.com/apache/spark/pull/14085#discussion_r122620464 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/commands.scala --- @@ -113,8 +113,9 @@ case class AddFile(path: String) extends

[GitHub] spark issue #18092: [SPARK-20640][CORE]Make rpc timeout and retry for shuffl...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18092 **[Test build #78246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78246/testReport)** for PR 18092 at commit

[GitHub] spark issue #18092: [SPARK-20640][CORE]Make rpc timeout and retry for shuffl...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18092 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17758: [SPARK-20460][SQL] Make it more consistent to handle col...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17758 shall we check duplicated columns in write path? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17395: [SPARK-20065][SS][WIP] Avoid to output empty parquet fil...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17395 hmmm. @uncleGen, shell we close this for now? reopening when it's ready would welcome. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17328: [SPARK-19975][Python][SQL] Add map_keys and map_values f...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17328 +1 for this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #18324: [SPARK-21045][PYSPARK]Fixed executor blocked beca...

2017-06-18 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18324#discussion_r122620265 --- Diff: python/pyspark/worker.py --- @@ -177,8 +180,11 @@ def process(): process() except Exception: try: +

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-06-18 Thread liyichao
Github user liyichao commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r122620196 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1281,6 +1286,59 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15417 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15417 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78241/ Test FAILed. ---

[GitHub] spark issue #15417: [SPARK-17851][SQL][TESTS] Make sure all test sqls in cat...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15417 **[Test build #78241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78241/testReport)** for PR 15417 at commit

[GitHub] spark issue #17084: [SPARK-18693][ML][MLLIB] ML Evaluators should use weight...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17084 gentle ping @imatiach-msft . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18025 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78244/ Test PASSed. ---

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18025 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18025 **[Test build #78244 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78244/testReport)** for PR 18025 at commit

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18343 @wangyum Thanks for updating. Can you try to disable kyro and try it again? So we can verify it. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17681 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78240/ Test FAILed. ---

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17681 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r122619579 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2603,12 +2603,27 @@ class Dataset[T] private[sql]( * current

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r122619526 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2603,12 +2603,27 @@ class Dataset[T] private[sql]( * current

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17681 **[Test build #78240 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78240/testReport)** for PR 17681 at commit

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18343 Because we write/read `hugeBlockSizes` in `writeExternal`/`readExternal`, it seems to me that it is intended to be serialized. So I think removing `transient` should be ok. LGTM cc

[GitHub] spark issue #18349: [SPARK-20927][SS] Change some operators in Dataset to no...

2017-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18349 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18349: [SPARK-20927][SS] Change some operators in Dataset to no...

2017-06-18 Thread ZiyueHuang
Github user ZiyueHuang commented on the issue: https://github.com/apache/spark/pull/18349 @zsxwing Could you please review this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/18343 @viirya Yes, I' using `org.apache.spark.serializer.KryoSerializer`, [master branch](https://github.com/apache/spark/tree/ce49428ef7d640c1734e91ffcddc49dbc8547ba7) still has this issue, error logs:

[GitHub] spark pull request #18349: [SPARK-20927][SS] Change some operators in Datase...

2017-06-18 Thread ZiyueHuang
GitHub user ZiyueHuang opened a pull request: https://github.com/apache/spark/pull/18349 [SPARK-20927][SS] Change some operators in Dataset to no-op for a streaming query. ## What changes were proposed in this pull request? Change some operators(persist, unpersist,

[GitHub] spark issue #16347: [SPARK-18934][SQL] Writing to dynamic partitions does no...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16347 gentle ping @junegunn on ^. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18343 I think this should be addressed before 2.2. I already asked notice of other committers on dev mailing list. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14957 @xuanyuanking, let's close this and help review #16578 if you agree on the comments above. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #18343: [SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeE...

2017-06-18 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18343 @wangyum Are you using kyro serializer? I think it is why you hit this issue. Once you use kyro, I think the `readExternal` in `HighlyCompressedMapStatus` won't be used to deserialize the

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-18 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18025 This is how the doc for column_aggregate_functions looks like (only snapshot of the main parts):

[GitHub] spark issue #18303: [SPARK-19824][Core] Update JsonProtocol to keep consiste...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18303 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18092: [SPARK-20640][CORE]Make rpc timeout and retry for shuffl...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18092 LGTM except 2 minor comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r122617592 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1281,6 +1286,59 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark pull request #18025: [SPARK-20889][SparkR] Grouped documentation for A...

2017-06-18 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18025#discussion_r122617531 --- Diff: R/pkg/R/stats.R --- @@ -52,22 +52,17 @@ setMethod("crosstab", collect(dataFrame(sct)) }) -#'

[GitHub] spark issue #18025: [SPARK-20889][SparkR] Grouped documentation for AGGREGAT...

2017-06-18 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18025 **[Test build #78244 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78244/testReport)** for PR 18025 at commit

[GitHub] spark pull request #18092: [SPARK-20640][CORE]Make rpc timeout and retry for...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18092#discussion_r122617448 --- Diff: core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala --- @@ -1281,6 +1286,59 @@ class BlockManagerSuite extends SparkFunSuite

[GitHub] spark issue #13893: [SPARK-14172][SQL] Hive table partition predicate not pa...

2017-06-18 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/13893 ya, this still exists. Let me find some time to resolve this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #18025: [SPARK-20889][SparkR] Grouped documentation for A...

2017-06-18 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18025#discussion_r122617405 --- Diff: R/pkg/R/stats.R --- @@ -52,22 +52,17 @@ setMethod("crosstab", collect(dataFrame(sct)) }) -#'

[GitHub] spark issue #12257: [SPARK-14483][WEBUI] Display user name for each job and ...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/12257 gentle ping @sarutak on ^ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18347: [SPARK-20599][SS] ConsoleSink should work with (b...

2017-06-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/18347#discussion_r122617147 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -465,6 +465,8 @@ case class DataSource(

[GitHub] spark issue #18296: [SPARK-21090][core]Optimize the unified memory manager c...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18296 Actually this is a bug fix, and it's a small fix(without tests, only 3 lines), so backporting in to 2.2 --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #11887: [SPARK-13041][Mesos]add driver sandbox uri to the dispat...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/11887 @skonto, do you maybe know (or a wild guess) when we would be able to proceed this? Probably, closing this for now and reopening might be an option. --- If your project is set up for it, you

[GitHub] spark pull request #18347: [SPARK-20599][SS] ConsoleSink should work with (b...

2017-06-18 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/18347#discussion_r122616876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala --- @@ -465,6 +465,8 @@ case class DataSource(

[GitHub] spark issue #11420: [SPARK-13493][SQL] Enable case sensitiveness in json sch...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/11420 Let's close this if it is not in a progress. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18025: [SPARK-20889][SparkR] Grouped documentation for A...

2017-06-18 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/18025#discussion_r122616787 --- Diff: R/pkg/R/stats.R --- @@ -52,22 +52,17 @@ setMethod("crosstab", collect(dataFrame(sct)) }) -#'

[GitHub] spark pull request #18296: [SPARK-21090][core]Optimize the unified memory ma...

2017-06-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18296 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #11205: [SPARK-11334][Core] Handle maximum task failure situatio...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/11205 gentle ping @rustgi, have you maybe had some time to confirm this patch maybe? It sounds the only thing we need here is the confirmation. --- If your project is set up for it, you can reply to

[GitHub] spark issue #18296: [SPARK-21090][core]Optimize the unified memory manager c...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18296 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #10861: SPARK-12948. [SQL]. Consider reducing size of broadcasts...

2017-06-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/10861 Hi @rajeshbalamohan, I think this should be a mergeable state at least and the conflicts and style issues should be resolved. Would you be able to update this for now? --- If your project is

[GitHub] spark issue #18334: [SPARK-21127] [SQL] Update statistics after data changin...

2017-06-18 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18334 +1 to provide a flag to automatically trigger the stats updates. We cat set it false by default to not surprise users --- If your project is set up for it, you can reply to this email and have

  1   2   3   >