[GitHub] spark pull request #18535: [SPARK-21312][SQL] correct offsetInBytes in Unsaf...

2017-07-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/18535#discussion_r125867556 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java --- @@ -550,7 +550,7 @@ public void copyFrom(UnsafeRow row) {

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-06 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18551 Thanks, @srowen . I have updated the doc. I also validated the current option in spark-env.sh, it works. Thanks. --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18551 **[Test build #79276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79276/testReport)** for PR 18551 at commit [`adfa6f1`](https://github.com/apache/spark/commit/ad

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125865937 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -150,7 +163,7 @@ object FileFormatWriter extends

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125865118 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79269/ Test PASSed. ---

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #79269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79269/testReport)** for PR 17980 at commit [`39c3851`](https://github.com/apache/spark/commit/3

[GitHub] spark pull request #18523: [SPARK-21285][ML] VectorAssembler reports the col...

2017-07-06 Thread facaiy
Github user facaiy commented on a diff in the pull request: https://github.com/apache/spark/pull/18523#discussion_r125860650 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala --- @@ -113,12 +113,15 @@ class VectorAssembler @Since("1.4.0") (@Since("1.

[GitHub] spark issue #18554: [SPARK-21306][ML] OneVsRest should cache weightCol if ne...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18554 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request #18554: [SPARK-21306][ML] OneVsRest should cache weightCo...

2017-07-06 Thread facaiy
GitHub user facaiy opened a pull request: https://github.com/apache/spark/pull/18554 [SPARK-21306][ML] OneVsRest should cache weightCol if necessary ## What changes were proposed in this pull request? cache weightCol if classifier inherits HasWeightCol trait. ## How

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-06 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/18551 Hm, can we try to set this with `System.setenv` anywhere in the code or will it be too late by the time this triggers? probably. I think this is OK, but should be documented also in places in

[GitHub] spark pull request #18444: [SPARK-16542][SQL][PYSPARK] Fix bugs about types ...

2017-07-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/18444#discussion_r125855962 --- Diff: python/pyspark/sql/types.py --- @@ -958,12 +1043,17 @@ def _infer_type(obj): return MapType(_infer_type(key), _infer_type(value

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparision should respect case-...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18460 **[Test build #79275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79275/testReport)** for PR 18460 at commit [`d3a9f73`](https://github.com/apache/spark/commit/d3

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18549 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18549 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79266/ Test PASSed. ---

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18549 **[Test build #79266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79266/testReport)** for PR 18549 at commit [`b51ab80`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18300 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79268/ Test PASSed. ---

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18300 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #79268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79268/testReport)** for PR 18300 at commit [`c43a968`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #18444: [SPARK-16542][SQL][PYSPARK] Fix bugs about types that re...

2017-07-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18444 I believe #18553 will fix the problem. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79267/ Test PASSed. ---

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18485 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18485 **[Test build #79267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79267/testReport)** for PR 18485 at commit [`df61136`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #18553: [SPARK-21327][SQL][PYSPARK] ArrayConstructor should hand...

2017-07-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/18553 cc @holdenk, @HyukjinKwon, @zasdfgbnm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #18553: [SPARK-21327][SQL][PYSPARK] ArrayConstructor should hand...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18553 **[Test build #79274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79274/testReport)** for PR 18553 at commit [`c729eff`](https://github.com/apache/spark/commit/c7

[GitHub] spark pull request #18553: [SPARK-21327][SQL][PYSPARK] ArrayConstructor shou...

2017-07-06 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/18553 [SPARK-21327][SQL][PYSPARK] ArrayConstructor should handle an array of typecode 'l' as long rather than int in Python 2. ## What changes were proposed in this pull request? Currently `Array

[GitHub] spark issue #18552: [Minor][Doc] Improve the docs about how to correctly set...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18552 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18552: [Minor][Doc] Improve the docs about how to correctly set...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18552 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79273/ Test PASSed. ---

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18485 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18552: [Minor][Doc] Improve the docs about how to correctly set...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18552 **[Test build #79273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79273/testReport)** for PR 18552 at commit [`b24ed6d`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79265/ Test PASSed. ---

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18485 **[Test build #79265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79265/testReport)** for PR 18485 at commit [`c995280`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #18523: [SPARK-21285][ML] VectorAssembler reports the col...

2017-07-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/18523#discussion_r125849537 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala --- @@ -113,12 +113,15 @@ class VectorAssembler @Since("1.4.0") (@Since("1.

[GitHub] spark pull request #18523: [SPARK-21285][ML] VectorAssembler reports the col...

2017-07-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/18523#discussion_r125849590 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala --- @@ -113,12 +113,15 @@ class VectorAssembler @Since("1.4.0") (@Since("1.

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125849527 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark pull request #18288: [SPARK-21066][ML] LibSVM load just one input file

2017-07-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18288#discussion_r125848152 --- Diff: mllib/src/main/scala/org/apache/spark/ml/source/libsvm/LibSVMRelation.scala --- @@ -91,12 +91,10 @@ private[libsvm] class LibSVMFileFormat ext

[GitHub] spark issue #18552: [Mindo][Doc] Improve the docs about how to correctly set...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18552 **[Test build #79273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79273/testReport)** for PR 18552 at commit [`b24ed6d`](https://github.com/apache/spark/commit/b2

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125846726 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark pull request #18552: [Mindo][Doc] Improve the docs about how to correc...

2017-07-06 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/18552 [Mindo][Doc] Improve the docs about how to correctly set configurations ## What changes were proposed in this pull request? Spark provides several ways to set configurations, either from

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125841619 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125841472 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125840979 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark pull request #18460: [SPARK-21247][SQL] Type comparision should respec...

2017-07-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18460#discussion_r125840809 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -144,6 +144,8 @@ object TypeCoercion {

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125838555 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark pull request #18460: [SPARK-21247][SQL] Type comparision should respec...

2017-07-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/18460#discussion_r125837832 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -144,6 +144,8 @@ object TypeCoercion {

[GitHub] spark pull request #18307: [SPARK-21100][SQL] Add summary method as alternat...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18307#discussion_r125836508 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2205,47 +2205,80 @@ class Dataset[T] private[sql]( * // max 92.0

[GitHub] spark pull request #18542: [SPARK-21317][SQL] Avoid sorting on bucket expres...

2017-07-06 Thread pwoody
Github user pwoody commented on a diff in the pull request: https://github.com/apache/spark/pull/18542#discussion_r125835525 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala --- @@ -116,13 +116,26 @@ object FileFormatWriter extend

[GitHub] spark issue #18551: [SPARK-21305][ML][MLLIB]Add options to disable multi-thr...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18551 **[Test build #79272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79272/testReport)** for PR 18551 at commit [`d0d94e1`](https://github.com/apache/spark/commit/d0

[GitHub] spark pull request #18551: [SPARK-21305][ML][MLLIB]Add options to disable mu...

2017-07-06 Thread mpjlu
GitHub user mpjlu opened a pull request: https://github.com/apache/spark/pull/18551 [SPARK-21305][ML][MLLIB]Add options to disable multi-threading of native BLAS ## What changes were proposed in this pull request? Many ML/MLLIB algorithms use native BLAS (like Intel MKL, AT

[GitHub] spark pull request #18545: [SPARK-21324][Test] Improve statistics test suite...

2017-07-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18545 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18545: [SPARK-21324][Test] Improve statistics test suites

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18545 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #18550: [Minor][SS][DOCS] Minor doc change for kafka integration

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18550 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18550: [Minor][SS][DOCS] Minor doc change for kafka integration

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18550 **[Test build #79270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79270/testReport)** for PR 18550 at commit [`95937e0`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #18550: [Minor][SS][DOCS] Minor doc change for kafka integration

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18550 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79270/ Test PASSed. ---

[GitHub] spark pull request #18537: [SPARK-20609][MLLIB][TEST]manually cleared 'spark...

2017-07-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18537#discussion_r125831601 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -777,7 +778,20 @@ class ALSSuite } } -clas

[GitHub] spark pull request #18460: [SPARK-21247][SQL] Type comparision should respec...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18460#discussion_r125832472 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -144,6 +144,8 @@ object TypeCoercion {

[GitHub] spark issue #18159: [SPARK-20703][SQL] Associate metrics with data writes on...

2017-07-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18159 Ok. Thanks! @cloud-fan @rxin @adrian-ionescu --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request #18159: [SPARK-20703][SQL] Associate metrics with data wr...

2017-07-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18159 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18253 **[Test build #79271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79271/testReport)** for PR 18253 at commit [`a3b193a`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18253 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79271/ Test FAILed. ---

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18253 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18159: [SPARK-20703][SQL] Associate metrics with data writes on...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18159 thanks, merging to master! we can have follow-up PRs if @rxin has more thoughts about https://github.com/apache/spark/pull/18159#discussion_r125461897 --- If your project is set up for it, you ca

[GitHub] spark pull request #18537: [SPARK-20609][MLLIB][TEST]manually cleared 'spark...

2017-07-06 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/18537#discussion_r125829886 --- Diff: mllib/src/test/scala/org/apache/spark/ml/recommendation/ALSSuite.scala --- @@ -777,7 +778,23 @@ class ALSSuite } } -class

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18253 **[Test build #79271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79271/testReport)** for PR 18253 at commit [`a3b193a`](https://github.com/apache/spark/commit/a3

[GitHub] spark issue #18550: [Minor][SS] Minor doc change for kafka integration

2017-07-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18550 cc @ithjz This may help for your question. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18550: [Minor][SS] Minor doc change for kafka integration

2017-07-06 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18550 cc @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] spark issue #18550: [Minor][SS] Minor doc change for kafka integration

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18550 **[Test build #79270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79270/testReport)** for PR 18550 at commit [`95937e0`](https://github.com/apache/spark/commit/95

[GitHub] spark pull request #18550: [Minor][SS] Minor doc change for kafka integratio...

2017-07-06 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/18550 [Minor][SS] Minor doc change for kafka integration ## What changes were proposed in this pull request? Minor change to kafka integration document for structured streaming. ## How wa

[GitHub] spark pull request #18235: [SPARK-21012][Submit] Add glob support for resour...

2017-07-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18235 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18531: [SPARK-21307] [SQL] Remove SQLConf parameters fro...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18531#discussion_r125829301 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -45,11 +45,9 @@ import org.apache.spark.util.random.Ra

[GitHub] spark issue #18253: [SPARK-18838][CORE] Introduce multiple queues in LiveLis...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18253 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #18235: [SPARK-21012][Submit] Add glob support for resources add...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18235 LGTM, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #79269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79269/testReport)** for PR 17980 at commit [`39c3851`](https://github.com/apache/spark/commit/39

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/17980 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark pull request #18394: [SPARK-20379][core] Allow SSL config to reference...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18394#discussion_r125828226 --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala --- @@ -373,6 +373,11 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logg

[GitHub] spark pull request #18235: [SPARK-21012][Submit] Add glob support for resour...

2017-07-06 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/18235#discussion_r125828051 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -520,7 +520,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark pull request #18442: [SS][Minor] Fix flaky test in DatastreamReaderWri...

2017-07-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18442 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #17862: [SPARK-20602] [ML]Adding LBFGS optimizer and Squared_hin...

2017-07-06 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/17862 I'm in favor of discarding OWLQN. Take LiR or LoR as examples, if you replace LBFGS with OWLQN for regression with L2 regularization, we can saw OWLQN may converge faster than LBFGS in a certain

[GitHub] spark pull request #18535: [SPARK-21312][SQL] correct offsetInBytes in Unsaf...

2017-07-06 Thread sumwale
Github user sumwale commented on a diff in the pull request: https://github.com/apache/spark/pull/18535#discussion_r125827423 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java --- @@ -550,7 +550,7 @@ public void copyFrom(UnsafeRow row) {

[GitHub] spark issue #18425: [SPARK-21217][SQL] Support ColumnVector.Array.toAr...

2017-07-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18425 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #18300: [SPARK-21043][SQL] Add unionByName in Dataset

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18300 **[Test build #79268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79268/testReport)** for PR 18300 at commit [`c43a968`](https://github.com/apache/spark/commit/c4

[GitHub] spark issue #18442: [SS][Minor] Fix flaky test in DatastreamReaderWriterSuit...

2017-07-06 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/18442 Merging this master and 2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishe

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18549 **[Test build #79266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79266/testReport)** for PR 18549 at commit [`b51ab80`](https://github.com/apache/spark/commit/b5

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18485 **[Test build #79267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79267/testReport)** for PR 18485 at commit [`df61136`](https://github.com/apache/spark/commit/df

[GitHub] spark pull request #18235: [SPARK-21012][Submit] Add glob support for resour...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18235#discussion_r125825762 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -520,7 +520,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark pull request #18485: [SPARK-21267][SS][DOCS] Update Structured Streami...

2017-07-06 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/18485#discussion_r125825619 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -1922,6 +1980,39 @@ Not available in R. +### Reporting Metrics using Dropw

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/18549 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #18235: [SPARK-21012][Submit] Add glob support for resour...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18235#discussion_r125825030 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -520,7 +520,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18549 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18549 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79261/ Test FAILed. ---

[GitHub] spark issue #18549: [SPARK-21323][SQL]Rename plans.logical.statsEstimation.R...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18549 **[Test build #79261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79261/testReport)** for PR 18549 at commit [`b51ab80`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17980 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/79264/ Test FAILed. ---

[GitHub] spark issue #17980: [SPARK-20728][SQL] Make ORCFileFormat configurable betwe...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17980 **[Test build #79264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79264/testReport)** for PR 17980 at commit [`39c3851`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18485: [SPARK-21267][SS][DOCS] Update Structured Streaming Docu...

2017-07-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18485 **[Test build #79265 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/79265/testReport)** for PR 18485 at commit [`c995280`](https://github.com/apache/spark/commit/c9

[GitHub] spark pull request #18416: [SPARK-21204][SQL] Add support for Scala Set coll...

2017-07-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18416#discussion_r125824425 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -326,7 +329,10 @@ object ScalaReflection extends ScalaRefle

[GitHub] spark pull request #18416: [SPARK-21204][SQL] Add support for Scala Set coll...

2017-07-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18416#discussion_r125824114 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -326,7 +329,10 @@ object ScalaReflection extends ScalaRe

<    1   2   3   4   5