[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16212 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16212 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69858/ Test FAILed. ---

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-08 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16148 Yeah, 2.1 RC2 has been cut, but I think we still need to merge it into branch-2.1, since it's possible we have new RC. Even if RC2 passed voting, we will have 2.1.1, 2.1.2, ... which need this

[GitHub] spark issue #9352: [SPARK-10849][SQL] Adding field metadata property to over...

2016-12-08 Thread sureshthalamati
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/9352 Opened two new [WIP] PRs to fix this issue using different approaches. https://github.com/apache/spark/pull/16208 https://github.com/apache/spark/pull/16209 Closing this PR.

[GitHub] spark pull request #9352: [SPARK-10849][SQL] Adding field metadata property ...

2016-12-08 Thread sureshthalamati
Github user sureshthalamati closed the pull request at: https://github.com/apache/spark/pull/9352 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16208: [WIP][SPARK-10849][SQL] Adds a new column metadata prope...

2016-12-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16208 @rxin @joshrosen @srowen Which options are preferred for supporting customized column types? Table-level JDBC option or column-level metadata property? Thanks! --- If your project is set up

[GitHub] spark pull request #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHAR...

2016-12-08 Thread maropu
GitHub user maropu opened a pull request: https://github.com/apache/spark/pull/16213 [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END to finish reading closed shards ## What changes were proposed in this pull request? This pr is to fix an issue occurred when resharding

[GitHub] spark pull request #16037: [SPARK-18471][MLLIB] In LBFGS, avoid sending huge...

2016-12-08 Thread AnthonyTruchet
Github user AnthonyTruchet commented on a diff in the pull request: https://github.com/apache/spark/pull/16037#discussion_r91493798 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/LBFGS.scala --- @@ -241,16 +241,24 @@ object LBFGS extends Logging {

[GitHub] spark pull request #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter met...

2016-12-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16017#discussion_r91495968 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -52,33 +52,49 @@ class DecisionTreeClassifier

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16213 **[Test build #69860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69860/consoleFull)** for PR 16213 at commit

[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16212 Most fixes won't matter for Spark because Spark only uses some client-side APIs. Is there a specific fix that's important? It's generally OK to update through maintenance releases regularly,

[GitHub] spark issue #16147: [SPARK-18718][TESTS] Skip some test failures due to path...

2016-12-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16147 (note to myself) I tracked down where the paths are added. It seems mostly they are added in

[GitHub] spark issue #16147: [SPARK-18718][TESTS] Skip some test failures due to path...

2016-12-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16147 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16211: [SPARK-18576][PYTHON] Add basic TaskContext information ...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16211 **[Test build #69857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69857/consoleFull)** for PR 16211 at commit

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-08 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/16214 @felixcheung @shivaram --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16214 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69861/ Test PASSed. ---

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16214 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16214 **[Test build #69861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69861/consoleFull)** for PR 16214 at commit

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16148 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69862/ Test PASSed. ---

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16148 **[Test build #69862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69862/consoleFull)** for PR 16148 at commit

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16213 **[Test build #69863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69863/consoleFull)** for PR 16213 at commit

[GitHub] spark issue #16161: [SPARK-18717][SQL] Make code generation for Scala Map wo...

2016-12-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16161 > there is no type that is assignable to both mutable.Map and immutable.Map Yea, and a general solution would be, remember the original type and create that type when converting catalyst

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16213 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16068: [SPARK-18637][SQL]Stateful UDF should be consider...

2016-12-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16068#discussion_r91526428 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala --- @@ -487,6 +489,26 @@ class HiveUDFSuite extends QueryTest

[GitHub] spark pull request #16147: [SPARK-18718][TESTS] Skip some test failures due ...

2016-12-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16147 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16211: [SPARK-18576][PYTHON] Add basic TaskContext information ...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16211 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16211: [SPARK-18576][PYTHON] Add basic TaskContext information ...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69857/ Test FAILed. ---

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16213 I'm looking into the failure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16148 **[Test build #69862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69862/consoleFull)** for PR 16148 at commit

[GitHub] spark issue #16214: [SPARK-18325][SPARKR] Add example for using native R pac...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16214 **[Test build #69861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69861/consoleFull)** for PR 16214 at commit

[GitHub] spark issue #11871: [SPARK-14050][ML] Add multiple languages support and add...

2016-12-08 Thread Halawa13
Github user Halawa13 commented on the issue: https://github.com/apache/spark/pull/11871 Hello, I'm new to programming pyspark, i have problem with this code **from pyspark.ml.feature import StopWordsRemover df = sc.createDataFrame([(0,["je", "suis", "malade",

[GitHub] spark pull request #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers exam...

2016-12-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16148 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/16212 http://www.openwall.com/lists/oss-security/2016/11/29/1 mentions Hadoop 2.7.x users should upgrade to 2.7.3 and Hadoop 2.6.x users should upgrade to 2.6.5, so if our Hadoop users are moving up to

[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16212 It sounds like that affects the HDFS NameNode, but Spark has nothing to do with that. It shouldn't be incompatible across maintenance releases in any event. Hence occasionally bumping it is

[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16212 **[Test build #69858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69858/consoleFull)** for PR 16212 at commit

[GitHub] spark pull request #16214: [[SPARK-18325]][SPARKR] Add example for using nat...

2016-12-08 Thread yanboliang
GitHub user yanboliang opened a pull request: https://github.com/apache/spark/pull/16214 [[SPARK-18325]][SPARKR] Add example for using native R package. ## What changes were proposed in this pull request? SparkR provides the ability to distribute the computations by

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16213 **[Test build #69860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69860/consoleFull)** for PR 16213 at commit

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16213 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69860/ Test FAILed. ---

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14365 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16187 **[Test build #3478 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3478/consoleFull)** for PR 16187 at commit

[GitHub] spark issue #11045: [SPARK-8321][SQL][WIP] Authorization Support(on all oper...

2016-12-08 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/11045 Hi @winningsix, I am interesting in your idea, and i am confusing about the field `userName` used in your code, 1)where is it be initialized? 2) is it used for privilege checking? --- If your

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16187 **[Test build #3478 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3478/consoleFull)** for PR 16187 at commit

[GitHub] spark issue #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Had...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16212 **[Test build #69858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69858/consoleFull)** for PR 16212 at commit

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-08 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16209 @rxin @joshrosen @srowen Which options are preferred for supporting customized column types? Table-level JDBC option or column-level metadata property? Thanks! --- If your project is set up

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16204 **[Test build #3477 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3477/consoleFull)** for PR 16204 at commit

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16204 cc @ericl --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14365 **[Test build #69856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69856/consoleFull)** for PR 14365 at commit

[GitHub] spark pull request #16211: [SPARK-18576][PYTHON] Add basic TaskContext infor...

2016-12-08 Thread holdenk
GitHub user holdenk opened a pull request: https://github.com/apache/spark/pull/16211 [SPARK-18576][PYTHON] Add basic TaskContext information to PySpark ## What changes were proposed in this pull request? Adds basic TaskContext information to PySpark. ## How was

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16195 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69846/ Test FAILed. ---

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69853/ Test FAILed. ---

[GitHub] spark issue #16193: [SPARK-18766] [SQL] Push Down Filter Through BatchEvalPy...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69850/ Test FAILed. ---

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16187 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69848/ Test FAILed. ---

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16195 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16207 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16204 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16208: [WIP][SPARK-10849][SQL] Adds a new column metadata prope...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16208 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69849/ Test FAILed. ---

[GitHub] spark issue #16193: [SPARK-18766] [SQL] Push Down Filter Through BatchEvalPy...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16193 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14365 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14365 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69852/ Test FAILed. ---

[GitHub] spark issue #16208: [WIP][SPARK-10849][SQL] Adds a new column metadata prope...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16208 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16187 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16207 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69847/ Test FAILed. ---

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16204 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16209 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69855/ Test FAILed. ---

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69854/ Test FAILed. ---

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16209 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16115: [SPARK-18667][PySpark][SQL] Change the way to group row ...

2016-12-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16115 ping @davies again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #11045: [SPARK-8321][SQL][WIP] Authorization Support(on all oper...

2016-12-08 Thread winningsix
Github user winningsix commented on the issue: https://github.com/apache/spark/pull/11045 @yaooqinn thank you for your interest. It's required by Hive side at SessionState. And it's not used for privilege check. Some companies have already used this pull request in their product

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14365 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69856/ Test PASSed. ---

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14365 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16204 **[Test build #3477 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3477/consoleFull)** for PR 16204 at commit

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14365 **[Test build #69856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69856/consoleFull)** for PR 14365 at commit

[GitHub] spark pull request #16017: [SPARK-18592][ML] Move DT/RF/GBT Param setter met...

2016-12-08 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/16017#discussion_r91484503 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/DecisionTreeClassifier.scala --- @@ -52,33 +52,49 @@ class DecisionTreeClassifier

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16213 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16213 **[Test build #69859 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69859/consoleFull)** for PR 16213 at commit

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16213 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69859/ Test FAILed. ---

[GitHub] spark issue #16213: [SPARK-18020][Streaming][Kinesis] Checkpoint SHARD_END t...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16213 **[Test build #69859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69859/consoleFull)** for PR 16213 at commit

[GitHub] spark issue #16211: [SPARK-18576][PYTHON] Add basic TaskContext information ...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16211 **[Test build #69857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69857/consoleFull)** for PR 16211 at commit

[GitHub] spark pull request #16212: [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to ...

2016-12-08 Thread a-roberts
GitHub user a-roberts opened a pull request: https://github.com/apache/spark/pull/16212 [SPARK-18782] [BUILD] Bump Hadoop 2.6 version to use Hadoop 2.6.5 ## What changes were proposed in this pull request? Use Hadoop 2.6.5 for the Hadoop 2.6 profile, I see a bunch of fixes

[GitHub] spark issue #16193: [SPARK-18766] [SQL] Push Down Filter Through BatchEvalPy...

2016-12-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16193 As we can push down predicates through data source scan, those predicates should be already pushed down if they are in the query plan above data source scan node. This seems only work on ExistingRDD

[GitHub] spark issue #16193: [SPARK-18766] [SQL] Push Down Filter Through BatchEvalPy...

2016-12-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16193 If we really want to do this, I'd suggest to push down predicates to rdd scan node during query planning stage. So we don't need to push down predicates to SparkPlan like this. --- If your project

[GitHub] spark issue #11045: [SPARK-8321][SQL][WIP] Authorization Support(on all oper...

2016-12-08 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/11045 @winningsix Glad to hear from you. And we can only use this through Spark ThriftServer, right? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #16189: [SPARK-18761][CORE][WIP] Introduce "task reaper" ...

2016-12-08 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/16189#discussion_r91497545 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -432,6 +435,57 @@ private[spark] class Executor( } /**

[GitHub] spark issue #16196: [SPARK-18231] Optimise SizeEstimator implementation

2016-12-08 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16196 +1 on @srowen's suggestion. This change is not surgical at all. It is going to be difficult to guarantee no behavior change. If anything I'd trade off performance over correctness here. --- If

[GitHub] spark issue #13944: [SPARK-16297][SQL] Fix mapping Microsoft SQLServer diale...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13944 **[Test build #3479 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3479/consoleFull)** for PR 13944 at commit

[GitHub] spark pull request #13579: [SPARK-15844] [core] HistoryServer doesn't come u...

2016-12-08 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/13579#discussion_r91546575 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala --- @@ -349,6 +349,17 @@ class HistoryServerSuite extends

[GitHub] spark pull request #13579: [SPARK-15844] [core] HistoryServer doesn't come u...

2016-12-08 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/13579#discussion_r91546615 --- Diff: core/src/test/scala/org/apache/spark/deploy/history/HistoryServerSuite.scala --- @@ -349,6 +349,17 @@ class HistoryServerSuite extends

[GitHub] spark issue #13579: [SPARK-15844] [core] HistoryServer doesn't come up if sp...

2016-12-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13579 **[Test build #69864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69864/consoleFull)** for PR 13579 at commit

[GitHub] spark issue #13579: [SPARK-15844] [core] HistoryServer doesn't come up if sp...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13579 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69864/ Test FAILed. ---

[GitHub] spark issue #13579: [SPARK-15844] [core] HistoryServer doesn't come up if sp...

2016-12-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13579 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16187 > We currently rely on FileFormat implementations to override toString in order to get a proper explain output. It'd be better to just depend on shortName for those. Looks the PR

[GitHub] spark issue #16115: [SPARK-18667][PySpark][SQL] Change the way to group row ...

2016-12-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16115 good catch! Merging to master/2.1, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16214: [SPARK-18325][SPARKR] Add example for using nativ...

2016-12-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16214#discussion_r91561992 --- Diff: examples/src/main/r/native-r-package.R --- @@ -0,0 +1,80 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark issue #16144: [MINOR][CORE][SQL][DOCS] Typo fixes

2016-12-08 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16144 Ping @jaceklaskowski --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-08 Thread daisukebe
Github user daisukebe commented on the issue: https://github.com/apache/spark/pull/16195 This doesn't look like correlating with my code change. Should I fix this, @vanzin? ``` Traceback (most recent call last): File "./dev/run-tests-jenkins.py", line 228, in

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/13909 ping @cloud-fan --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark pull request #16175: [SPARK-17460][SQL]check if statistics.sizeInBytes...

2016-12-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16175#discussion_r91541946 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala --- @@ -115,7 +115,8 @@ abstract class SparkStrategies extends

[GitHub] spark pull request #16214: [SPARK-18325][SPARKR] Add example for using nativ...

2016-12-08 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16214#discussion_r91561800 --- Diff: examples/src/main/r/native-r-package.R --- @@ -0,0 +1,80 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #16115: [SPARK-18667][PySpark][SQL] Change the way to gro...

2016-12-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16115 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

  1   2   3   4   5   6   >