[GitHub] spark issue #16210: [Core][SPARK-18778]Fix the scala classpath under some en...

2016-12-07 Thread djvulee
Github user djvulee commented on the issue: https://github.com/apache/spark/pull/16210 @rxin our jdk is jdk1.8.0_91, and we do not install the scala, the OS is Debian 4.6.4. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16209 **[Test build #69855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69855/consoleFull)** for PR 16209 at commit

[GitHub] spark issue #16210: [Core][SPARK-18778]Fix the scala classpath under some en...

2016-12-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16210 Paging @jodersky I'm not sure about this ... isn't `-usejavacp` a legacy option? I don't know of any other reports of this so it is possibly specific to your env. --- If your project is set up

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16148 LGTM. Looks like we are locked down for 2.1. Good to have with all the new examples but seems like a lot of code (example) changes? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16210: [Core][SPARK-18778]Fix the scala classpath under some en...

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16210 What are the environments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16158: [SPARK-18724][ML] Add TuningSummary for TrainValidationS...

2016-12-07 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/16158 @MLnick Does this match your thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16014: [SPARK-18590][SPARKR] build R source package when...

2016-12-07 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16014#discussion_r91459472 --- Diff: dev/create-release/release-build.sh --- @@ -221,14 +235,13 @@ if [[ "$1" == "package" ]]; then # We increment the Zinc port each

[GitHub] spark issue #16150: [SPARK-18349][SparkR]:Update R API documentation on ml m...

2016-12-07 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/16150 LGTM. @rxin I know rc2 has been cut but can this still go to branch-2.1? There're only 2 lines of code change and API doc improvements could really help usability in 2.1. + @shivaram

[GitHub] spark issue #16210: [Core][SPARK-18778]Fix the scala classpath under some en...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16210 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69843/ Test PASSed. ---

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16204 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16204 **[Test build #69843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69843/consoleFull)** for PR 16204 at commit

[GitHub] spark pull request #16210: [Core][SPARK-18778]Fix the scala classpath under ...

2016-12-07 Thread djvulee
GitHub user djvulee opened a pull request: https://github.com/apache/spark/pull/16210 [Core][SPARK-18778]Fix the scala classpath under some environment ## What changes were proposed in this pull request? under some environment, the Dscala.usejavacp=true option seems not work,

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2016-12-07 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/16043 @HyukjinKwon , thanks for the quick response :-) I'll tackle these later today (gott'a work sometimes) --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91453918 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91454208 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91455499 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91455356 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91457070 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/complexTypesSuite.scala --- @@ -0,0 +1,482 @@ +/* + * Licensed to

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91453489 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,78 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91453426 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91453593 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91457208 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/complexTypesSuite.scala --- @@ -0,0 +1,482 @@ +/* + * Licensed to

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r9148 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/complexTypesSuite.scala --- @@ -0,0 +1,482 @@ +/* + * Licensed to

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91454252 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r91457315 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/complexTypesSuite.scala --- @@ -0,0 +1,482 @@ +/* + * Licensed to

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16204 **[Test build #69854 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69854/consoleFull)** for PR 16204 at commit

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16204 **[Test build #69853 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69853/consoleFull)** for PR 16204 at commit

[GitHub] spark issue #14282: [SPARK-16628][SQL] Don't convert Orc Metastore tables to...

2016-12-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14282 This can be closed now because we don't infer schema from Orc files when converting Hive Orc tables to data source tables anymore. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #14282: [SPARK-16628][SQL] Don't convert Orc Metastore ta...

2016-12-07 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/14282 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16135: [SPARK-18700][SQL] Add ReadWriteLock for each tab...

2016-12-07 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16135#discussion_r91455760 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -95,7 +95,7 @@ private[sql] class HiveSessionCatalog( }

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16209 **[Test build #69851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69851/consoleFull)** for PR 16209 at commit

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16209 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69851/ Test FAILed. ---

[GitHub] spark pull request #16135: [SPARK-18700][SQL] Add ReadWriteLock for each tab...

2016-12-07 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/16135#discussion_r91455755 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -105,7 +105,8 @@ private[sql] class HiveSessionCatalog( //

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14365 @cloud-fan @yhuai @dongjoon-hyun I've updated this as: * Assume metastore schema matches with physical Orc schema by column, disregarding column names. * Mapping required schema to

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16209 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC data sou...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16209 **[Test build #69851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69851/consoleFull)** for PR 16209 at commit

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14365 **[Test build #69852 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69852/consoleFull)** for PR 14365 at commit

[GitHub] spark pull request #16209: [WIP][SPARK-10849][SQL] Adds option to the JDBC d...

2016-12-07 Thread sureshthalamati
GitHub user sureshthalamati opened a pull request: https://github.com/apache/spark/pull/16209 [WIP][SPARK-10849][SQL] Adds option to the JDBC data source for user to specify database column type for the create table ## What changes were proposed in this pull request? Currently

[GitHub] spark issue #16193: [SPARK-18766] [SQL] Push Down Filter Through BatchEvalPy...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16193 **[Test build #69850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69850/consoleFull)** for PR 16193 at commit

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add ReadWriteLock for each table's re...

2016-12-07 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/16135 I guess the large number of lock sites is confusing me. We only want to prevent concurrent instantiation of a single table, so shouldn't you only need 1 lock for that site? Also, we should

[GitHub] spark issue #16208: [WIP][SPARK-10849][SQL] Adds a new column metadata prope...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16208 **[Test build #69849 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69849/consoleFull)** for PR 16208 at commit

[GitHub] spark issue #16193: [SPARK-18766] [SQL] Push Down Filter Through BatchEvalPy...

2016-12-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16193 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16208: [WIP][SPARK-10849][SQL] Adds a new column metadat...

2016-12-07 Thread sureshthalamati
GitHub user sureshthalamati opened a pull request: https://github.com/apache/spark/pull/16208 [WIP][SPARK-10849][SQL] Adds a new column metadata property to the jdbc data source for users to specify database column type using the metadata ## What changes were proposed in this pull

[GitHub] spark pull request #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial...

2016-12-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16149#discussion_r91453574 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -215,6 +215,7 @@ class GeneralizedLinearRegression

[GitHub] spark pull request #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial...

2016-12-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16149#discussion_r91453386 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -715,7 +715,7 @@ class

[GitHub] spark pull request #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial...

2016-12-07 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16149#discussion_r91453428 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -468,11 +469,7 @@ object GeneralizedLinearRegression

[GitHub] spark pull request #16203: [SPARK-18774][Core][SQL]Ignore non-existing files...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16203 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16207 Yea i'd say in general that `--allow-empty` be there. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16203: [SPARK-18774][Core][SQL]Ignore non-existing files when i...

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16203 Actually this doesn't merge cleanly in branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16203: [SPARK-18774][Core][SQL]Ignore non-existing files when i...

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16203 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16207 Yes I have to edit the merge script to `git commit --allow-empty ...` but I don't know that we should always set it. It could prompt or something but I was too lazy to implement that.

[GitHub] spark issue #16202: [SPARK-18662][hotfix] Add new resource-managers director...

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16202 Was mesos never included? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #12064: [SPARK-14272][ML] Evaluate GaussianMixtureModel with Log...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12064 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69845/ Test PASSed. ---

[GitHub] spark issue #12064: [SPARK-14272][ML] Evaluate GaussianMixtureModel with Log...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12064 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12064: [SPARK-14272][ML] Evaluate GaussianMixtureModel with Log...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12064 **[Test build #69845 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69845/consoleFull)** for PR 12064 at commit

[GitHub] spark pull request #14640: [SPARK-17055] [MLLIB] add groupKFold to CrossVali...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14640 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16187 **[Test build #69848 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69848/consoleFull)** for PR 16187 at commit

[GitHub] spark pull request #15917: SPARK-18252: Using RoaringBitmap for bloom filter...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15917 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #15689: [SPARK-9487] Use the same num. worker threads in ...

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15689 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16188: Branch 1.6 decision tree

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16188 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #16206: Branch 2.0

2016-12-07 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16206 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16207 Sure, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16207 BTW, I believe we should add `--allow-empty` when it is merged. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/16207 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16207 For some reason I couldn't merge this one. I pushed a commit directly to master. Can you close this one now? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16187: [SPARK-18760][SQL] Consistent format specification for F...

2016-12-07 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16187 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16207 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2016-12-07 Thread eyalfa
Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/16043 @hvanhovell, can you please comment on the latest changes? @gatorsmile, @HyukjinKwon, I think I've sorted out most of the formatting issues you guys mentioned, please let me know I missed

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16207 **[Test build #69847 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69847/consoleFull)** for PR 16207 at commit

[GitHub] spark issue #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16207 cc @srowen Could you take a look and see if they are reasonable please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16135: [SPARK-18700][SQL] Add ReadWriteLock for each table's re...

2016-12-07 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/16135 @ericl Thanks for your review. > Is it sufficient to lock around the catalog.filterPartitions(Nil)? Yes, this patch port from 1.6.2 and I missed the diff here. Fixed in next patch.

[GitHub] spark pull request #16207: [BUILD] Closing some stale/inappropriate PRs

2016-12-07 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/16207 [BUILD] Closing some stale/inappropriate PRs ## What changes were proposed in this pull request? This PR proposes to close some stale PRs and ones suggested to be closed by

[GitHub] spark issue #16205: [SPARK-18776][SS] Make Offset for FileStreamSource corre...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16205 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16205: [SPARK-18776][SS] Make Offset for FileStreamSource corre...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16205 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69841/ Test PASSed. ---

[GitHub] spark issue #16205: [SPARK-18776][SS] Make Offset for FileStreamSource corre...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16205 **[Test build #69841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69841/consoleFull)** for PR 16205 at commit

[GitHub] spark pull request #16199: [SPARK-18772][SQL] NaN/Infinite float parsing in ...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/16199#discussion_r91450072 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1764,4 +1764,37 @@ class JsonSuite extends

[GitHub] spark issue #16199: [SPARK-18772][SQL] NaN/Infinite float parsing in JSON is...

2016-12-07 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16199 @NathanHowell, while tracking down the history, I found similar PR including this in https://github.com/apache/spark/pull/9759/files#diff-8affe5ec7d691943a88e43eb30af656e (this seems reverted

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16195 **[Test build #69846 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69846/consoleFull)** for PR 16195 at commit

[GitHub] spark issue #16206: Branch 2.0

2016-12-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16206 @ming616 please close this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16206: Branch 2.0

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16206 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16119: [SPARK-18687][Pyspark][SQL]Backward compatibility - crea...

2016-12-07 Thread vijoshi
Github user vijoshi commented on the issue: https://github.com/apache/spark/pull/16119 @holdenk test case added --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #12064: [SPARK-14272][ML] Evaluate GaussianMixtureModel with Log...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12064 **[Test build #69845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69845/consoleFull)** for PR 12064 at commit

[GitHub] spark pull request #16206: Branch 2.0

2016-12-07 Thread ming616
GitHub user ming616 opened a pull request: https://github.com/apache/spark/pull/16206 Branch 2.0 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how this patch

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16195 **[Test build #69844 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69844/consoleFull)** for PR 16195 at commit

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16195 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16195 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69844/ Test FAILed. ---

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16195 **[Test build #69844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69844/consoleFull)** for PR 16195 at commit

[GitHub] spark issue #16204: [SPARK-18775][SQL] Limit the max number of records writt...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16204 **[Test build #69843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69843/consoleFull)** for PR 16204 at commit

[GitHub] spark issue #16195: [Spark-18765] [CORE] Make values for spark.yarn.{am|driv...

2016-12-07 Thread daisukebe
Github user daisukebe commented on the issue: https://github.com/apache/spark/pull/16195 Per @vanzin's suggestion, - revised the code style, - dded a new default variable, - and also fixed the warning: " --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-07 Thread windpiger
Github user windpiger commented on the issue: https://github.com/apache/spark/pull/16173 ok,someone else can tell if it is resonable to readd the conf now? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16157: [SPARK-18723][DOC] Expanded programming guide inf...

2016-12-07 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/16157#discussion_r91445291 --- Diff: docs/programming-guide.md --- @@ -347,7 +347,7 @@ Some notes on reading files with Spark: Apart from text files, Spark's Scala API also

[GitHub] spark issue #16173: [SPARK-18742][CORE]readd spark.broadcast.factory conf to...

2016-12-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16173 Yeah those were Spark classes. This property doesn't seem to be used now. It's possible to restore this but I don't know if it's intended now. Yes I suppose you could update the comment instead,

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14365 We have two options. First one is to map metastore schema to physical Orc schema like this. But we don't infer physical schema of Orc file now. I will update this to have this mapping in

[GitHub] spark issue #14365: [SPARK-16628][SQL] Translate file-based relation schema ...

2016-12-07 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14365 @dongjoon-hyun yeah, I see. Because we directly use metastore schema of converted Orc table, when the physical schema in Orc file and metastore schema mismatch, this issue happens. --- If your

[GitHub] spark issue #16201: [SPARK-3359][DOCS] Fix greater-than symbols in Javadoc t...

2016-12-07 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/16201 I see, so this is just another case where changes will keep breaking this. We do need a build that can run this at some point soon here to catch it. But yes, just keep fixing. I would just

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16148 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69842/ Test PASSed. ---

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16148 **[Test build #69842 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69842/consoleFull)** for PR 16148 at commit

[GitHub] spark issue #16148: [SPARK-18325][SparkR][ML] SparkR ML wrappers example cod...

2016-12-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16148 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16149: [SPARK-18715][ML]Fix AIC calculations in Binomial GLM

2016-12-07 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16149 @sethah @srowen I have added a comment to the weigthCol doc for the Binomial case. I also updated to test the case `weight < 0.5`, i.e., `round(weight) = 0`. All tests passed. --- If

  1   2   3   4   5   6   >