[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14803 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14805 **[Test build #64414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64414/consoleFull)** for PR 14805 at commit

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #64405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64405/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #14766: [SPARK-17197] [ML] [PySpark] PySpark LiR/LoR supports tr...

2016-08-25 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14766 Yes, thanks for review. Merged into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64409/ Test FAILed. ---

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #64409 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64409/consoleFull)** for PR 14750 at commit

[GitHub] spark issue #14698: [SPARK-17061][SPARK-17093][SQL] `MapObjects` should make...

2016-08-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14698 LGTM - merging to master/2.0. Thanks for working on this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14802 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64403/ Test PASSed. ---

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14802 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14802 **[Test build #64403 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64403/consoleFull)** for PR 14802 at commit

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14805 **[Test build #64414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64414/consoleFull)** for PR 14805 at commit

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14805 OK, can you perhaps quickly search for other instances of the same in Python code? it's worth a skim if you're up for it. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14805 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14800 LGTM as a targeted fix --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14805: [MINOR][DOCS] Fix minor typos in python example code

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14805 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #14766: [SPARK-17197] [ML] [PySpark] PySpark LiR/LoR supports tr...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14766 OK, so it's just exposing an existing parameter to python? seems OK. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14744: [SPARK-17178][SPARKR][SPARKSUBMIT] Allow to set sparkr s...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14744 **[Test build #64413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64413/consoleFull)** for PR 14744 at commit

[GitHub] spark pull request #14805: [MINOR][DOCS] Fix minor typos in python example c...

2016-08-25 Thread silentsokolov
GitHub user silentsokolov opened a pull request: https://github.com/apache/spark/pull/14805 [MINOR][DOCS] Fix minor typos in python example code ## What changes were proposed in this pull request? Fix minor typos python example code in streaming programming guide

[GitHub] spark issue #14766: [SPARK-17197] [ML] [PySpark] PySpark LiR/LoR supports tr...

2016-08-25 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14766 @srowen Would you mind to have a look at this one? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #14744: [SPARK-17178][SPARKR][SPARKSUBMIT] Allow to set s...

2016-08-25 Thread zjffdu
Github user zjffdu commented on a diff in the pull request: https://github.com/apache/spark/pull/14744#discussion_r76204928 --- Diff: docs/configuration.md --- @@ -1752,6 +1752,14 @@ showDF(properties, numRows = 200, truncate = FALSE) Executable for executing R scripts in

[GitHub] spark issue #14663: [SPARK-17001] [ML] Enable standardScaler to standardize ...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14663 I'll go for this tomorrow if there are no other comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14804 Meh, it's still ambiguous and there's a defined way to disambiguate, so it's unfortunate, but I'm OK with a step towards consistency in any event. --- If your project is set up for it, you can

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14800 **[Test build #64412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64412/consoleFull)** for PR 14800 at commit

[GitHub] spark issue #14637: [SPARK-16967] move mesos to module

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14637 Nice one, LGTM. I'll leave it open for final comments until tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14804 I think [here](http://ux.stackexchange.com/questions/13815/files-size-units-kib-vs-kb-vs-kb) has a precise definition. AFAIK in Spark the conversion is 1024 based either KB, K, or kb, KiB is not

[GitHub] spark issue #10896: [SPARK-12978][SQL] Skip unnecessary final group-by when ...

2016-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/10896 @hvanhovell could you also give me comments on #13852? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn't work

2016-08-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/14433 that's a good point actually - how about we use `args.primaryResource` or `args.isR` that already exists in SparkSubmit? --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #14786: [SPARK-17212][SQL] TypeCoercion supports widening conver...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14786 **[Test build #64411 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64411/consoleFull)** for PR 14786 at commit

[GitHub] spark pull request #14760: [SPARK-17193] [CORE] HadoopRDD NPE at DEBUG log l...

2016-08-25 Thread srowen
Github user srowen closed the pull request at: https://github.com/apache/spark/pull/14760 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14760: [SPARK-17193] [CORE] HadoopRDD NPE at DEBUG log level wh...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14760 Merged to master/2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14796: [SPARK-17229][SQL] PostgresDialect shouldn't widen float...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14796 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64402/ Test PASSed. ---

[GitHub] spark issue #14796: [SPARK-17229][SQL] PostgresDialect shouldn't widen float...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14796 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14796: [SPARK-17229][SQL] PostgresDialect shouldn't widen float...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14796 **[Test build #64402 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64402/consoleFull)** for PR 14796 at commit

[GitHub] spark pull request #14744: [SPARK-17178][SPARKR][SPARKSUBMIT] Allow to set s...

2016-08-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/14744#discussion_r76203143 --- Diff: docs/configuration.md --- @@ -1752,6 +1752,14 @@ showDF(properties, numRows = 200, truncate = FALSE) Executable for executing R

[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14804 Ugh, yeah that's wrong in the sense that we are not showing MB, but MiB. I'd favor fixing the labels here and in Utils.bytesToString? Then again, I see that we will also parse input of

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14801 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64404/ Test FAILed. ---

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14801 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14801 **[Test build #64404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64404/consoleFull)** for PR 14801 at commit

[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14804 Because in the log it shows Memory MB in 1024 based, while in the web UI it is 1000 based, so this is slightly different. You could check `Utils#bytesToString`. I think we unify this.

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/14800 No problem, thanks your attention :) okay, I'll remove this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14800 I am okay with both too. I apologise for the irrelevant comment @maropu . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14804 KB = 1000 bytes, KiB = 1024 bytes. According to the suffixes we're using, 1000 is correct at the moment. Is the display inconsistent with something else in the UI or logs? --- If your project is

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/14800 yea, I see. I also have no strong opinion on this. So, both is okay to me. For now, I'll remove the requirement. What do u think? cc: @HyukjinKwon --- If your project is set up for it, you can

[GitHub] spark pull request #14786: [SPARK-17212][SQL] TypeCoercion supports widening...

2016-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/14786#discussion_r76200868 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -134,6 +134,8 @@ object TypeCoercion {

[GitHub] spark issue #14798: [SPARK-17231][CORE] Avoid building debug or trace log me...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14798 Seems fine to me. I think you'd be welcome to fix up the other log messages you see in these files to use {} placeholders, but that's entirely optional. --- If your project is set up for it, you

[GitHub] spark pull request #14798: [SPARK-17231][CORE] Avoid building debug or trace...

2016-08-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/14798#discussion_r76200529 --- Diff: common/network-common/src/main/java/org/apache/spark/network/server/TransportChannelHandler.java --- @@ -29,7 +29,7 @@ import

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14800 True, but, in the with-replacement case, you're no longer selecting a subset to begin with, because an element can appear twice. "Sample" does generally mean "take a smaller set" but it also means

[GitHub] spark issue #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14804 **[Test build #64410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64410/consoleFull)** for PR 14804 at commit

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/8880 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64401/ Test PASSed. ---

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/8880 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14804: [MINOR][Web UI] Correctly convert bytes in web UI

2016-08-25 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/14804 [MINOR][Web UI] Correctly convert bytes in web UI ## What changes were proposed in this pull request? should be 1024 based, not 1000. ## How was this patch tested?

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/8880 **[Test build #64401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64401/consoleFull)** for PR 8880 at commit

[GitHub] spark issue #14783: SPARK-16785 R dapply doesn't return array or raw columns

2016-08-25 Thread clarkfitzg
Github user clarkfitzg commented on the issue: https://github.com/apache/spark/pull/14783 Not sure why these timings are so bad. Found out today that by using bytes and calling directly into Java's `org.apache.spark.api.r.RRDD` these can be improved by 2 orders of magnitude. --- If

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/14800 In the definition of statistic terms, Sampling is to select a `subset` of whole data So, I think the sample rate to be <= 1 is more reasonable. See:

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #64409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64409/consoleFull)** for PR 14750 at commit

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14800 @srowen Actually, we are already enforcing it to 100% when the replacement is disabled. So, I suggested this to match this up when it is enabled. Yes, it seems not related with the bug this PR

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14802 Looks like this is a little similar to this one #13513 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14579: [SPARK-16921][PYSPARK] RDD/DataFrame persist()/cache() s...

2016-08-25 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14579 I like it personally - if no one has a good reason why not it seems like a very reasonable approach. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #14786: [SPARK-17212][SQL] TypeCoercion supports widening conver...

2016-08-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14786 MySQL and PostgreSQL support this **MySQL** - Greatest/leastest ```sql mysql> SELECT GREATEST(CAST("1990-02-24 12:00:00" AS DATETIME), CAST("1990-02-25" AS DATE));

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14801 Can we avoid introducing new exception types? It is super annoying to match those in Python. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14800 Also, is it really necessary to limit the sample rate to be <= 1? It's not incoherent to want to sample 200% of a data set if it is with replacement. You'd just be generating a data set 2x the size

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/8880 **[Test build #64408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64408/consoleFull)** for PR 8880 at commit

[GitHub] spark issue #8880: [SPARK-5682][Core] Add encrypted shuffle in spark

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/8880 **[Test build #64407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64407/consoleFull)** for PR 8880 at commit

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14617 **[Test build #64406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64406/consoleFull)** for PR 14617 at commit

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14800 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64400/ Test PASSed. ---

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14800 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14800: [SPARK-15382][SQL] Fix a bug in sampling with replacemen...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14800 **[Test build #64400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64400/consoleFull)** for PR 14800 at commit

[GitHub] spark issue #14617: [SPARK-17019][Core] Expose on-heap and off-heap memory u...

2016-08-25 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/14617 @mallman I changed the UI based on your comment, here is the new one (separate the on heap and off heap memory usage in two columns): ![screen shot 2016-08-25 at 3 28 31

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14803 **[Test build #64405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64405/consoleFull)** for PR 14803 at commit

[GitHub] spark issue #14803: [SPARK-17153][SQL] Should read partition data when readi...

2016-08-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14803 cc @marmbrus --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #14433: [SPARK-16829][SparkR]:sparkR sc.setLogLevel doesn't work

2016-08-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/14433 It feels like some overkill unless there are going to be more uses for changing logic based on whether it's running a shell. It seems not so bad to define `setRootLevel` in Scala as an alias when in

[GitHub] spark pull request #14803: [SPARK-17153][SQL] Should read partition data whe...

2016-08-25 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/14803 [SPARK-17153][SQL] Should read partition data when reading new files in filestream without globbing ## What changes were proposed in this pull request? When reading file stream with

[GitHub] spark issue #11119: [SPARK-10780][ML][WIP] Add initial model to kmeans

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64398/ Test PASSed. ---

[GitHub] spark issue #11119: [SPARK-10780][ML][WIP] Add initial model to kmeans

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/9 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #11119: [SPARK-10780][ML][WIP] Add initial model to kmeans

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/9 **[Test build #64398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64398/consoleFull)** for PR 9 at commit

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14802 **[Test build #64403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64403/consoleFull)** for PR 14802 at commit

[GitHub] spark issue #14801: [SPARK-17234] [SQL] Table Existence Checking when Index ...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14801 **[Test build #64404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64404/consoleFull)** for PR 14801 at commit

[GitHub] spark pull request #14802: [SPARK-17235][SQL] Support purging of old logs in...

2016-08-25 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/14802#discussion_r76191571 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLogSuite.scala --- @@ -155,8 +174,8 @@ class

[GitHub] spark pull request #14802: [SPARK-17235][SQL] Support purging of old logs in...

2016-08-25 Thread petermaxlee
GitHub user petermaxlee opened a pull request: https://github.com/apache/spark/pull/14802 [SPARK-17235][SQL] Support purging of old logs in MetadataLog ## What changes were proposed in this pull request? This patch adds a purge interface to MetadataLog, and an implementation in

[GitHub] spark issue #14802: [SPARK-17235][SQL] Support purging of old logs in Metada...

2016-08-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/14802 @tdas and @zsxwing can you take a look at this? It's a pretty simple change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #14801: [SPARK-17234] [SQL] Table Existence Checking when...

2016-08-25 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/14801 [SPARK-17234] [SQL] Table Existence Checking when Index Table with the Same Name Exists ### What changes were proposed in this pull request? Hive Index tables are not supported by Spark

[GitHub] spark pull request #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14746#discussion_r76190937 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -105,7 +105,13 @@ case class CreateViewCommand( }

[GitHub] spark pull request #14753: [SPARK-17187][SQL] Supports using arbitrary Java ...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14753#discussion_r76189865 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/TypedImperativeAggregateSuite.scala --- @@ -0,0 +1,300 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #13780: [SPARK-16063][SQL] Add storageLevel to Dataset

2016-08-25 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/13780 ping @rxin @marmbrus @davies @gatorsmile for comment on the Python storage level issue I mention at https://github.com/apache/spark/pull/13780#discussion_r67833027 --- If your project is set up

[GitHub] spark pull request #14537: [SPARK-16948][SQL] Querying empty partitioned orc...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14537#discussion_r76189706 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcQuerySuite.scala --- @@ -372,6 +373,40 @@ class OrcQuerySuite extends QueryTest with

[GitHub] spark issue #14579: [SPARK-16921][PYSPARK] RDD/DataFrame persist()/cache() s...

2016-08-25 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/14579 @nchammas @holdenk @davies @rxin how about the approach of @MechCoder in https://github.com/apache/spark/pull/14579#discussion_r74813935? I think this will work well, so we could raise an

[GitHub] spark pull request #14537: [SPARK-16948][SQL] Querying empty partitioned orc...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14537#discussion_r76189474 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala --- @@ -54,10 +57,12 @@ class OrcFileFormat extends FileFormat with

[GitHub] spark pull request #14537: [SPARK-16948][SQL] Querying empty partitioned orc...

2016-08-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14537#discussion_r76189247 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala --- @@ -237,21 +237,27 @@ private[hive] class

[GitHub] spark pull request #14579: [SPARK-16921][PYSPARK] RDD/DataFrame persist()/ca...

2016-08-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/14579#discussion_r76189167 --- Diff: python/pyspark/rdd.py --- @@ -188,6 +188,12 @@ def __init__(self, jrdd, ctx, jrdd_deserializer=AutoBatchedSerializer(PickleSeri

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64399/ Test PASSed. ---

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14537 **[Test build #64399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64399/consoleFull)** for PR 14537 at commit

[GitHub] spark issue #14698: [SPARK-17061][SPARK-17093][SQL] `MapObjects` should make...

2016-08-25 Thread lw-lin
Github user lw-lin commented on the issue: https://github.com/apache/spark/pull/14698 Thanks @hvanhovell for the review! This patch has been updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ALTER V...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14746 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64396/ Test PASSed. ---

[GitHub] spark issue #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ALTER V...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14746 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14746: [SPARK-17180] [SQL] Fix View Resolution Order in ALTER V...

2016-08-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14746 **[Test build #64396 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64396/consoleFull)** for PR 14746 at commit

[GitHub] spark issue #14762: [SPARK-16962][CORE][SQL] Fix misaligned record accesses ...

2016-08-25 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14762 Does your change have any performance impact? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14790: [SPARK-17215][SQL] Method `SQLContext.parseDataTy...

2016-08-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14790 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14537: [SPARK-16948][SQL] Querying empty partitioned orc tables...

2016-08-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14537 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64397/ Test PASSed. ---

<    1   2   3   4   5   6   >