Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18344
**[Test build #78228 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78228/testReport)**
for PR 18344 at commit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18334#discussion_r122600223
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -161,6 +161,11 @@ case
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18334
These commands will automatically trigger the stats updates, which could be
expensive. Another way is to simply set it to zero or mark it unreliable? Can
we provide a SQLConf conf for this?
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18334
How about add a partition?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/18343
@jinxing64 `big_table` may be need big enough, my `big_table` is 270.7 G:
```sql
spark-sql -e "
set spark.sql.shuffle.partitions=2001;
drop table if exists spark_hcms_npe;
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/18345
Hi, @srowen . This is the new PR for master branch .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18309
**[Test build #78231 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78231/testReport)**
for PR 18309 at commit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18309#discussion_r122599863
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzeTableCommand.scala
---
@@ -81,6 +83,21 @@ case class
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18309
ok to test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18343
**[Test build #78230 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78230/testReport)**
for PR 18343 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18345
**[Test build #78229 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78229/testReport)**
for PR 18345 at commit
GitHub user dongjoon-hyun opened a pull request:
https://github.com/apache/spark/pull/18345
[MINOR][BUILD] Fix Java linter errors
## What changes were proposed in this pull request?
This PR cleans up a few Java linter errors for Apache Spark 2.2 release.
## How was
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/18344
Sure. Thank you!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18344
Yeah you can start with `master` for completeness. If the changes for 2.2
are pretty similar, we can just back-port that. If it's very different, OK, we
can merge a manually crafted change for 2.2
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/18344
For now, it's only for 2.2. In `master` branch, there are 26 errors.
Shall I create another PR for master branch, too?
---
If your project is set up for it, you can reply to this email
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18344
Were this already in master? or only in 2.2?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18303
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18303
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78227/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18303
**[Test build #78227 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78227/testReport)**
for PR 18303 at commit
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/18341
+1 with what srowen says.
ditto with python actually
https://github.com/apache/spark/blob/master/python/pyspark/version.py
i wonder if we should simply change the release/branch process
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18335
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/18335
merged to master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18344
**[Test build #78228 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78228/testReport)**
for PR 18344 at commit
GitHub user dongjoon-hyun opened a pull request:
https://github.com/apache/spark/pull/18344
[MINOR][BUILD] Fix Java linter errors
## What changes were proposed in this pull request?
This PR cleans up a few Java linter errors for Apache Spark 2.2 release.
## How was
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18253
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18253
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78226/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18253
**[Test build #78226 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78226/testReport)**
for PR 18253 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18303
LGTM except a few comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596758
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596741
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596663
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596651
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596647
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596621
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596609
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596599
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596579
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596590
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596570
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596562
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596533
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596518
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596434
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18303#discussion_r122596343
--- Diff: core/src/main/scala/org/apache/spark/deploy/JsonProtocol.scala ---
@@ -21,74 +21,192 @@ import org.json4s.JsonAST.JObject
import
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17758#discussion_r122596253
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala ---
@@ -248,6 +249,10 @@ private[hive] class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17758#discussion_r122595919
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -328,6 +333,9 @@ case class DataSource(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17758#discussion_r122595783
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -182,6 +183,10 @@ case class DataSource(
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17758#discussion_r122595701
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/util/SchemaUtils.scala ---
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/17758#discussion_r122595287
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/test/DataFrameReaderWriterSuite.scala
---
@@ -687,4 +688,52 @@ class DataFrameReaderWriterSuite
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18303
**[Test build #78227 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78227/testReport)**
for PR 18303 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18253
**[Test build #78226 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78226/testReport)**
for PR 18253 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18253
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78225/
Test PASSed.
---
Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/18343
Thanks for ping.
If I understand correctly, `HighlyCompressedStatus` is initialized when 2
situations:
1. Creating `MapStatus` when shuffle-write and the reduce partitions is
over 2000;
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18253
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18253
**[Test build #78225 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78225/testReport)**
for PR 18253 at commit
Github user bbossy commented on the issue:
https://github.com/apache/spark/pull/18269
@mallman I'm not sure where this difference in behaviour is coming from.
The following test in `FileIndexSuite` passes:
```
test("mallman's scenario") {
Github user bbossy commented on the issue:
https://github.com/apache/spark/pull/18269
@cloud-fan Could you take another look, please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18253
**[Test build #78225 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78225/testReport)**
for PR 18253 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18343
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78224/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18343
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18343
**[Test build #78224 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78224/testReport)**
for PR 18343 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18338
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78223/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18338
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18338
**[Test build #78223 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78223/testReport)**
for PR 18338 at commit
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/18338#discussion_r122585242
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
---
@@ -231,7 +231,6 @@ class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18342
**[Test build #3801 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3801/testReport)**
for PR 18342 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18342
**[Test build #3801 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3801/testReport)**
for PR 18342 at commit
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/17758#discussion_r122584251
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JsonDataSource.scala
---
@@ -59,9 +59,7 @@ abstract class JsonDataSource
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/18343
cc @jinxing64
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18336
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18336
Merged to master/2.2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18343
**[Test build #78224 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78224/testReport)**
for PR 18343 at commit
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18343
I'm not sure that's a valid fix. This makes this field serialize, when it
wasn't intended to. It's either supposed to be recreated on demand, or else,
the code needs to deal with it not existing.
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/18343
[SPARK-21133][CORE] Fix HighlyCompressedMapStatus#writeExternal throws NPE
## What changes were proposed in this pull request?
Fix HighlyCompressedMapStatus#writeExternal NPE:
```
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18333
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18333
Merged to master/2.2
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18338
**[Test build #78223 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78223/testReport)**
for PR 18338 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/18338
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18338
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18338
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78222/
Test FAILed.
---
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/18338#discussion_r122583408
--- Diff:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ---
@@ -561,6 +561,7 @@ primaryExpression
| CASE
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18342
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
GitHub user assafmendelson opened a pull request:
https://github.com/apache/spark/pull/18342
[Spark 21123][Docs][Structured Streaming] Options for file stream source
are in a wrong table
## What changes were proposed in this pull request?
The description for several
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18338
**[Test build #78222 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78222/testReport)**
for PR 18338 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18341
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18341
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78221/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18341
**[Test build #78221 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78221/testReport)**
for PR 18341 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/18340
+1, LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or
201 - 288 of 288 matches
Mail list logo
