Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/15983
I see the plan, but the behavior difference will still be affected by the
value of `spark.sql.hive.manageFilesourcePartitions`, right?
I might need more time to chew over it to find out
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/15983
> Should we support EXTERNAL for data source tables?
I'd like to hide the `EXTERNAL` concept to users, and only expose the
`path` option or `LOCATION` syntax, for both data source and
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/15983
We are facing another behavior inconsistency between `CTAS + Append` and
`Insert Into`, because CTAS always repairt the table but InsertInto does not do
it. Below is an example.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/15983
@cloud-fan
First, if this behavior change is required, we need to document it. I think
this is not clear to external users when they do not realize the underlying
change since
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/15983
@gatorsmile I think this behavior change is necessary for the new "Scalable
Partition Handling" feature. BTW, the table created by Spark 2.0 can still be
read by Spark 2.1 without `REPAIR TABLE`,
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/15983
Regarding the concern of the repair cost, I think we still face the same
issue. Each time when we append an extra row, we also repair the table, right?
That is still expensive.
---
If your
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/15983
Yeah, table repair is expensive, but this causes an external behavior
change. I tried it in 2.0. It can show the whole data source table without
repairing the table. In 2.1, it returns empty
Github user ericl commented on the issue:
https://github.com/apache/spark/pull/15983
It's the hive behavior to not repair the table. Otherwise, create table can
have an unbounded cost if there are many partitions.
On Sat, Dec 17, 2016, 5:12 PM Xiao Li
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15983
Merging in master/branch-2.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69281/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69281 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69281/consoleFull)**
for PR 15983 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69281 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69281/consoleFull)**
for PR 15983 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/15983
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69223/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69223 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69223/consoleFull)**
for PR 15983 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/15983
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69045/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69045 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69045/consoleFull)**
for PR 15983 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69045 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69045/consoleFull)**
for PR 15983 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69039/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15983
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69039 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69039/consoleFull)**
for PR 15983 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15983
**[Test build #69039 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69039/consoleFull)**
for PR 15983 at commit
Github user ericl commented on the issue:
https://github.com/apache/spark/pull/15983
cc @yhuai @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so,
27 matches
Mail list logo
