GitHub user sounakr opened a pull request:
https://github.com/apache/incubator-carbondata/pull/723
[CARBONDATA-845] Insert Select Into Same Table
Problem : Insert Select Implementation in current Spark-2.1 prohibits to
read and write into same table.
Analysis : This restriction is imposed by spark in case InsertableRelation
is extended. Current insert select implementation in spark-2.1
CarbonDatasourceHadoopRelation extends InsertableRelation and override the
insert method. Spark rules in case of InsertableRelation checks if source and
target tables are same, then it fails and prompts "Cannot insert overwrite
into table that is also being read from."
Fix:: Spark 1.5 implementation of insert select in carbon allows reading
and writing into same table, so as Hive and Parquet. Therfore as part of
current fix reimplemented insert-select logic of Spark-2.1 similar to
Spark-1.5.
a) CarbonDatasourceHadoopRelation doesn't extends InsertableRelation.
b) New CarbonPreInsertionCasts Rule is written for spark-2.1 (For
Spark-1.5 it was already present).
d) This new Rule is registered CarbonSessionState,
c) During Insert Select processing the Logical Plan wrapped by
InsertIntoCarbonTable and later execute LoadTableByInsert to process insert
select.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sounakr/incubator-carbondata insert_select
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/723.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #723
----
commit beb26625dc8698574f9be4701a0b80504097f049
Author: sounakr <[email protected]>
Date: 2017-04-03T07:18:20Z
Insert Select Into Same Table
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
