GitHub user yhuai opened a pull request:
https://github.com/apache/spark/pull/4373
[SPARK-5595][SPARK-5603][SQL] Add a rule to do PreInsert type casting and
field renaming and invalidating in memory cache after INSERT
This PR adds a rule to Analyzer that will add preinsert data type casting
and field renaming to the select clause in an `INSERT INTO/OVERWRITE`
statement. Also, with the change of this PR, we always invalidate our in memory
data cache after inserting into a BaseRelation.
cc @marmbrus @liancheng
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yhuai/spark insertFollowUp
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4373.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4373
----
commit 727f21ae8503e0ed010d43ddc528e06b31418716
Author: Yin Huai <[email protected]>
Date: 2015-02-04T23:18:45Z
Preinsert casting and renaming.
commit c18da349b1c514633ad44fc473b6a1fc4a4a0d8c
Author: Yin Huai <[email protected]>
Date: 2015-02-04T23:18:58Z
Invalidate cache after insert.
commit 1951fe1f55a5be6c0ce0c6debf0b5b8c2b17cd9c
Author: Yin Huai <[email protected]>
Date: 2015-02-04T23:19:38Z
Merge remote-tracking branch 'upstream/master'
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
