emanuelebardelli commented on a change in pull request #24780:
[SPARK-26388][SQL]_implement_hive_ddl_alter_table_replace_columns
URL: https://github.com/apache/spark/pull/24780#discussion_r290703044
##########
File path:
sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4
##########
@@ -115,6 +115,8 @@ statement
(identifier | FOR COLUMNS identifierSeq | FOR ALL COLUMNS)? #analyze
| ALTER TABLE tableIdentifier
ADD COLUMNS '(' columns=colTypeList ')'
#addTableColumns
+ | ALTER TABLE tableIdentifier
+ REPLACE COLUMNS '(' columns=colTypeList ')'
#replaceTableColumns
Review comment:
@gatorsmile the idea is actually to implement `alter table replace colu mns`
fully:
- drop column if not preset in the new schema
- add column, if not preset in the table’s schema
- keep column if colum_name and colum_type match between schemas
- raise exception if trying to replace an existing column with a different
data type
I believe this is a much needed feature to be able to manage meta-stores
fully from Spark; however I understand the complexity and risks of this
operation, so please let’s take the time to ensure it’s the right thing to do
and all possibile ramifications.
I tried to make the commit history as clean and as descriptive as possibile
and just added some more details in the PR description `known issues` section.
I've also added an extra validation step to prevent columns to be replaced
if the data type doesn't match (8b6da23)
Let me know if anything isn’t clear and I'll be happy add more details.
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
