xiarixiaoyao commented on pull request #3668:
URL: https://github.com/apache/hudi/pull/3668#issuecomment-930001505
@bvaradar @codope @leesf . could you pls help me review this pr again,
thanks
code changes
1) support mor(incremental/realtime/optimize) read/write
2) support cow (incremental/realtime) read/write
3) support spark3 DDL. include:
alter statement:
* ALTER TABLE table1 ALTER COLUMN a.b.c TYPE bigint
* ALTER TABLE table1 ALTER COLUMN a.b.c SET NOT NULL
* ALTER TABLE table1 ALTER COLUMN a.b.c DROP NOT NULL
* ALTER TABLE table1 ALTER COLUMN a.b.c COMMENT 'new comment'
* ALTER TABLE table1 ALTER COLUMN a.b.c FIRST
* ALTER TABLE table1 ALTER COLUMN a.b.c AFTER x
add statement:
* ALTER TABLE table1 ADD COLUMNS (col_name data_type [COMMENT
col_comment], ...);
rename:
* ALTER TABLE table1 RENAME COLUMN a.b.c TO x
drop:
* ALTER TABLE table1 DROP COLUMN a.b.c
* ALTER TABLE table1 DROP COLUMNS a.b.c, x, y
set/unset Properties:
* ALTER TABLE table SET TBLPROPERTIES ('table_property' =
'property_value');
* ALTER TABLE table UNSET TBLPROPERTIES [IF EXISTS] ('comment', 'key');
4) support spark2 DDL.
5) add FileBaseInternalSchemasManager to manger history schemas, and save
historySchemas in "./hoodie/.schema" . now we no need to save historySchemas
into commit file.
6) add segment lock to TableInternalSchemaUtils to support concurrent read
and write cache.
7) rename mergeSchemaAction to SchemaMerger; remove helper methods from
TableChanges to a helper class, now TableChanges is ok; use visitor mode to
produce nameToId for internalSchema; and other samll fixed.
Remaining problem: add more UT for this pr, add support for bootstrap table.
@bvaradar forgive me this change is too large, i still use squahsing.
Subsequent modifications will be in the form of add commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
