parisni commented on issue #6342: URL: https://github.com/apache/hudi/issues/6342#issuecomment-1218406180
> reconcilation of schemas work only if missing columns are in the end @nsivabalan Do you mean https://hudi.apache.org/docs/configurations#hoodiedatasourcewritereconcileschema ? That's not the behavior I see: i'am able to write data with missing column at any position, until only one is missing. Also the doc don't mention such limitation nor its source code. @xiarixiaoyao as the Dev of the reconcile feature, do you have some insight ? On August 15, 2022 5:42:26 PM UTC, Sivabalan Narayanan ***@***.***> wrote: >reconcilation of schemas work only if missing columns are in the end. > >For eg: >Commit1: >Schema: col1, col2, col3, col4 > >Commit2: >Schema: col1, col2, col3, col4, col5, col6 > >Commit3: old writer. >Schema: col1, col2, col3, col4 >Commit3 will succeed when you enable reconcile schema config. > >In general, adding new colunmns at the end of existing schema works. it may break if you try to add new columns inbetween. for eg, in commit2, if your schema as col1, col5, col6, col2, col3, col4, your write or read may break. > > > > >-- >Reply to this email directly or view it on GitHub: >https://github.com/apache/hudi/issues/6342#issuecomment-1215479286 >You are receiving this because you authored the thread. > >Message ID: ***@***.***> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
