vinothchandar commented on pull request #3285: URL: https://github.com/apache/hudi/pull/3285#issuecomment-885764276
@codope please help me close out the schema evolution story here. my point was: when mixing old files where _hoodie_operation is NOT present with new files where it is present, the queries can break? @nsivabalan's response > yes. Check row 4 here https://hudi.apache.org/docs/writing_data.html#schema-evolution > Here is what could be happening. Your new batch of write (with new operation field), might be routed to all existing file groups and hence it might have worked. Can you try sending to only partial file groups. > If you are doing some sanity test, you can do this. > batch1: old schema: route to partition_0 > batch2: new schema w/ operation field: route to partition_1 > unless operation field is appended to the end of the existing schema, this may not work. > Let us know what do you see. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
