prashantwason commented on issue #1845: URL: https://github.com/apache/hudi/issues/1845#issuecomment-663169546
The issue is probably related to using the wrong schema for Upsert. Let me expand your original steps to reproduce and you can correct me: Steps to reproduce: 1. We have some old schema (SCHEMA_V1) and events. 2. We update the schema with a new (SCHEMA_V2), optional union field and restart the DeltaStreamer 3. We ingest new events (Events with SCHEMA_V2) 4. We ingest old events again (there are some upserts). ?????? **What schema is being used here?** If in step 4 you are using SCHEMA_V1 then it wont be able to read in records ingested at step 3 as that is a newer schema. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
