bvaradar commented on issue #2193: URL: https://github.com/apache/hudi/issues/2193#issuecomment-716713865
@spyzzz : Sorry for the delay. I guess the commit metadata that you pasted was part of the initial bootstrap of the rows sent by debezium. You have 2 kinds of workloads - 1. Snapshot of the mysql table where there won't be duplicates but this will be throughput intensive. 2. CDC of your mysql table containing new inserts/updates and deletes. Those are very different workloads with (1) happening only once (or whenever you need to re-bootstrap) and then (2) being the steady state. They need different different configurations for optimal performance Insert vs upsert. In the steady state (after initial snapshot), Are you able to see steady performance. Your last comment seems to indicate that the write time is holding up and not increasing. Is this correct ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
