Yizhou-Yang opened a new pull request, #7580:
URL: https://github.com/apache/inlong/pull/7580
### Prepare a Pull Request
- Fixes #7249
### Motivation
JDBC needs to archive dirty data. In this pr, I tried to explore to
possibility of accurate dirty data archive for jdbc by modifying the executors.
The modifications are well-tested and stable in most cases, and in other cases
it will give a warning "jdbc enhance failed for class:{}", and won't affect
normal code.
### Modification
Support dirty data accurate archive by using reflection to replace some
kinds of executors, and replace the tablesimplestatementexecutor with one that
adds metrics.
### To-do and to-improve
1) There are 8 total implementations of flink-cdc-executors.
The currently supported sink executor types are:
TableBufferedStatementExecutor
TableBufferReducedStatementExecutor
TableSimpleStatementExecutor
unsupported executor types: (will not archive dirty data, but will not break
code)
KeyedBatchStatementExecutor
NoOPStatementExecutor
TableInsertOrUpdateStatementExecutor
no need to support:
InsertOrUpdateJdbcExecutor (TableInsertOrUpdateStatementExecutor is enough)
SimpleBatchStatementExecutor (TableSimpleStatementExecutor is enough)
2) This pr uses Java reflection depending on flink-cdc-connectors. This
design can be improved by introducing the executors into Inlong-jdbc-connector,
and modifying the builder to directly change the executor class, instead of
doing all this reflection. Also, throughput can increase by refactoring too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
