shaohui hong created FLINK-36775: ------------------------------------ Summary: fetchSize set in OracleIncrementalSource is not working Key: FLINK-36775 URL: https://issues.apache.org/jira/browse/FLINK-36775 Project: Flink Issue Type: Bug Components: Flink CDC Affects Versions: 3.0.0 Reporter: shaohui hong Attachments: OracleScanFetchTask.java
FetchSize is an import parameter which affects the speed of selecting data from database using jdbc, because it tells jdbc the max size of records to get for each query. A bigger fetchSize leads to a smaller query times, whichi can siganifcantly reduce the time of getting all snapshot data. OracleIncrementalSource.java uses OracleScnFetchTask.java to do all the snapshot work by jdbc, and has a config named fetchSize. This config is used in jdbc as described as above, but it does not actually take affect. At 332nd line of OracleScnFetchTask.java, the function named readTableSplitDataStatement formats the selectStatement which is used to execute query, and then set fetchSize of this selectStatement. The value of fetchSize comes from OracleConnectorConfig.java the parent class of which is CommonConnectorConfig.java where a field named query.fetch.size exists. This is the config place of fetchSize that is actually used. In order to make fetchSize parameter set in OracleIncrementalSource take affect, it just needs to add one line of code in OracleSourceConfigFactory.java: props.setProperty(CommonConnectorConfig.QUERY_FETCH_SIZE.name(), String.valueOf(fetchSize)) -- This message was sent by Atlassian Jira (v8.20.10#820010)