shaohui hong created FLINK-36775:
------------------------------------

             Summary: fetchSize set in OracleIncrementalSource is not working
                 Key: FLINK-36775
                 URL: https://issues.apache.org/jira/browse/FLINK-36775
             Project: Flink
          Issue Type: Bug
          Components: Flink CDC
    Affects Versions: 3.0.0
            Reporter: shaohui hong
         Attachments: OracleScanFetchTask.java

FetchSize is an import parameter which affects the speed of selecting data from 
database using jdbc, because it tells jdbc the max size of records to get for 
each query. A bigger fetchSize leads to a smaller query times, whichi can 
siganifcantly reduce the time of getting all snapshot data.

OracleIncrementalSource.java uses OracleScnFetchTask.java to do all the 
snapshot work by jdbc, and has a config named fetchSize. This config is used in 
jdbc as described as above, but it does not actually take affect. At 332nd line 
of OracleScnFetchTask.java, the function named readTableSplitDataStatement 
formats the selectStatement which is used to execute query, and then set 
fetchSize of this selectStatement. The value of fetchSize comes from 
OracleConnectorConfig.java the parent class of which is 
CommonConnectorConfig.java where a field named query.fetch.size exists. This is 
the config place of fetchSize that is actually used.

In order to make fetchSize parameter set in OracleIncrementalSource take 
affect, it just needs to add one line of code in OracleSourceConfigFactory.java:

props.setProperty(CommonConnectorConfig.QUERY_FETCH_SIZE.name(), 
String.valueOf(fetchSize))



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to