loserwang1024 commented on code in PR #4534:
URL: https://github.com/apache/flink-cdc/pull/4534#discussion_r4032875771
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-cdc-base/src/main/java/org/apache/flink/cdc/connectors/base/source/assigner/splitter/JdbcSourceChunkSplitter.java:
##########
@@ -102,35 +103,43 @@ public JdbcSourceChunkSplitter(
@Override
public void open() {
- this.jdbcConnection = dialect.openJdbcConnection(sourceConfig);
+ // The connection is acquired lazily and released once a table is
split, so an assigner
+ // that has nothing to split (e.g. restored after the snapshot phase)
holds no connection.
}
/** Generates all snapshot splits (chunks) for the give table path. */
@Override
public Collection<SnapshotSplit> generateSplits(TableId tableId) throws
Exception {
Review Comment:
What about using temp jdbc connection each time for generateSplits and
release immediately.
```suggestion
@Override
public Collection<SnapshotSplit> generateSplits(TableId tableId) throws
Exception {
try (JdbcConnection jdbcConnection =
dialect.openJdbcConnection(sourceConfig)){
return generateSplits(tableId, jdbcConnection);
}
}
private Collection<SnapshotSplit> generateSplits(TableId tableId,
JdbcConnection jdbcConnection) throws Exception {
if (!hasNextChunk()) {
// split a new table.
analyzeTable(tableId, jdbcConnection);
Optional<List<SnapshotSplit>> evenlySplitChunks =
trySplitAllEvenlySizedChunks(tableId);
if (evenlySplitChunks.isPresent()) {
return evenlySplitChunks.get();
} else {
synchronized (lock) {
this.currentSplittingTableId = tableId;
this.nextChunkStart =
ChunkSplitterState.ChunkBound.START_BOUND;
this.nextChunkId = 0;
return
Collections.singletonList(splitOneUnevenlySizedChunk(tableId, jdbcConnection));
}
}
} else {
Preconditions.checkState(
currentSplittingTableId.equals(tableId),
"Can not split a new table before the previous table
splitting finish.");
if (currentSplittingTable == null) {
analyzeTable(currentSplittingTableId, jdbcConnection);
}
synchronized (lock) {
return
Collections.singletonList(splitOneUnevenlySizedChunk(tableId, jdbcConnection));
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
