EricJoy2048 commented on code in PR #6160:
URL: https://github.com/apache/seatunnel/pull/6160#discussion_r1451221706
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/DynamicChunkSplitter.java:
##########
@@ -309,6 +329,71 @@ private List<ChunkRange> splitUnevenlySizedChunks(
return splits;
}
+ /**
+ * split by date type column
+ *
+ * @param table
+ * @param splitColumnName
+ * @param min
+ * @param max
+ * @param chunkSize
+ * @return
+ * @throws SQLException
+ */
+ private List<ChunkRange> dateColumnSplitChunks(
+ JdbcSourceTable table, String splitColumnName, Object min, Object
max, int chunkSize)
+ throws SQLException {
+ log.info("Use date chunks for table {}", table.getTablePath());
+ final List<ChunkRange> splits = new ArrayList<>();
+ Date sqlDateMin = null;
+ Date sqlDateMax = null;
+ if (min instanceof Date) {
+ sqlDateMin = (Date) min;
+ sqlDateMax = (Date) max;
+ } else if (min instanceof Timestamp) {
+ sqlDateMin = new Date(((Timestamp) min).getTime());
+ sqlDateMax = new Date(((Timestamp) max).getTime());
+ }
+ List<LocalDate> dateRange =
+ getDateRange(sqlDateMin.toLocalDate(),
sqlDateMax.toLocalDate());
+ if (dateRange.size() > 20 * 365) {
+ // TODO: If dateRange granter than 20 year, need get the real date
in the table
Review Comment:
> Can the todo logic here be reflected in the document? This is also
convenient for user perception
In fact, each type of split will be calculated differently in different
situations. At present, we do not have a special document for this section. In
the future, we need a special introduction document for split algorithm, and we
need to discuss what directory it should be placed in through the mailing list.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
