CalvinKirs commented on code in PR #68135:
URL: https://github.com/apache/doris/pull/68135#discussion_r4044944466
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonCatalogOps.java:
##########
@@ -422,13 +423,49 @@ public boolean branchExists(Table table, String
branchName) {
@Override
public long rowCount(Table table) {
- // Legacy PaimonExternalTable.fetchRowCount /
PaimonSysExternalTable.fetchRowCount: sum
- // the planned-split record counts.
- long rowCount = 0;
- for (Split split :
table.newReadBuilder().newScan().plan().splits()) {
- rowCount += split.rowCount();
+ // System/format tables have no data snapshot count. A fallback
pair combines two
+ // branches, so its main snapshot alone cannot estimate the
relation either.
+ if (!(table instanceof FileStoreTable)
+ ||
PaimonTableDecorators.unwrapToFallbackOrBase((FileStoreTable) table)
+ instanceof FallbackReadFileStoreTable) {
+ return -1;
}
- return rowCount;
+ FileStoreTable fileStoreTable = (FileStoreTable) table;
+ CoreOptions options = fileStoreTable.coreOptions();
+ // Batch scans can exclude level-0 files or postponed buckets even
in full-snapshot mode.
+ // The snapshot counter includes those files; do not enumerate
manifests to correct it.
+ if ((!fileStoreTable.primaryKeys().isEmpty() &&
options.batchScanSkipLevel0()
+ &&
options.toConfiguration().get(CoreOptions.BATCH_SCAN_MODE) ==
CoreOptions.BatchScanMode.NONE)
+ || options.bucket() == BucketMode.POSTPONE_BUCKET) {
Review Comment:
Please be extra careful here. This condition is a hand copy of the built-in
filters in Paimon 1.3.1's `DataTableBatchScan` constructor. If a future
`paimon.version` bump adds another filter there, this will silently return a
wrong count, and no test will catch it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
