raghav-reglobe commented on code in PR #67361:
URL: https://github.com/apache/doris/pull/67361#discussion_r4446354189
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/mvcc/PluginDrivenMvccExternalTable.java:
##########
@@ -324,25 +327,47 @@ private void listPartitions(ConnectorMetadata metadata,
ConnectorSession session
toListPartitionItem(partitionName, types,
part.getOrderedPartitionValues(),
part.getPartitionValueNullFlags()));
} catch (Exception e) {
- LOG.warn("toListPartitionItem failed, partitionColumns: {},
partitionName: {}",
- partitionColumns, partitionName, e);
+ // Counted here and reported once below. Both skip shapes are
per-TABLE conditions that
+ // repeat on EVERY snapshot load, i.e. on every query planned
against the table: a spec
+ // whose values are not representable in the partition
columns' types (an iceberg spec
+ // mixing an identity field with a month()/bucket() transform,
say - the connector
+ // supplies the transform result, keyed under the source
column) fails for every
+ // partition, every time. A WARN with a stack trace per
partition is then thousands of
+ // log lines per query, enough to fill an FE log volume; the
per-partition detail stays
+ // available at DEBUG.
+ skipped++;
+ if (firstSkippedCause == null) {
+ firstSkippedName = partitionName;
+ firstSkippedCause = e;
+ }
Review Comment:
Addressed in 717e3a50ac2 (second commit): `PartitionKey.getDateTimeLiteral`
no longer logs a WARN with a stack per failed value. It throws the
`AnalysisException` with the parser's message and cause attached, so the
listing's one summary line names the reason in its `cause:` field, and keeps
the per-value detail at DEBUG. No other per-value parser on the listing path
logs. The branch is also rebased onto current master, where the listing now
lives in `buildListPartitionView` (#69299); the summary fires once per
partition-view cache miss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
