github-actions[bot] commented on code in PR #66244:
URL: https://github.com/apache/doris/pull/66244#discussion_r3679520734
##########
fe/be-java-extensions/paimon-scanner/src/main/java/org/apache/doris/paimon/PaimonColumnValue.java:
##########
@@ -38,20 +38,47 @@
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.ZoneId;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
public class PaimonColumnValue implements ColumnValue {
private static final Logger LOG =
LoggerFactory.getLogger(PaimonColumnValue.class);
+ private static final Map<String, String> DORIS_TIME_ZONE_ALIASES;
+
+ static {
+ Map<String, String> aliases = new HashMap<>(ZoneId.SHORT_IDS);
+ // The scanner cannot depend on FE's TimeUtils, so keep its accepted
aliases and CST
+ // interpretation identical at this JNI boundary.
+ aliases.put("CST", "Asia/Shanghai");
+ aliases.put("PRC", "Asia/Shanghai");
+ aliases.put("UTC", "UTC");
+ aliases.put("GMT", "UTC");
+ DORIS_TIME_ZONE_ALIASES = Collections.unmodifiableMap(aliases);
+ }
+
private int idx;
private DataGetters record;
private ColumnType dorisType;
private DataType dataType;
- private String timeZone;
+ private ZoneId timeZone;
+ // Keep these caches lazy so scalar columns do not pay for complex-type
reuse bookkeeping.
+ private List<PaimonColumnValue> arrayValues;
+ private List<PaimonColumnValue> mapKeys;
Review Comment:
These caches belong to the scanner-wide root value, which `PaimonJniScanner`
reassigns to every projected field. For stable `a ARRAY<INT>` of size N
followed by empty `b ARRAY<INT>`, `b` trims this same `arrayValues` to zero, so
the next row recreates all N wrappers for `a`; MAP has the same issue. A later
null scalar is worse: `isNull()` clears all four cache families. Thus ordinary
multi-column scans can still allocate O(rows * elements) wrappers despite
stable shapes. Please scope the root cache graph per projected column (while
keeping recursive pruning within each graph) and add mixed-column tests
covering different collection sizes and a later null scalar.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
