924060929 commented on code in PR #68934:
URL: https://github.com/apache/doris/pull/68934#discussion_r4256383241
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergConnectorTransaction.java:
##########
@@ -929,14 +930,19 @@ private Expression buildPartitionFilter(Map<String,
String> staticPartitions, Pa
* both paths that build this predicate: DELETE/UPDATE/MERGE conflict
detection and
* {@code INSERT OVERWRITE ... PARTITION(d='nan')}.
*/
- private static Expression identityPartitionPredicate(String sourceColName,
Object value) {
+ private static Expression identityPartitionPredicate(Types.NestedField
sourceField, Object value) {
+ String sourceColName = sourceField.name();
if (value == null) {
return Expressions.isNull(sourceColName);
}
if ((value instanceof Double || value instanceof Float)
&& Double.isNaN(((Number) value).doubleValue())) {
return Expressions.isNaN(sourceColName);
}
+ if (sourceField.type().typeId() == TypeID.TIMESTAMP_NANO) {
+ // Partition values are already nanos; a raw Long would be rebound
as microseconds.
+ return Expressions.equal(sourceColName, Expressions.nanos((Long)
value));
Review Comment:
Rechecked at `19ef150201f4c4928c97b1efbe6003db6b83caf3`: this FE issue is
addressed. Scan, static overwrite and generated conflict predicates now share
`unsafeNanosFields`. Static overwrite rejects an unsafe non-null partition
predicate without widening deletion; conflict-only filtering conservatively
omits unsafe conjuncts. The added retained-DATE-file transaction test checks
that rejection leaves the snapshot unchanged. This is safe rejection of the
unsupported overwrite, rather than support for completing it. Source and test
assertions inspected; I did not independently run the tests.
##########
fe/fe-connector/fe-connector-iceberg/src/main/java/org/apache/doris/connector/iceberg/IcebergPartitionUtils.java:
##########
@@ -801,6 +828,11 @@ static boolean isValidRelatedTable(Table table) {
return false;
}
PartitionField partitionField = fields.get(0);
+ if (spec.schema().findType(partitionField.sourceId()).typeId() ==
Type.TypeID.TIMESTAMP_NANO) {
Review Comment:
Rechecked at `19ef150201f4c4928c97b1efbe6003db6b83caf3`: this FE issue is
addressed. A missing historical source type now makes `isValidRelatedTable`
return false, preserving the UNPARTITIONED view used by query snapshot
planning. The new test exercises an actual v1 remove-partition/drop-source
lifecycle. The null check handles the concrete reachable metadata state
identified here. Source and test assertions inspected; I did not independently
run the tests.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -156,6 +156,10 @@ private static ConnectorType toDecimalType(DecimalType
type) {
private static ConnectorType toTimestampType(DataType dataType) {
int scale = DEFAULT_TIMESTAMP_SCALE;
if (dataType instanceof TimestampType) {
+ // Keep all source fractional digits instead of truncating
high-precision wall-clock values.
+ if (((TimestampType) dataType).getPrecision() >
MAX_TIMESTAMP_SCALE) {
+ return ConnectorType.of("TIMESTAMP_NS");
Review Comment:
Rechecked at `19ef150201f4c4928c97b1efbe6003db6b83caf3`: the mapping and
out-of-range failure remain unchanged. I understand the stated decision to
leave range compatibility outside this PR; this thread should therefore remain
a known limitation rather than be marked fixed. Please explicitly disclose in
the PR description/release note and type-mapping documentation that existing
Paimon TIMESTAMP(7/8/9) columns now use the signed Int64 nanosecond range
(approximately 1677–2262), and previously readable values outside that range
fail queries. I found no new distinct FE issue in the latest delta. Static
review only; I did not independently run tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
