github-actions[bot] commented on code in PR #65446:
URL: https://github.com/apache/doris/pull/65446#discussion_r3703146697
##########
be/src/core/data_type_serde/data_type_serde.cpp:
##########
@@ -477,6 +477,38 @@ int64_t find_struct_child_index(const ::orc::Type& type,
const std::string& fiel
return -1;
}
+struct RoundedOrcTimestamp {
+ int64_t seconds;
+ uint64_t microseconds;
+};
+
+Status round_orc_timestamp_to_microseconds(int64_t seconds, int64_t
nanoseconds,
+ RoundedOrcTimestamp* result) {
+ constexpr int64_t NANOS_PER_SECOND = 1000000000;
+ constexpr int64_t NANOS_PER_MICROSECOND = 1000;
+ constexpr int64_t MICROS_PER_SECOND = 1000000;
+ constexpr int64_t MIN_DORIS_TIMESTAMP_MICROS = -62135596800000000LL;
+ constexpr int64_t MAX_DORIS_TIMESTAMP_MICROS = 253402300799999999LL;
+ DORIS_CHECK(result != nullptr);
+ DORIS_CHECK(nanoseconds >= 0 && nanoseconds < NANOS_PER_SECOND);
+ // Doris stores six fractional digits, so use half-up rounding and carry
999999500ns into the
+ // next second instead of silently truncating the ORC value.
+ const auto rounded_microseconds =
+ (nanoseconds + NANOS_PER_MICROSECOND / 2) / NANOS_PER_MICROSECOND;
+ // Validate in widened epoch-microsecond arithmetic so the rounding carry
cannot wrap an
+ // invalid external value into a plausible Doris timestamp.
+ const __int128 epoch_microseconds =
+ static_cast<__int128>(seconds) * MICROS_PER_SECOND +
rounded_microseconds;
+ if (epoch_microseconds < MIN_DORIS_TIMESTAMP_MICROS ||
+ epoch_microseconds > MAX_DORIS_TIMESTAMP_MICROS) {
Review Comment:
[P2] Validate DATETIMEV2 after timezone conversion
This precheck uses the UTC 0001..9999 interval before the caller converts
the instant to the session timezone. For example, epoch second 253402329599
with 999999000ns is outside the UTC maximum, but in a -08:00 session it is the
valid local value 9999-12-31 23:59:59.999999; this helper now returns
DataQualityError before the later target-local is_valid_date() check. The lower
boundary is symmetric in +14:00, and the new boundary tests use UTC only.
Please retain the widened overflow/carry guard, but validate DATETIMEV2 against
the converted local value (while TIMESTAMPTZ can keep UTC-domain validation),
with nonzero-offset endpoint tests. This is distinct from the existing carry
thread: that case admitted an invalid year-10000 result, while this one rejects
a valid target-local endpoint.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/plugin/PluginDrivenExternalCatalog.java:
##########
@@ -218,6 +220,13 @@ protected Connector createConnectorFromProperties() {
public void checkProperties() throws DdlException {
super.checkProperties();
String catalogType = getType();
+ if ("hms".equalsIgnoreCase(catalogType) ||
"hudi".equalsIgnoreCase(catalogType)) {
+ String hiveParquetTimeZone = catalogProperty.getOrDefault(
Review Comment:
[P2] Apply this validation to ALTER CATALOG too
This parser runs during checkProperties(), but
PluginDrivenExternalCatalog.validatePropertiesBeforeUpdate() builds the
detached candidate and returns true after only the base checks plus
ConnectorFactory.validatePropertiesForUpdate(). The Hive provider does not
validate this key, so CatalogMgr skips the legacy checkProperties() branch and
an ALTER such as hive.parquet.time-zone=CST can be journaled successfully; the
next scan then throws when getConfiguredHiveParquetTimeZone() parses it. Please
reuse this validation for the candidate update path (ideally in the owning
provider so create and alter share it) and add an ALTER regression. This is
distinct from the existing custom-provider routing thread: it affects invalid
built-in HMS updates rather than dropping a valid custom-provider value.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
