github-actions[bot] commented on code in PR #69032:
URL: https://github.com/apache/doris/pull/69032#discussion_r4368059405
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/MinMap.java:
##########
@@ -53,6 +53,7 @@ protected DataType returnValueType(DataType valueType) {
@Override
public void checkLegalityBeforeTypeCoercion() {
+ super.checkLegalityBeforeTypeCoercion();
Review Comment:
[P2] Reject spatial values in MIN_MAP and MAX_MAP
The inherited check rejects spatial MAP keys and DISTINCT arguments, but
`checkMinMaxValueType()` still accepts spatial values. For the reduced plan
`Aggregate[min_map(map(1, geom))] -> IcebergScan[geom]`, BE's map combinator
creates nested `min`, whose new spatial factory uses `SingleValueDataColumn`
and `ColumnSpatial::compare_at` on raw WKB. With POINT(1 2) and POINT(3 4) both
little-endian, the selected minimum is POINT(3 4); encoding only POINT(1 2) in
big-endian changes the minimum to POINT(1 2), without changing either geometry.
MAX_MAP and the state paths use the same ordering. This is a separate
nondistinct, per-key consumer from the earlier direct MIN/MAX and DISTINCT-MAP
threads. Reject recursively spatial values in the shared MIN_MAP/MAX_MAP value
check, including state/rewrite paths, and replace the positive spatial-extrema
expectations with rejection tests while retaining MAP_AGG/COUNT_MAP spatial
values.
##########
be/src/format/parquet/schema_desc.cpp:
##########
@@ -268,7 +269,22 @@ std::pair<DataTypePtr, bool>
FieldDescriptor::convert_to_doris_type(
tparquet::LogicalType logicalType, bool nullable) {
std::pair<DataTypePtr, bool> ans = {std::make_shared<DataTypeNothing>(),
false};
bool& is_type_compatibility = ans.second;
- if (logicalType.__isset.STRING) {
+ if (logicalType.__isset.GEOMETRY) {
+ const std::string crs =
+ logicalType.GEOMETRY.__isset.crs ? logicalType.GEOMETRY.crs :
"OGC:CRS84";
+ // OPTIONAL spatial fields must retain their definition-level
threshold for COUNT.
+ auto spatial_type = std::make_shared<DataTypeSpatial>(TYPE_GEOMETRY,
crs);
Review Comment:
[P2] Preserve Hive BINARY reads of spatial-annotated Parquet
This file-global annotation handling also affects an existing Hive external
table declared `raw(shape BINARY) STORED AS PARQUET` over a valid
GEOMETRY/GEOGRAPHY-annotated BYTE_ARRAY file. HMS still maps `shape` to
VARBINARY. Previously the unrecognized annotation fell back to physical binary,
so `SELECT hex(shape) FROM raw` returned the original bytes; it now infers
DataTypeSpatial. V1 then rejects the missing Spatial-to-VARBINARY logical
conversion, while the parallel native-schema branch makes v2 build an
EvolutionCast that fails `create_varbinary_wrapper`'s String-only source check.
This is an existing catalog BINARY read, separate from the deliberate
restriction on native spatial inference through file TVFs. Preserve
table-driven opaque binary reads in both scanners while retaining Iceberg
spatial semantics, and cover both annotations plus nullable/nested binary
leaves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
