Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24540
to look at the new patch set (#3).
Change subject: POC: Port native C++ geospatial functions to WKB mode
......................................................................
POC: Port native C++ geospatial functions to WKB mode
Port of https://gerrit.cloudera.org/#/c/20602/ to
WKB_EXPERIMENTAL mode. Functions that previously ran as Java
UDFs in WKB mode now have native C++ implementations using
Boost.Geometry. Only 2D geometries are supported — 3D/4D
types (Z/M dimensions) are rejected at the WKB header
parsing level.
Architecture:
The C++ implementation is layered into five concerns:
- wkb-format.h: Low-level WKB binary parsing (header
validation, coordinate extraction, type mapping). Assumes
little-endian byte order throughout.
- wkb-serialization: Reads WKB bytes into Boost.Geometry
model types (point2d, linestring2d, polygon2d, etc.) and
writes them back. Also provides bounding-box computation
directly from raw WKB without full deserialization.
- geometry-wrapper-wkb: High-level wrapper that holds one
geometry instance (tagged union style via OGCType enum).
Provides FromWkb/FromWkt/FromCoordinates deserialization,
ToWkb/ToWkt serialization, and geometry property methods
(Area, Length, Distance, NumPoints, IsSimple, etc.) that
delegate to Boost.Geometry algorithms.
- relation-wrapper-wkb: Template-based spatial predicate
evaluator. Uses a policy pattern (ContainsPredicate,
IntersectsPredicate, etc.) with double dispatch over
LHS/RHS geometry types.
- geospatial-functions-wkb-ir.cc: Thin entry points
registered as Impala BUILTINs. These parse UDF arguments,
delegate to the wrapper/serialization layers above, and
return Impala UDF result types.
Supporting headers:
- wkt.h: WKT parsing via Boost.Geometry read_wkt(), with
polygon correction.
- utils.h: Binning logic (ported from ESRI BinUtils.java),
ring orientation checks, and polygon assembly from rings.
- formatted-double.h: Custom double type for controlled WKT
output formatting.
- common.h: Boost.Geometry type aliases and OGC type
constants.
On the frontend, HiveEsriGeospatialBuiltins is reorganized
so that in WKB mode, functions with C++ natives are no longer
registered as Java UDFs. A new addWkbNatives() method
registers them as BUILTINs with GEOMETRY type signatures.
Tests for 3D/4D functions (ST_Z, ST_M, ST_Is3D, etc.) are
moved to a separate HIVE_ESRI-only test file since they are
not applicable to the WKB native path.
Change-Id: I0047acebd4aaa774bf6bb8ec0618515d38ad6aae
---
M be/CMakeLists.txt
M be/src/codegen/impala-ir.cc
M be/src/exprs/geo/CMakeLists.txt
M be/src/exprs/geo/common.h
A be/src/exprs/geo/geometry-wrapper-wkb.cc
A be/src/exprs/geo/geometry-wrapper-wkb.h
A be/src/exprs/geo/geospatial-functions-wkb-ir.cc
M be/src/exprs/geo/geospatial-functions.h
A be/src/exprs/geo/relation-wrapper-wkb-test.cc
A be/src/exprs/geo/relation-wrapper-wkb.cc
A be/src/exprs/geo/relation-wrapper-wkb.h
A be/src/exprs/geo/utils.h
A be/src/exprs/geo/wkb-format.h
A be/src/exprs/geo/wkb-serialization-test.cc
A be/src/exprs/geo/wkb-serialization.cc
A be/src/exprs/geo/wkb-serialization.h
A be/src/exprs/geo/wkt.h
M be/src/exprs/scalar-expr-evaluator.cc
M
fe/src/compat-hive-3/java/org/apache/impala/compat/HiveEsriGeospatialBuiltins.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
A
testdata/workloads/functional-query/queries/QueryTest/geospatial-esri-high-dimension.test
M testdata/workloads/functional-query/queries/QueryTest/geospatial-esri.test
A
testdata/workloads/functional-query/queries/QueryTest/geospatial-relations-table.test
A
testdata/workloads/functional-query/queries/QueryTest/geospatial-wkb-serialization.test
M tests/custom_cluster/test_geospatial_library.py
M tests/query_test/test_geospatial_functions.py
27 files changed, 4,700 insertions(+), 372 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/40/24540/3
--
To view, visit http://gerrit.cloudera.org:8080/24540
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0047acebd4aaa774bf6bb8ec0618515d38ad6aae
Gerrit-Change-Number: 24540
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
