2010YOUY01 commented on code in PR #173:
URL: https://github.com/apache/sedona-db/pull/173#discussion_r2397882600
##########
rust/sedona-geoparquet/src/format.rs:
##########
@@ -364,15 +364,21 @@ impl GeoParquetFileSource {
predicate: Option<Arc<dyn PhysicalExpr>>,
) -> Result<Self> {
if let Some(parquet_source) =
inner.as_any().downcast_ref::<ParquetSource>() {
- let mut parquet_source = parquet_source.clone();
+ let parquet_source = parquet_source.clone();
// Extract the predicate from the existing source if it exists so
we can keep a copy of it
let new_predicate = match (parquet_source.predicate().cloned(),
predicate) {
(None, None) => None,
(None, Some(specified_predicate)) => Some(specified_predicate),
(Some(inner_predicate), None) => Some(inner_predicate),
- (Some(_), Some(specified_predicate)) => {
- parquet_source =
parquet_source.with_predicate(specified_predicate.clone());
Review Comment:
DataFusion's `with_predicate()` API will reset metrics unexpectedly. See
https://github.com/apache/datafusion/pull/17858
I checked the related implementation, the predicate inside inner
`ParquetSource` and the predicate in `GeoParquetFileSource` should always be
the same, so here I made the implementation more defensive, to avoid the
datafusion bug that clear the metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
