paleolimbot commented on issue #379: URL: https://github.com/apache/sedona-db/issues/379#issuecomment-3591008486
Thank you! Not updating the ordinal of a column reference (or I suppose assigning a bogus one) also seems to be the issue for the pruning. > I wonder is there any assumptions that ProjectionExec is never applied after SortExec in physical plan Maybe there's a logical plan optimizer rule that makes it unlikely? I dug up the execution plan wrapper I had in a much earlier version that might be a workaround: https://github.com/apache/sedona-db/blob/8c0c7e3b0596ac38bde45921acf9b5d9d134fde7/rust/sedona-geoparquet/src/wrap.rs ...and I managed to make a test that panics in the same way: ```rust #[tokio::test] async fn geoparquet_1_1_with_sort_by_expr() { let example = test_geoparquet("ns-water", "water-point"); // Requires submodules/download-assets.py which not all contributors need let example = match example { Ok(path) => path, Err(err) => { println!("ns-water/water-point is not available: {err}"); return; } }; let ctx = setup_context(); let fns = sedona_functions::register::default_function_set(); let geometry_udf: ScalarUDF = fns.scalar_udf("sd_format").unwrap().clone().into(); let bbox_udf: ScalarUDF = geoparquet_bbox_udf().into(); let df = ctx .table(&example) .await .unwrap() .sort_by(vec![geometry_udf.call(vec![col("geometry")])]) .unwrap() .select(vec![ Expr::Cast(Cast::new( geometry_udf.call(vec![col("geometry")]).alias("txt").into(), DataType::Utf8View, )), col("geometry"), ]) .unwrap(); let mut options = TableGeoParquetOptions::new(); options.geoparquet_version = GeoParquetVersion::V1_1; let df_batches_with_bbox = df .clone() .select(vec![ col("txt"), bbox_udf.call(vec![col("geometry")]).alias("bbox"), col("geometry"), ]) .unwrap() .collect() .await .unwrap(); test_write_dataframe(ctx, df, df_batches_with_bbox, options, vec![]) .await .unwrap(); } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
