alamb commented on code in PR #11994:
URL: https://github.com/apache/datafusion/pull/11994#discussion_r1717433490
##########
datafusion/core/src/datasource/physical_plan/parquet/mod.rs:
##########
@@ -144,6 +142,29 @@ pub use writer::plan_to_parquet;
/// * User provided [`ParquetAccessPlan`]s to skip row groups and/or pages
/// based on external information. See "Implementing External Indexes" below
///
+/// # Predicate Pushdown
Review Comment:
I tried to consolidate the description of what predicate pushdown is done in
the ParquetExec
##########
datafusion/core/src/datasource/physical_plan/parquet/row_filter.rs:
##########
@@ -312,8 +369,10 @@ fn size_of_columns(
Ok(total_size)
}
-/// For a given set of `Column`s required for predicate `Expr` determine
whether all
-/// columns are sorted. Sorted columns may be queried more efficiently in the
presence of
+/// For a given set of `Column`s required for predicate `Expr` determine
whether
+/// all columns are sorted.
+///
+/// Sorted columns may be queried more efficiently in the presence of
/// a PageIndex.
fn columns_sorted(
Review Comment:
This is interesting that we never connected up the "columns_sorted"
information -- is this on your list @thinkharderdev ?
Should I file a ticket to do this?
##########
datafusion/core/src/datasource/physical_plan/parquet/row_filter.rs:
##########
@@ -15,6 +15,49 @@
// specific language governing permissions and limitations
// under the License.
+//! Utilities to push down of DataFusion filter predicates (any DataFusion
Review Comment:
this is mostly the same content, reformatted and made more concise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
