notfilippo commented on code in PR #13343:
URL: https://github.com/apache/datafusion/pull/13343#discussion_r1835688264
##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -293,7 +293,9 @@ impl SessionState {
.resolve(&catalog.default_catalog, &catalog.default_schema)
}
- pub(crate) fn schema_for_ref(
+ /// Retrieve the [`SchemaProvider`] for a specific [`TableReference`], if
it
+ /// esists.
+ pub fn schema_for_ref(
Review Comment:
Not sure if there is a way in which we could keep this method as
`pub(crate)`, but I don't see many reasons to make this method not public.
##########
datafusion/substrait/src/logical_plan/consumer.rs:
##########
@@ -841,16 +836,35 @@ pub async fn from_substrait_rel(
left.cross_join(right.build()?)?.build()
}
Some(RelType::Read(read)) => {
- fn read_with_schema(
- df: DataFrame,
+ async fn read_with_schema(
+ ctx: &SessionState,
+ table_ref: TableReference,
schema: DFSchema,
projection: &Option<MaskExpression>,
) -> Result<LogicalPlan> {
- ensure_schema_compatability(df.schema().to_owned(),
schema.clone())?;
+ let schema = schema.replace_qualifier(table_ref.clone());
+
+ let plan = {
+ let table = table_ref.table().to_string();
+ let schema = ctx.schema_for_ref(table_ref.clone())?;
+ let provider = match schema.table(&table).await? {
+ Some(ref provider) => Arc::clone(provider),
+ _ => return plan_err!("No table named '{table}'"),
+ };
+
+ LogicalPlanBuilder::scan(
+ table_ref,
+ provider_as_source(Arc::clone(&provider)),
+ None,
+ )?
Review Comment:
Most of this logic is flattened from the SessionContext::table method. Not
sure if there is any value to provide an abstraction directly in the
SessionState object.
##########
datafusion/substrait/src/logical_plan/consumer.rs:
##########
@@ -203,7 +201,7 @@ fn split_eq_and_noneq_join_predicate_with_nulls_equality(
async fn union_rels(
rels: &[Rel],
- ctx: &SessionContext,
+ ctx: &SessionState,
Review Comment:
In the original issue I also mention that we could declare a trait with just
the method used by the substrait producer / consumer so users could provide
their own shim.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
