This is an automated email from the ASF dual-hosted git repository.
comphead pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new 8756197cf2 added explaination for Schema and DFSchema to documentation
(#15329)
8756197cf2 is described below
commit 8756197cf2f1e5994710700129aaa547a6276815
Author: Jiashu Hu <[email protected]>
AuthorDate: Fri Mar 21 14:06:04 2025 -0500
added explaination for Schema and DFSchema to documentation (#15329)
* added explaination for Schema and DFSchema to documentation
* change code block to quote block since CICL request a full code but we
are only introducing syntax
* improved language
* Update docs/source/library-user-guide/working-with-exprs.md
---------
Co-authored-by: Oleks V <[email protected]>
---
docs/source/library-user-guide/working-with-exprs.md | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/docs/source/library-user-guide/working-with-exprs.md
b/docs/source/library-user-guide/working-with-exprs.md
index 1a6e912308..df4e5e3940 100644
--- a/docs/source/library-user-guide/working-with-exprs.md
+++ b/docs/source/library-user-guide/working-with-exprs.md
@@ -50,6 +50,25 @@ As another example, the SQL expression `a + b * c` would be
represented as an `E
As the writer of a library, you can use `Expr`s to represent computations that
you want to perform. This guide will walk you through how to make your own
scalar UDF as an `Expr` and how to rewrite `Expr`s to inline the simple UDF.
+## Arrow Schema and DataFusion DFSchema
+
+Apache Arrow `Schema` provides a lightweight structure for defining data, and
Apache Datafusion`DFSchema` extends it with extra information such as column
qualifiers and functional dependencies. Column qualifiers are multi part path
to the table e.g table, schema, catalog. Functional Dependency is the
relationship between attributes(characteristics) of a table related to each
other.
+
+### Difference between Schema and DFSchema
+
+- Schema: A fundamental component of Apache Arrow, `Schema` defines a
dataset's structure, specifying column names and their data types.
+
+ > Please see [Struct
Schema](https://docs.rs/arrow-schema/54.2.1/arrow_schema/struct.Schema.html)
for a detailed document of Arrow Schema.
+
+- DFSchema: Extending `Schema`, `DFSchema` incorporates qualifiers such as
table names, enabling it to carry additional context when required. This is
particularly valuable for managing queries across multiple tables.
+ > Please see [Struct
DFSchema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html)
for a detailed document of DFSchema.
+
+### How to convert between Schema and DFSchema
+
+From Schema to DFSchema: Use `DFSchema::try_from_qualified_schema` with a
table name and original schema, for detailed code example please see
[creating-qualified-schemas](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#creating-qualified-schemas).
+
+From DFSchema to Schema: Since the `Into` trait has been implemented for
DFSchema to convert it into an Arrow Schema, for detailed code example please
see
[converting-back-to-arrow-schema](https://docs.rs/datafusion/latest/datafusion/common/struct.DFSchema.html#converting-back-to-arrow-schema).
+
## Creating and Evaluating `Expr`s
Please see
[expr_api.rs](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/expr_api.rs)
for well commented code for creating, evaluating, simplifying, and analyzing
`Expr`s.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
