This is an automated email from the ASF dual-hosted git repository.
agrove pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/arrow-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 670ad05c86 DataFusion 12.0.0 user guide (#237)
670ad05c86 is described below
commit 670ad05c86e60874f1e796312634717a832186ac
Author: Andy Grove <[email protected]>
AuthorDate: Thu Sep 15 14:09:33 2022 -0600
DataFusion 12.0.0 user guide (#237)
---
.../contributor-guide/communication.md.txt | 3 +
datafusion/_sources/user-guide/cli.md.txt | 10 +-
datafusion/_sources/user-guide/dataframe.md.txt | 15 +-
datafusion/_sources/user-guide/expressions.md.txt | 10 +
datafusion/_sources/user-guide/library.md.txt | 17 +-
.../user-guide/sql/aggregate_functions.md.txt | 6 +
.../_sources/user-guide/sql/data_types.md.txt | 93 ++++++---
datafusion/_sources/user-guide/sql/ddl.md.txt | 36 ++--
.../user-guide/sql/information_schema.md.txt | 41 ++--
.../user-guide/sql/scalar_functions.md.txt | 5 +-
.../_sources/user-guide/sql/sql_status.md.txt | 26 +--
datafusion/contributor-guide/communication.html | 2 +
datafusion/objects.inv | Bin 4660 -> 4775 bytes
datafusion/searchindex.js | 2 +-
datafusion/user-guide/cli.html | 10 +-
datafusion/user-guide/dataframe.html | 7 +-
datafusion/user-guide/expressions.html | 32 ++++
datafusion/user-guide/library.html | 28 ++-
datafusion/user-guide/sql/aggregate_functions.html | 3 +
datafusion/user-guide/sql/data_types.html | 210 +++++++++++++++++----
datafusion/user-guide/sql/ddl.html | 45 +++--
datafusion/user-guide/sql/index.html | 15 +-
datafusion/user-guide/sql/information_schema.html | 61 +++---
datafusion/user-guide/sql/scalar_functions.html | 5 +-
datafusion/user-guide/sql/sql_status.html | 37 +---
25 files changed, 485 insertions(+), 234 deletions(-)
diff --git a/datafusion/_sources/contributor-guide/communication.md.txt
b/datafusion/_sources/contributor-guide/communication.md.txt
index 6c176610c4..cab4305f63 100644
--- a/datafusion/_sources/contributor-guide/communication.md.txt
+++ b/datafusion/_sources/contributor-guide/communication.md.txt
@@ -26,6 +26,9 @@ All participation in the Apache Arrow DataFusion project is
governed by the
Apache Software Foundation's [code of
conduct](https://www.apache.org/foundation/policies/conduct.html).
+The vast majority of communication occurs in the open on our
+[github repository](https://github.com/apache/arrow-datafusion).
+
## Questions?
### Mailing list
diff --git a/datafusion/_sources/user-guide/cli.md.txt
b/datafusion/_sources/user-guide/cli.md.txt
index 5e993774a9..4299990c09 100644
--- a/datafusion/_sources/user-guide/cli.md.txt
+++ b/datafusion/_sources/user-guide/cli.md.txt
@@ -33,7 +33,7 @@ $ echo "1,2" > data.csv
```bash
$ datafusion-cli
-DataFusion CLI v11.0.0
+DataFusion CLI v12.0.0
❯ CREATE EXTERNAL TABLE foo STORED AS CSV LOCATION 'data.csv';
0 rows in set. Query took 0.017 seconds.
❯ select * from foo;
@@ -57,13 +57,13 @@ DataFusion CLI can also be installed via Homebrew (on
MacOS). Install it as any
```bash
brew install datafusion
-# ==> Downloading https://ghcr.io/v2/homebrew/core/datafusion/manifests/5.0.0
+# ==> Downloading https://ghcr.io/v2/homebrew/core/datafusion/manifests/12.0.0
# ########################################################################
100.0%
# ==> Downloading
https://ghcr.io/v2/homebrew/core/datafusion/blobs/sha256:9ecc8a01be47ceb9a53b39976696afa87c0a8
# ==> Downloading from
https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:9ecc8a01be47ceb9a53b39976
# ########################################################################
100.0%
-# ==> Pouring datafusion--5.0.0.big_sur.bottle.tar.gz
-# 🍺 /usr/local/Cellar/datafusion/5.0.0: 9 files, 17.4MB
+# ==> Pouring datafusion--12.0.0.big_sur.bottle.tar.gz
+# 🍺 /usr/local/Cellar/datafusion/12.0.0: 9 files, 17.4MB
datafusion-cli
```
@@ -79,7 +79,7 @@ this to work.
```bash
git clone https://github.com/apache/arrow-datafusion
-git checkout 11.0.0
+git checkout 12.0.0
cd arrow-datafusion
docker build -f datafusion-cli/Dockerfile . --tag datafusion-cli
docker run -it -v $(your_data_location):/data datafusion-cli
diff --git a/datafusion/_sources/user-guide/dataframe.md.txt
b/datafusion/_sources/user-guide/dataframe.md.txt
index d21be0e426..e860242fc6 100644
--- a/datafusion/_sources/user-guide/dataframe.md.txt
+++ b/datafusion/_sources/user-guide/dataframe.md.txt
@@ -42,7 +42,7 @@ let ctx = SessionContext::new();
let df = ctx.read_csv("tests/example.csv", CsvReadOptions::new()).await?;
let df = df.filter(col("a").lt_eq(col("b")))?
.aggregate(vec![col("a")], vec![min(col("b"))])?
- .limit(None, Some(100))?;
+ .limit(0, Some(100))?;
// Print results
df.show();
```
@@ -96,9 +96,10 @@ These methods execute the logical plan represented by the
DataFrame and either c
## Other DataFrame Methods
-| Function | Notes
|
-| --------------- |
------------------------------------------------------------------------------------------------------------------------------------------------------------
|
-| explain | Return a DataFrame with the explanation of its plan so
far.
|
-| registry | Return a `FunctionRegistry` used to plan udf's calls.
|
-| schema | Returns the schema describing the output of this DataFrame
in terms of columns returned, where each column has a name, data type, and
nullability attribute. |
-| to_logical_plan | Return the logical plan represented by this DataFrame.
|
+| Function | Notes
|
+| ------------------- |
------------------------------------------------------------------------------------------------------------------------------------------------------------
|
+| explain | Return a DataFrame with the explanation of its plan so
far.
|
+| registry | Return a `FunctionRegistry` used to plan udf's calls.
|
+| schema | Returns the schema describing the output of this
DataFrame in terms of columns returned, where each column has a name, data
type, and nullability attribute. |
+| to_logical_plan | Return the optimized logical plan represented by this
DataFrame.
|
+| to_unoptimized_plan | Return the unoptimized logical plan represented by
this DataFrame.
|
diff --git a/datafusion/_sources/user-guide/expressions.md.txt
b/datafusion/_sources/user-guide/expressions.md.txt
index 79ca0e8daf..5dc3520d10 100644
--- a/datafusion/_sources/user-guide/expressions.md.txt
+++ b/datafusion/_sources/user-guide/expressions.md.txt
@@ -87,6 +87,16 @@ expressions such as `col("a") + col("b")` to be used.
| tan(x) | tangent |
| trunc(x) | truncate toward zero |
+## Bitwise Operators
+
+| Operator | Notes |
+| -------- | ----------------------------------------------- |
+| & | Bitwise AND => `(expr1 & expr2)` |
+| | | Bitwise OR => <code>(expr1 | expr2)</code> |
+| # | Bitwise XOR => `(expr1 # expr2)` |
+| << | Bitwise left shift => `(expr1 << expr2)` |
+| >> | Bitwise right shift => `(expr1 << expr2)` |
+
## Conditional Expressions
| Function | Notes
|
diff --git a/datafusion/_sources/user-guide/library.md.txt
b/datafusion/_sources/user-guide/library.md.txt
index 688520f9c7..c7cc1ec425 100644
--- a/datafusion/_sources/user-guide/library.md.txt
+++ b/datafusion/_sources/user-guide/library.md.txt
@@ -69,6 +69,21 @@ async fn main() -> datafusion::error::Result<()> {
}
```
+## Extensibility
+
+DataFusion is designed to be extensible at all points. To that end, you can
provide your own custom:
+
+- [x] User Defined Functions (UDFs)
+- [x] User Defined Aggregate Functions (UDAFs)
+- [x] User Defined Table Source (`TableProvider`) for tables
+- [x] User Defined `Optimizer` passes (plan rewrites)
+- [x] User Defined `LogicalPlan` nodes
+- [x] User Defined `ExecutionPlan` nodes
+
+## Rust Version Compatibility
+
+This crate is tested with the latest stable version of Rust. We do not
currently test against other, older versions of the Rust compiler.
+
## Optimized Configuration
For an optimized build several steps are required. First, use the below in
your `Cargo.toml`. It is
@@ -94,7 +109,7 @@ use datafusion::prelude::*;
static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
async fn main() -> datafusion::error::Result<()> {
- ...
+ Ok(())
}
```
diff --git a/datafusion/_sources/user-guide/sql/aggregate_functions.md.txt
b/datafusion/_sources/user-guide/sql/aggregate_functions.md.txt
index d3472a7f53..e8299b6193 100644
--- a/datafusion/_sources/user-guide/sql/aggregate_functions.md.txt
+++ b/datafusion/_sources/user-guide/sql/aggregate_functions.md.txt
@@ -53,6 +53,12 @@ Aggregate functions operate on a set of values to compute a
single result. Pleas
It supports raw data as input and build Tdigest sketches during query time,
and is approximately equal to `approx_percentile_cont_with_weight(x, 1, p)`.
+`approx_percentile_cont(x, p, n) -> x` return the approximate percentile
(TDigest) of input values, where `p` is a float64 between 0 and 1 (inclusive),
+
+and `n` (default 100) is the number of centroids in Tdigest which means that
if there are `n` or fewer unique values in `x`, you can expect an exact result.
+
+A higher value of `n` results in a more accurate approximation and the cost of
higher memory usage.
+
### approx_percentile_cont_with_weight
`approx_percentile_cont_with_weight(x, w, p) -> x` returns the approximate
percentile (TDigest) of input values with weight, where `w` is weight column
expression and `p` is a float64 between 0 and 1 (inclusive).
diff --git a/datafusion/_sources/user-guide/sql/data_types.md.txt
b/datafusion/_sources/user-guide/sql/data_types.md.txt
index 7a3ed7adb4..1cd79743ba 100644
--- a/datafusion/_sources/user-guide/sql/data_types.md.txt
+++ b/datafusion/_sources/user-guide/sql/data_types.md.txt
@@ -22,30 +22,69 @@
DataFusion uses Arrow, and thus the Arrow type system, for query
execution. The SQL types from
[sqlparser-rs](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/src/ast/data_type.rs#L27)
-are mapped to Arrow types according to the following table
-
-| SQL Data Type | Arrow DataType
|
-| ------------- |
------------------------------------------------------------------------ |
-| `CHAR` | `Utf8`
|
-| `VARCHAR` | `Utf8`
|
-| `UUID` | _Not yet supported_
|
-| `CLOB` | _Not yet supported_
|
-| `BINARY` | _Not yet supported_
|
-| `VARBINARY` | _Not yet supported_
|
-| `DECIMAL` | `Float64`
|
-| `FLOAT` | `Float32`
|
-| `SMALLINT` | `Int16`
|
-| `INT` | `Int32`
|
-| `BIGINT` | `Int64`
|
-| `REAL` | `Float32`
|
-| `DOUBLE` | `Float64`
|
-| `BOOLEAN` | `Boolean`
|
-| `DATE` | `Date32`
|
-| `TIME` | `Time64(TimeUnit::Nanosecond)`
|
-| `TIMESTAMP` | `Timestamp(TimeUnit::Nanosecond)`
|
-| `INTERVAL` | `Interval(YearMonth)` or `Interval(MonthDayNano)` or
`Interval(DayTime)` |
-| `REGCLASS` | _Not yet supported_
|
-| `TEXT` | `Utf8`
|
-| `BYTEA` | `Binary`
|
-| `CUSTOM` | _Not yet supported_
|
-| `ARRAY` | _Not yet supported_
|
+are mapped to [Arrow data
types](https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html)
according to the following table.
+This mapping occurs when defining the schema in a `CREATE EXTERNAL TABLE`
command or when performing a SQL `CAST` operation.
+
+## Character Types
+
+| SQL DataType | Arrow DataType |
+| ------------ | -------------- |
+| `CHAR` | `Utf8` |
+| `VARCHAR` | `Utf8` |
+| `TEXT` | `Utf8` |
+
+## Numeric Types
+
+| SQL DataType | Arrow DataType |
+| ------------------------------------ | :---------------- |
+| `TINYINT` | `Int8` |
+| `SMALLINT` | `Int16` |
+| `INT` or `INTEGER` | `Int32` |
+| `BIGINT` | `Int64` |
+| `TINYINT UNSIGNED` | `UInt8` |
+| `SMALLINT UNSIGNED` | `UInt16` |
+| `INT UNSIGNED` or `INTEGER UNSIGNED` | `UInt32` |
+| `BIGINT UNSIGNED` | `UInt64` |
+| `FLOAT` | `Float32` |
+| `REAL` | `Float32` |
+| `DOUBLE` | `Float64` |
+| `DECIMAL(p,s)` | `Decimal128(p,s)` |
+
+## Date/Time Types
+
+| SQL DataType | Arrow DataType |
+| ------------ | :-------------------------------------- |
+| `DATE` | `Date32` |
+| `TIME` | `Time64(TimeUnit::Nanosecond)` |
+| `TIMESTAMP` | `Timestamp(TimeUnit::Nanosecond, None)` |
+
+## Boolean Types
+
+| SQL DataType | Arrow DataType |
+| ------------ | :------------- |
+| `BOOLEAN` | `Boolean` |
+
+## Binary Types
+
+| SQL DataType | Arrow DataType |
+| ------------ | :------------- |
+| `BYTEA` | `Binary` |
+
+## Unsupported Types
+
+| SQL Data Type | Arrow DataType |
+| ------------- | :------------------ |
+| `UUID` | _Not yet supported_ |
+| `BLOB` | _Not yet supported_ |
+| `CLOB` | _Not yet supported_ |
+| `BINARY` | _Not yet supported_ |
+| `VARBINARY` | _Not yet supported_ |
+| `REGCLASS` | _Not yet supported_ |
+| `NVARCHAR` | _Not yet supported_ |
+| `STRING` | _Not yet supported_ |
+| `CUSTOM` | _Not yet supported_ |
+| `ARRAY` | _Not yet supported_ |
+| `ENUM` | _Not yet supported_ |
+| `SET` | _Not yet supported_ |
+| `INTERVAL` | _Not yet supported_ |
+| `DATETIME` | _Not yet supported_ |
diff --git a/datafusion/_sources/user-guide/sql/ddl.md.txt
b/datafusion/_sources/user-guide/sql/ddl.md.txt
index ee73370e0c..118314fdc0 100644
--- a/datafusion/_sources/user-guide/sql/ddl.md.txt
+++ b/datafusion/_sources/user-guide/sql/ddl.md.txt
@@ -78,30 +78,44 @@ PARTITIONED BY (year, month)
LOCATION '/mnt/nyctaxi';
```
-## CREATE MEMORY TABLE
+## CREATE TABLE
-Memory table can be created with query.
+An in-memory table can be created with a query or values list.
-```
-CREATE TABLE TABLE_NAME AS [SELECT | VALUES LIST]
-```
+<pre>
+CREATE [OR REPLACE] TABLE [IF NOT EXISTS] <b><i>table_name</i></b> AS [SELECT
| VALUES LIST];
+</pre>
```sql
-CREATE TABLE valuetable AS VALUES(1,'HELLO'),(12,'DATAFUSION');
+CREATE TABLE valuetable IF NOT EXISTS AS VALUES(1,'HELLO'),(12,'DATAFUSION');
CREATE TABLE memtable as select * from valuetable;
```
## DROP TABLE
-The table can be deleted.
+Removes the table from DataFusion's catalog.
-```
-DROP TABLE [ IF EXISTS ] name
-```
+<pre>
+DROP TABLE [ IF EXISTS ] <b><i>table_name</i></b>;
+</pre>
```sql
CREATE TABLE users AS VALUES(1,2),(2,3);
-
DROP TABLE users;
+-- or use 'if exists' to silently ignore if the table doesn't exist
+DROP TABLE IF EXISTS nonexistent_table;
+```
+
+## DROP VIEW
+
+Removes the view from DataFusion's catalog.
+
+<pre>
+DROP VIEW [ IF EXISTS ] <b><i>view_name</i></b>;
+</pre>
+
+```sql
+-- drop users_v view from the customer_a schema
+DROP VIEW IF EXISTS customer_a.users_v;
```
diff --git a/datafusion/_sources/user-guide/sql/information_schema.md.txt
b/datafusion/_sources/user-guide/sql/information_schema.md.txt
index ee0fbfd373..3e04fad553 100644
--- a/datafusion/_sources/user-guide/sql/information_schema.md.txt
+++ b/datafusion/_sources/user-guide/sql/information_schema.md.txt
@@ -19,50 +19,35 @@
# Information Schema
-DataFusion supports showing metadata about the tables available. This
information can be accessed using the views
-of the ISO SQL `information_schema` schema or the DataFusion specific `SHOW
TABLES` and `SHOW COLUMNS` commands.
+DataFusion supports showing metadata about the tables and views available.
This information can be accessed using the
+views of the ISO SQL `information_schema` schema or the DataFusion specific
`SHOW TABLES` and `SHOW COLUMNS` commands.
-More information can be found in the [Postgres
docs](https://www.postgresql.org/docs/13/infoschema-schema.html)).
+To show tables in the DataFusion catalog, use the `SHOW TABLES` command or the
`information_schema.tables` view:
-To show tables available for use in DataFusion, use the `SHOW TABLES` command
or the `information_schema.tables` view:
-
-```sql
+```text
> show tables;
+or
+> select * from information_schema.tables;
+---------------+--------------------+------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------------+------------+------------+
| datafusion | public | t | BASE TABLE |
| datafusion | information_schema | tables | VIEW |
+| datafusion | information_schema | views | VIEW |
+| datafusion | information_schema | columns | VIEW |
+---------------+--------------------+------------+------------+
-> select * from information_schema.tables;
-
-+---------------+--------------------+------------+--------------+
-| table_catalog | table_schema | table_name | table_type |
-+---------------+--------------------+------------+--------------+
-| datafusion | public | t | BASE TABLE |
-| datafusion | information_schema | TABLES | SYSTEM TABLE |
-+---------------+--------------------+------------+--------------+
```
-To show the schema of a table in DataFusion, use the `SHOW COLUMNS` command or
the or `information_schema.columns` view:
+To show the schema of a table in DataFusion, use the `SHOW COLUMNS` command or
the `information_schema.columns` view:
-```sql
+```text
> show columns from t;
+or
+> select table_catalog, table_schema, table_name, column_name, data_type,
is_nullable from information_schema.columns;
+---------------+--------------+------------+-------------+-----------+-------------+
| table_catalog | table_schema | table_name | column_name | data_type |
is_nullable |
+---------------+--------------+------------+-------------+-----------+-------------+
-| datafusion | public | t | a | Int32 | NO
|
-| datafusion | public | t | b | Utf8 | NO
|
-| datafusion | public | t | c | Float32 | NO
|
+| datafusion | public | t1 | Int64(1) | Int64 | NO
|
+---------------+--------------+------------+-------------+-----------+-------------+
-
-> select table_name, column_name, ordinal_position, is_nullable, data_type
from information_schema.columns;
-+------------+-------------+------------------+-------------+-----------+
-| table_name | column_name | ordinal_position | is_nullable | data_type |
-+------------+-------------+------------------+-------------+-----------+
-| t | a | 0 | NO | Int32 |
-| t | b | 1 | NO | Utf8 |
-| t | c | 2 | NO | Float32 |
-+------------+-------------+------------------+-------------+-----------+
```
diff --git a/datafusion/_sources/user-guide/sql/scalar_functions.md.txt
b/datafusion/_sources/user-guide/sql/scalar_functions.md.txt
index 0791cdf3af..11725f90d9 100644
--- a/datafusion/_sources/user-guide/sql/scalar_functions.md.txt
+++ b/datafusion/_sources/user-guide/sql/scalar_functions.md.txt
@@ -246,7 +246,7 @@ Note that `CAST(.. AS Timestamp)` converts to Timestamps
with Nanosecond resolut
`extract(field FROM source)`
- The `extract` function retrieves subfields such as year or hour from
date/time values.
- `source` must be a value expression of type timestamp, Data32, or Data64.
`field` is an identifier that selects what field to extract from the source
value.
+ `source` must be a value expression of type timestamp, Date32, or Date64.
`field` is an identifier that selects what field to extract from the source
value.
The `extract` function returns values of type u32.
- `year` :`extract(year FROM to_timestamp('2020-09-08T12:00:00+00:00')) ->
2020`
- `month`:`extract(month FROM to_timestamp('2020-09-08T12:00:00+00:00')) ->
9`
@@ -273,7 +273,8 @@ Note that `CAST(.. AS Timestamp)` converts to Timestamps
with Nanosecond resolut
### `now`
-current time
+Returns current time as `Timestamp(Nanoseconds, UTC)`. Returns same value for
the function
+wherever it appears in the statement, using a value chosen at planning time.
## Other Functions
diff --git a/datafusion/_sources/user-guide/sql/sql_status.md.txt
b/datafusion/_sources/user-guide/sql/sql_status.md.txt
index 70747b5d04..b260ecb4ba 100644
--- a/datafusion/_sources/user-guide/sql/sql_status.md.txt
+++ b/datafusion/_sources/user-guide/sql/sql_status.md.txt
@@ -72,8 +72,10 @@
- [x] to_hex
- [x] translate
- [x] trim
-- Miscellaneous/Boolean functions
+- Conditional functions
- [x] nullif
+ - [x] case
+ - [x] coalesce
- Approximation functions
- [x] approx_distinct
- [x] approx_median
@@ -93,8 +95,9 @@
- [x] Array of columns
- [x] Schema Queries
- [x] SHOW TABLES
- - [x] SHOW COLUMNS
- - [x] information_schema.{tables, columns}
+ - [x] SHOW COLUMNS FROM <table/view>
+ - [x] SHOW CREATE TABLE <view>
+ - [x] information_schema.{tables, columns, views}
- [ ] information_schema other views
- [x] Sorting
- [ ] Nested types
@@ -128,18 +131,5 @@
- [x] CSV
- [x] Parquet primitive types
- [ ] Parquet nested types
-
-## Extensibility
-
-DataFusion is designed to be extensible at all points. To that end, you can
provide your own custom:
-
-- [x] User Defined Functions (UDFs)
-- [x] User Defined Aggregate Functions (UDAFs)
-- [x] User Defined Table Source (`TableProvider`) for tables
-- [x] User Defined `Optimizer` passes (plan rewrites)
-- [x] User Defined `LogicalPlan` nodes
-- [x] User Defined `ExecutionPlan` nodes
-
-## Rust Version Compatibility
-
-This crate is tested with the latest stable version of Rust. We do not
currently test against other, older versions of the Rust compiler.
+- [x] JSON
+- [x] Avro
diff --git a/datafusion/contributor-guide/communication.html
b/datafusion/contributor-guide/communication.html
index 13716e40d8..c2abe8e423 100644
--- a/datafusion/contributor-guide/communication.html
+++ b/datafusion/contributor-guide/communication.html
@@ -332,6 +332,8 @@ questions, and get involved.</p>
<p>All participation in the Apache Arrow DataFusion project is governed by the
Apache Software Foundation’s <a class="reference external"
href="https://www.apache.org/foundation/policies/conduct.html";>code of
conduct</a>.</p>
+<p>The vast majority of communication occurs in the open on our
+<a class="reference external"
href="https://github.com/apache/arrow-datafusion";>github repository</a>.</p>
<section id="questions">
<h2>Questions?<a class="headerlink" href="#questions" title="Permalink to this
heading">¶</a></h2>
<section id="mailing-list">
diff --git a/datafusion/objects.inv b/datafusion/objects.inv
index 9d159ec284..6bb369ab0d 100644
Binary files a/datafusion/objects.inv and b/datafusion/objects.inv differ
diff --git a/datafusion/searchindex.js b/datafusion/searchindex.js
index d360c95e89..658243e926 100644
--- a/datafusion/searchindex.js
+++ b/datafusion/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["contributor-guide/communication",
"contributor-guide/index", "contributor-guide/quarterly_roadmap",
"contributor-guide/roadmap", "contributor-guide/specification/index",
"contributor-guide/specification/invariants",
"contributor-guide/specification/output-field-name-semantic", "index",
"user-guide/cli", "user-guide/configs", "user-guide/dataframe",
"user-guide/example-usage", "user-guide/expressions", "user-guide/faq",
"user-guide/introduction", "user-guide [...]
\ No newline at end of file
+Search.setIndex({"docnames": ["contributor-guide/communication",
"contributor-guide/index", "contributor-guide/quarterly_roadmap",
"contributor-guide/roadmap", "contributor-guide/specification/index",
"contributor-guide/specification/invariants",
"contributor-guide/specification/output-field-name-semantic", "index",
"user-guide/cli", "user-guide/configs", "user-guide/dataframe",
"user-guide/example-usage", "user-guide/expressions", "user-guide/faq",
"user-guide/introduction", "user-guide [...]
\ No newline at end of file
diff --git a/datafusion/user-guide/cli.html b/datafusion/user-guide/cli.html
index 247578b259..8453aa0793 100644
--- a/datafusion/user-guide/cli.html
+++ b/datafusion/user-guide/cli.html
@@ -362,7 +362,7 @@ try DataFusion out with your own data sources.</p>
</pre></div>
</div>
<div class="highlight-bash notranslate"><div
class="highlight"><pre><span></span>$ datafusion-cli
-DataFusion CLI v11.0.0
+DataFusion CLI v12.0.0
❯ CREATE EXTERNAL TABLE foo STORED AS CSV LOCATION <span
class="s1">'data.csv'</span><span class="p">;</span>
<span class="m">0</span> rows <span class="k">in</span> set. Query took <span
class="m">0</span>.017 seconds.
❯ <span class="k">select</span> * from foo<span class="p">;</span>
@@ -385,13 +385,13 @@ DataFusion CLI v11.0.0
<h3>Install and run using Homebrew (on MacOS)<a class="headerlink"
href="#install-and-run-using-homebrew-on-macos" title="Permalink to this
heading">¶</a></h3>
<p>DataFusion CLI can also be installed via Homebrew (on MacOS). Install it as
any other pre-built software like this:</p>
<div class="highlight-bash notranslate"><div
class="highlight"><pre><span></span>brew install datafusion
-<span class="c1"># ==> Downloading
https://ghcr.io/v2/homebrew/core/datafusion/manifests/5.0.0</span>
+<span class="c1"># ==> Downloading
https://ghcr.io/v2/homebrew/core/datafusion/manifests/12.0.0</span>
<span class="c1">#
########################################################################
100.0%</span>
<span class="c1"># ==> Downloading
https://ghcr.io/v2/homebrew/core/datafusion/blobs/sha256:9ecc8a01be47ceb9a53b39976696afa87c0a8</span>
<span class="c1"># ==> Downloading from
https://pkg-containers.githubusercontent.com/ghcr1/blobs/sha256:9ecc8a01be47ceb9a53b39976</span>
<span class="c1">#
########################################################################
100.0%</span>
-<span class="c1"># ==> Pouring
datafusion--5.0.0.big_sur.bottle.tar.gz</span>
-<span class="c1"># 🍺 /usr/local/Cellar/datafusion/5.0.0: 9 files,
17.4MB</span>
+<span class="c1"># ==> Pouring
datafusion--12.0.0.big_sur.bottle.tar.gz</span>
+<span class="c1"># 🍺 /usr/local/Cellar/datafusion/12.0.0: 9 files,
17.4MB</span>
datafusion-cli
</pre></div>
@@ -405,7 +405,7 @@ instead.</p>
that there is <code class="docutils literal notranslate"><span
class="pre">.dockerignore</span></code> file in the root of the repository that
may need to be deleted in order for
this to work.</p>
<div class="highlight-bash notranslate"><div
class="highlight"><pre><span></span>git clone
https://github.com/apache/arrow-datafusion
-git checkout <span class="m">11</span>.0.0
+git checkout <span class="m">12</span>.0.0
<span class="nb">cd</span> arrow-datafusion
docker build -f datafusion-cli/Dockerfile . --tag datafusion-cli
docker run -it -v <span class="k">$(</span>your_data_location<span
class="k">)</span>:/data datafusion-cli
diff --git a/datafusion/user-guide/dataframe.html
b/datafusion/user-guide/dataframe.html
index 5821042a9b..a0f8b7fcd8 100644
--- a/datafusion/user-guide/dataframe.html
+++ b/datafusion/user-guide/dataframe.html
@@ -336,7 +336,7 @@ to build up a query definition.</p>
<span class="kd">let</span><span class="w"> </span><span
class="n">df</span><span class="w"> </span><span class="o">=</span><span
class="w"> </span><span class="n">ctx</span><span class="p">.</span><span
class="n">read_csv</span><span class="p">(</span><span
class="s">"tests/example.csv"</span><span class="p">,</span><span
class="w"> </span><span class="n">CsvReadOptions</span>::<span
class="n">new</span><span class="p">()).</span><span
class="k">await</span><span class="o">?</ [...]
<span class="kd">let</span><span class="w"> </span><span
class="n">df</span><span class="w"> </span><span class="o">=</span><span
class="w"> </span><span class="n">df</span><span class="p">.</span><span
class="n">filter</span><span class="p">(</span><span class="n">col</span><span
class="p">(</span><span class="s">"a"</span><span
class="p">).</span><span class="n">lt_eq</span><span class="p">(</span><span
class="n">col</span><span class="p">(</span><span class="s">"b" [...]
<span class="w"> </span><span class="p">.</span><span
class="n">aggregate</span><span class="p">(</span><span
class="fm">vec!</span><span class="p">[</span><span class="n">col</span><span
class="p">(</span><span class="s">"a"</span><span
class="p">)],</span><span class="w"> </span><span class="fm">vec!</span><span
class="p">[</span><span class="n">min</span><span class="p">(</span><span
class="n">col</span><span class="p">(</span><span
class="s">"b"</span><s [...]
-<span class="w"> </span><span class="p">.</span><span
class="n">limit</span><span class="p">(</span><span class="nb">None</span><span
class="p">,</span><span class="w"> </span><span class="nb">Some</span><span
class="p">(</span><span class="mi">100</span><span class="p">))</span><span
class="o">?</span><span class="p">;</span><span class="w"></span>
+<span class="w"> </span><span class="p">.</span><span
class="n">limit</span><span class="p">(</span><span class="mi">0</span><span
class="p">,</span><span class="w"> </span><span class="nb">Some</span><span
class="p">(</span><span class="mi">100</span><span class="p">))</span><span
class="o">?</span><span class="p">;</span><span class="w"></span>
<span class="c1">// Print results</span>
<span class="n">df</span><span class="p">.</span><span
class="n">show</span><span class="p">();</span><span class="w"></span>
</pre></div>
@@ -464,7 +464,10 @@ execution. The plan is evaluated (executed) when an action
method is invoked, su
<td><p>Returns the schema describing the output of this DataFrame in terms of
columns returned, where each column has a name, data type, and nullability
attribute.</p></td>
</tr>
<tr class="row-odd"><td><p>to_logical_plan</p></td>
-<td><p>Return the logical plan represented by this DataFrame.</p></td>
+<td><p>Return the optimized logical plan represented by this
DataFrame.</p></td>
+</tr>
+<tr class="row-even"><td><p>to_unoptimized_plan</p></td>
+<td><p>Return the unoptimized logical plan represented by this
DataFrame.</p></td>
</tr>
</tbody>
</table>
diff --git a/datafusion/user-guide/expressions.html
b/datafusion/user-guide/expressions.html
index 458f0a4e1c..a273f602bd 100644
--- a/datafusion/user-guide/expressions.html
+++ b/datafusion/user-guide/expressions.html
@@ -282,6 +282,11 @@
Math Functions
</a>
</li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#bitwise-operators">
+ Bitwise Operators
+ </a>
+ </li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#conditional-expressions">
Conditional Expressions
@@ -529,6 +534,33 @@ expressions such as <code class="docutils literal
notranslate"><span class="pre"
</tbody>
</table>
</section>
+<section id="bitwise-operators">
+<h2>Bitwise Operators<a class="headerlink" href="#bitwise-operators"
title="Permalink to this heading">¶</a></h2>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Operator</p></th>
+<th class="head"><p>Notes</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p>&</p></td>
+<td><p>Bitwise AND => <code class="docutils literal notranslate"><span
class="pre">(expr1</span> <span class="pre">&</span> <span
class="pre">expr2)</span></code></p></td>
+</tr>
+<tr class="row-odd"><td><p>|</p></td>
+<td><p>Bitwise OR => <code>(expr1 | expr2)</code></p></td>
+</tr>
+<tr class="row-even"><td><p>#</p></td>
+<td><p>Bitwise XOR => <code class="docutils literal notranslate"><span
class="pre">(expr1</span> <span class="pre">#</span> <span
class="pre">expr2)</span></code></p></td>
+</tr>
+<tr class="row-odd"><td><p><<</p></td>
+<td><p>Bitwise left shift => <code class="docutils literal
notranslate"><span class="pre">(expr1</span> <span class="pre"><<</span>
<span class="pre">expr2)</span></code></p></td>
+</tr>
+<tr class="row-even"><td><p>>></p></td>
+<td><p>Bitwise right shift => <code class="docutils literal
notranslate"><span class="pre">(expr1</span> <span class="pre"><<</span>
<span class="pre">expr2)</span></code></p></td>
+</tr>
+</tbody>
+</table>
+</section>
<section id="conditional-expressions">
<h2>Conditional Expressions<a class="headerlink"
href="#conditional-expressions" title="Permalink to this heading">¶</a></h2>
<table class="table">
diff --git a/datafusion/user-guide/library.html
b/datafusion/user-guide/library.html
index 117623486e..8c9869f617 100644
--- a/datafusion/user-guide/library.html
+++ b/datafusion/user-guide/library.html
@@ -272,6 +272,16 @@
Create a main function
</a>
</li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#extensibility">
+ Extensibility
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#rust-version-compatibility">
+ Rust Version Compatibility
+ </a>
+ </li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#optimized-configuration">
Optimized Configuration
@@ -371,6 +381,22 @@ $ tree .
</pre></div>
</div>
</section>
+<section id="extensibility">
+<h2>Extensibility<a class="headerlink" href="#extensibility" title="Permalink
to this heading">¶</a></h2>
+<p>DataFusion is designed to be extensible at all points. To that end, you can
provide your own custom:</p>
+<ul class="contains-task-list simple">
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined Functions
(UDFs)</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined Aggregate
Functions (UDAFs)</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined Table
Source (<code class="docutils literal notranslate"><span
class="pre">TableProvider</span></code>) for tables</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined <code
class="docutils literal notranslate"><span class="pre">Optimizer</span></code>
passes (plan rewrites)</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined <code
class="docutils literal notranslate"><span
class="pre">LogicalPlan</span></code> nodes</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined <code
class="docutils literal notranslate"><span
class="pre">ExecutionPlan</span></code> nodes</p></li>
+</ul>
+</section>
+<section id="rust-version-compatibility">
+<h2>Rust Version Compatibility<a class="headerlink"
href="#rust-version-compatibility" title="Permalink to this heading">¶</a></h2>
+<p>This crate is tested with the latest stable version of Rust. We do not
currently test against other, older versions of the Rust compiler.</p>
+</section>
<section id="optimized-configuration">
<h2>Optimized Configuration<a class="headerlink"
href="#optimized-configuration" title="Permalink to this heading">¶</a></h2>
<p>For an optimized build several steps are required. First, use the below in
your <code class="docutils literal notranslate"><span
class="pre">Cargo.toml</span></code>. It is
@@ -392,7 +418,7 @@ worth noting that using the settings in the <code
class="docutils literal notran
<span class="k">static</span><span class="w"> </span><span
class="n">ALLOC</span>: <span class="nc">snmalloc_rs</span>::<span
class="n">SnMalloc</span><span class="w"> </span><span class="o">=</span><span
class="w"> </span><span class="n">snmalloc_rs</span>::<span
class="n">SnMalloc</span><span class="p">;</span><span class="w"></span>
<span class="k">async</span><span class="w"> </span><span class="k">fn</span>
<span class="nf">main</span><span class="p">()</span><span class="w">
</span>-> <span class="nc">datafusion</span>::<span
class="n">error</span>::<span class="nb">Result</span><span
class="o"><</span><span class="p">()</span><span class="o">></span><span
class="w"> </span><span class="p">{</span><span class="w"></span>
-<span class="w"> </span><span class="o">..</span><span
class="p">.</span><span class="w"></span>
+<span class="w"> </span><span class="nb">Ok</span><span
class="p">(())</span><span class="w"></span>
<span class="p">}</span><span class="w"></span>
</pre></div>
</div>
diff --git a/datafusion/user-guide/sql/aggregate_functions.html
b/datafusion/user-guide/sql/aggregate_functions.html
index 3f41c63faa..d48407e785 100644
--- a/datafusion/user-guide/sql/aggregate_functions.html
+++ b/datafusion/user-guide/sql/aggregate_functions.html
@@ -377,6 +377,9 @@
<h3>approx_percentile_cont<a class="headerlink" href="#approx-percentile-cont"
title="Permalink to this heading">¶</a></h3>
<p><code class="docutils literal notranslate"><span
class="pre">approx_percentile_cont(x,</span> <span class="pre">p)</span> <span
class="pre">-></span> <span class="pre">x</span></code> return the
approximate percentile (TDigest) of input values, where <code class="docutils
literal notranslate"><span class="pre">p</span></code> is a float64 between 0
and 1 (inclusive).</p>
<p>It supports raw data as input and build Tdigest sketches during query time,
and is approximately equal to <code class="docutils literal notranslate"><span
class="pre">approx_percentile_cont_with_weight(x,</span> <span
class="pre">1,</span> <span class="pre">p)</span></code>.</p>
+<p><code class="docutils literal notranslate"><span
class="pre">approx_percentile_cont(x,</span> <span class="pre">p,</span> <span
class="pre">n)</span> <span class="pre">-></span> <span
class="pre">x</span></code> return the approximate percentile (TDigest) of
input values, where <code class="docutils literal notranslate"><span
class="pre">p</span></code> is a float64 between 0 and 1 (inclusive),</p>
+<p>and <code class="docutils literal notranslate"><span
class="pre">n</span></code> (default 100) is the number of centroids in Tdigest
which means that if there are <code class="docutils literal notranslate"><span
class="pre">n</span></code> or fewer unique values in <code class="docutils
literal notranslate"><span class="pre">x</span></code>, you can expect an exact
result.</p>
+<p>A higher value of <code class="docutils literal notranslate"><span
class="pre">n</span></code> results in a more accurate approximation and the
cost of higher memory usage.</p>
</section>
<section id="approx-percentile-cont-with-weight">
<h3>approx_percentile_cont_with_weight<a class="headerlink"
href="#approx-percentile-cont-with-weight" title="Permalink to this
heading">¶</a></h3>
diff --git a/datafusion/user-guide/sql/data_types.html
b/datafusion/user-guide/sql/data_types.html
index 902ce9cf91..2037c50eb7 100644
--- a/datafusion/user-guide/sql/data_types.html
+++ b/datafusion/user-guide/sql/data_types.html
@@ -251,9 +251,44 @@
<div class="toc-item">
+<div class="tocsection onthispage pt-5 pb-3">
+ <i class="fas fa-list"></i> On this page
+</div>
<nav id="bd-toc-nav">
-
+ <ul class="visible nav section-nav flex-column">
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#character-types">
+ Character Types
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#numeric-types">
+ Numeric Types
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#date-time-types">
+ Date/Time Types
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#boolean-types">
+ Boolean Types
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#binary-types">
+ Binary Types
+ </a>
+ </li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#unsupported-types">
+ Unsupported Types
+ </a>
+ </li>
+</ul>
+
</nav>
</div>
@@ -303,10 +338,13 @@
<p>DataFusion uses Arrow, and thus the Arrow type system, for query
execution. The SQL types from
<a class="reference external"
href="https://github.com/sqlparser-rs/sqlparser-rs/blob/main/src/ast/data_type.rs#L27";>sqlparser-rs</a>
-are mapped to Arrow types according to the following table</p>
+are mapped to <a class="reference external"
href="https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html";>Arrow
data types</a> according to the following table.
+This mapping occurs when defining the schema in a <code class="docutils
literal notranslate"><span class="pre">CREATE</span> <span
class="pre">EXTERNAL</span> <span class="pre">TABLE</span></code> command or
when performing a SQL <code class="docutils literal notranslate"><span
class="pre">CAST</span></code> operation.</p>
+<section id="character-types">
+<h2>Character Types<a class="headerlink" href="#character-types"
title="Permalink to this heading">¶</a></h2>
<table class="table">
<thead>
-<tr class="row-odd"><th class="head"><p>SQL Data Type</p></th>
+<tr class="row-odd"><th class="head"><p>SQL DataType</p></th>
<th class="head"><p>Arrow DataType</p></th>
</tr>
</thead>
@@ -317,71 +355,165 @@ are mapped to Arrow types according to the following
table</p>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">VARCHAR</span></code></p></td>
<td><p><code class="docutils literal notranslate"><span
class="pre">Utf8</span></code></p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">UUID</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">TEXT</span></code></p></td>
+<td><p><code class="docutils literal notranslate"><span
class="pre">Utf8</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">CLOB</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+</tbody>
+</table>
+</section>
+<section id="numeric-types">
+<h2>Numeric Types<a class="headerlink" href="#numeric-types" title="Permalink
to this heading">¶</a></h2>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>SQL DataType</p></th>
+<th class="head text-left"><p>Arrow DataType</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">TINYINT</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Int8</span></code></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">SMALLINT</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Int16</span></code></p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">BINARY</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">INT</span></code> or <code class="docutils literal
notranslate"><span class="pre">INTEGER</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Int32</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">VARBINARY</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">BIGINT</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Int64</span></code></p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">DECIMAL</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Float64</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">TINYINT</span> <span class="pre">UNSIGNED</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">UInt8</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">FLOAT</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Float32</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">SMALLINT</span> <span class="pre">UNSIGNED</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">UInt16</span></code></p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">SMALLINT</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Int16</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">INT</span> <span class="pre">UNSIGNED</span></code> or <code
class="docutils literal notranslate"><span class="pre">INTEGER</span> <span
class="pre">UNSIGNED</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">UInt32</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">INT</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Int32</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">BIGINT</span> <span class="pre">UNSIGNED</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">UInt64</span></code></p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">BIGINT</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Int64</span></code></p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">FLOAT</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Float32</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">REAL</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Float32</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Float32</span></code></p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">DOUBLE</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Float64</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Float64</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">BOOLEAN</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Boolean</span></code></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">DECIMAL(p,s)</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Decimal128(p,s)</span></code></p></td>
</tr>
+</tbody>
+</table>
+</section>
+<section id="date-time-types">
+<h2>Date/Time Types<a class="headerlink" href="#date-time-types"
title="Permalink to this heading">¶</a></h2>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>SQL DataType</p></th>
+<th class="head text-left"><p>Arrow DataType</p></th>
+</tr>
+</thead>
+<tbody>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">DATE</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Date32</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Date32</span></code></p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">TIME</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Time64(TimeUnit::Nanosecond)</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Time64(TimeUnit::Nanosecond)</span></code></p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">TIMESTAMP</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Timestamp(TimeUnit::Nanosecond)</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Timestamp(TimeUnit::Nanosecond,</span> <span
class="pre">None)</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">INTERVAL</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Interval(YearMonth)</span></code> or <code class="docutils literal
notranslate"><span class="pre">Interval(MonthDayNano)</span></code> or <code
class="docutils literal notranslate"><span
class="pre">Interval(DayTime)</span></code></p></td>
+</tbody>
+</table>
+</section>
+<section id="boolean-types">
+<h2>Boolean Types<a class="headerlink" href="#boolean-types" title="Permalink
to this heading">¶</a></h2>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>SQL DataType</p></th>
+<th class="head text-left"><p>Arrow DataType</p></th>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">REGCLASS</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">BOOLEAN</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Boolean</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">TEXT</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Utf8</span></code></p></td>
+</tbody>
+</table>
+</section>
+<section id="binary-types">
+<h2>Binary Types<a class="headerlink" href="#binary-types" title="Permalink to
this heading">¶</a></h2>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>SQL DataType</p></th>
+<th class="head text-left"><p>Arrow DataType</p></th>
</tr>
+</thead>
+<tbody>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">BYTEA</span></code></p></td>
-<td><p><code class="docutils literal notranslate"><span
class="pre">Binary</span></code></p></td>
+<td class="text-left"><p><code class="docutils literal notranslate"><span
class="pre">Binary</span></code></p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">CUSTOM</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+</tbody>
+</table>
+</section>
+<section id="unsupported-types">
+<h2>Unsupported Types<a class="headerlink" href="#unsupported-types"
title="Permalink to this heading">¶</a></h2>
+<table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>SQL Data Type</p></th>
+<th class="head text-left"><p>Arrow DataType</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">UUID</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">BLOB</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">CLOB</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">BINARY</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">VARBINARY</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
</tr>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">ARRAY</span></code></p></td>
-<td><p><em>Not yet supported</em></p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">REGCLASS</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">NVARCHAR</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">STRING</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">CUSTOM</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">ARRAY</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">ENUM</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">SET</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span
class="pre">INTERVAL</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
+</tr>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span
class="pre">DATETIME</span></code></p></td>
+<td class="text-left"><p><em>Not yet supported</em></p></td>
</tr>
</tbody>
</table>
+</section>
</section>
diff --git a/datafusion/user-guide/sql/ddl.html
b/datafusion/user-guide/sql/ddl.html
index d717ddb9cd..c7ec7ac059 100644
--- a/datafusion/user-guide/sql/ddl.html
+++ b/datafusion/user-guide/sql/ddl.html
@@ -263,8 +263,8 @@
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#create-memory-table">
- CREATE MEMORY TABLE
+ <a class="reference internal nav-link" href="#create-table">
+ CREATE TABLE
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
@@ -272,6 +272,11 @@
DROP TABLE
</a>
</li>
+ <li class="toc-h2 nav-item toc-entry">
+ <a class="reference internal nav-link" href="#drop-view">
+ DROP VIEW
+ </a>
+ </li>
</ul>
</nav>
@@ -371,13 +376,13 @@ inferred based on scanning a subset of the file.</p>
</pre></div>
</div>
</section>
-<section id="create-memory-table">
-<h2>CREATE MEMORY TABLE<a class="headerlink" href="#create-memory-table"
title="Permalink to this heading">¶</a></h2>
-<p>Memory table can be created with query.</p>
-<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">CREATE</span> <span
class="n">TABLE</span> <span class="n">TABLE_NAME</span> <span
class="n">AS</span> <span class="p">[</span><span class="n">SELECT</span> <span
class="o">|</span> <span class="n">VALUES</span> <span
class="n">LIST</span><span class="p">]</span>
-</pre></div>
-</div>
-<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="k">CREATE</span><span
class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span
class="n">valuetable</span><span class="w"> </span><span
class="k">AS</span><span class="w"> </span><span class="k">VALUES</span><span
class="p">(</span><span class="mi">1</span><span class="p">,</span><span
class="s1">'HELLO'</span><span class="p">),(</span><span
class="mi">12</span><span cl [...]
+<section id="create-table">
+<h2>CREATE TABLE<a class="headerlink" href="#create-table" title="Permalink to
this heading">¶</a></h2>
+<p>An in-memory table can be created with a query or values list.</p>
+<pre>
+CREATE [OR REPLACE] TABLE [IF NOT EXISTS] <b><i>table_name</i></b> AS [SELECT
| VALUES LIST];
+</pre>
+<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="k">CREATE</span><span
class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span
class="n">valuetable</span><span class="w"> </span><span
class="k">IF</span><span class="w"> </span><span class="k">NOT</span><span
class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span
class="k">AS</span><span class="w"> </span><span class="k">VALUES</span><span
class="p">(</s [...]
<span class="k">CREATE</span><span class="w"> </span><span
class="k">TABLE</span><span class="w"> </span><span
class="n">memtable</span><span class="w"> </span><span class="k">as</span><span
class="w"> </span><span class="k">select</span><span class="w"> </span><span
class="o">*</span><span class="w"> </span><span class="k">from</span><span
class="w"> </span><span class="n">valuetable</span><span
class="p">;</span><span class="w"></span>
</pre></div>
@@ -385,13 +390,25 @@ inferred based on scanning a subset of the file.</p>
</section>
<section id="drop-table">
<h2>DROP TABLE<a class="headerlink" href="#drop-table" title="Permalink to
this heading">¶</a></h2>
-<p>The table can be deleted.</p>
-<div class="highlight-default notranslate"><div
class="highlight"><pre><span></span><span class="n">DROP</span> <span
class="n">TABLE</span> <span class="p">[</span> <span class="n">IF</span> <span
class="n">EXISTS</span> <span class="p">]</span> <span class="n">name</span>
-</pre></div>
-</div>
+<p>Removes the table from DataFusion’s catalog.</p>
+<pre>
+DROP TABLE [ IF EXISTS ] <b><i>table_name</i></b>;
+</pre>
<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="k">CREATE</span><span
class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span
class="n">users</span><span class="w"> </span><span class="k">AS</span><span
class="w"> </span><span class="k">VALUES</span><span class="p">(</span><span
class="mi">1</span><span class="p">,</span><span class="mi">2</span><span
class="p">),(</span><span class="mi">2</span><span class="p">,</span><spa [...]
-
<span class="k">DROP</span><span class="w"> </span><span
class="k">TABLE</span><span class="w"> </span><span class="n">users</span><span
class="p">;</span><span class="w"></span>
+<span class="c1">-- or use 'if exists' to silently ignore if the table
doesn't exist</span>
+<span class="k">DROP</span><span class="w"> </span><span
class="k">TABLE</span><span class="w"> </span><span class="k">IF</span><span
class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span
class="n">nonexistent_table</span><span class="p">;</span><span
class="w"></span>
+</pre></div>
+</div>
+</section>
+<section id="drop-view">
+<h2>DROP VIEW<a class="headerlink" href="#drop-view" title="Permalink to this
heading">¶</a></h2>
+<p>Removes the view from DataFusion’s catalog.</p>
+<pre>
+DROP VIEW [ IF EXISTS ] <b><i>view_name</i></b>;
+</pre>
+<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="c1">-- drop users_v view from
the customer_a schema</span>
+<span class="k">DROP</span><span class="w"> </span><span
class="k">VIEW</span><span class="w"> </span><span class="k">IF</span><span
class="w"> </span><span class="k">EXISTS</span><span class="w"> </span><span
class="n">customer_a</span><span class="p">.</span><span
class="n">users_v</span><span class="p">;</span><span class="w"></span>
</pre></div>
</div>
</section>
diff --git a/datafusion/user-guide/sql/index.html
b/datafusion/user-guide/sql/index.html
index 22c119ac37..dce3b5e3e3 100644
--- a/datafusion/user-guide/sql/index.html
+++ b/datafusion/user-guide/sql/index.html
@@ -284,7 +284,15 @@
<h1>SQL Reference<a class="headerlink" href="#sql-reference" title="Permalink
to this heading">¶</a></h1>
<div class="toctree-wrapper compound">
<ul>
-<li class="toctree-l1"><a class="reference internal"
href="data_types.html">Data Types</a></li>
+<li class="toctree-l1"><a class="reference internal"
href="data_types.html">Data Types</a><ul>
+<li class="toctree-l2"><a class="reference internal"
href="data_types.html#character-types">Character Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="data_types.html#numeric-types">Numeric Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="data_types.html#date-time-types">Date/Time Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="data_types.html#boolean-types">Boolean Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="data_types.html#binary-types">Binary Types</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="data_types.html#unsupported-types">Unsupported Types</a></li>
+</ul>
+</li>
<li class="toctree-l1"><a class="reference internal" href="select.html">SELECT
syntax</a><ul>
<li class="toctree-l2"><a class="reference internal"
href="select.html#with-clause">WITH clause</a></li>
<li class="toctree-l2"><a class="reference internal"
href="select.html#select-clause">SELECT clause</a></li>
@@ -308,8 +316,9 @@
</li>
<li class="toctree-l1"><a class="reference internal"
href="ddl.html">DDL</a><ul>
<li class="toctree-l2"><a class="reference internal"
href="ddl.html#create-external-table">CREATE EXTERNAL TABLE</a></li>
-<li class="toctree-l2"><a class="reference internal"
href="ddl.html#create-memory-table">CREATE MEMORY TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="ddl.html#create-table">CREATE TABLE</a></li>
<li class="toctree-l2"><a class="reference internal"
href="ddl.html#drop-table">DROP TABLE</a></li>
+<li class="toctree-l2"><a class="reference internal"
href="ddl.html#drop-view">DROP VIEW</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal"
href="information_schema.html">Information Schema</a></li>
@@ -332,8 +341,6 @@
<li class="toctree-l2"><a class="reference internal"
href="sql_status.html#general">General</a></li>
<li class="toctree-l2"><a class="reference internal"
href="sql_status.html#sql-support">SQL Support</a></li>
<li class="toctree-l2"><a class="reference internal"
href="sql_status.html#data-sources">Data Sources</a></li>
-<li class="toctree-l2"><a class="reference internal"
href="sql_status.html#extensibility">Extensibility</a></li>
-<li class="toctree-l2"><a class="reference internal"
href="sql_status.html#rust-version-compatibility">Rust Version
Compatibility</a></li>
</ul>
</li>
</ul>
diff --git a/datafusion/user-guide/sql/information_schema.html
b/datafusion/user-guide/sql/information_schema.html
index 41cea16db4..863f847c55 100644
--- a/datafusion/user-guide/sql/information_schema.html
+++ b/datafusion/user-guide/sql/information_schema.html
@@ -300,46 +300,31 @@
-->
<section id="information-schema">
<h1>Information Schema<a class="headerlink" href="#information-schema"
title="Permalink to this heading">¶</a></h1>
-<p>DataFusion supports showing metadata about the tables available. This
information can be accessed using the views
-of the ISO SQL <code class="docutils literal notranslate"><span
class="pre">information_schema</span></code> schema or the DataFusion specific
<code class="docutils literal notranslate"><span class="pre">SHOW</span> <span
class="pre">TABLES</span></code> and <code class="docutils literal
notranslate"><span class="pre">SHOW</span> <span
class="pre">COLUMNS</span></code> commands.</p>
-<p>More information can be found in the <a class="reference external"
href="https://www.postgresql.org/docs/13/infoschema-schema.html";>Postgres
docs</a>).</p>
-<p>To show tables available for use in DataFusion, use the <code
class="docutils literal notranslate"><span class="pre">SHOW</span> <span
class="pre">TABLES</span></code> command or the <code class="docutils literal
notranslate"><span class="pre">information_schema.tables</span></code> view:</p>
-<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="o">></span><span class="w">
</span><span class="k">show</span><span class="w"> </span><span
class="n">tables</span><span class="p">;</span><span class="w"></span>
-<span class="o">+</span><span
class="c1">---------------+--------------------+------------+------------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="n">table_catalog</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">table_schema</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="k">table_name</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">table_type</span><span class="w"> </span><span
class="o">|</span><span class="w"></span>
-<span class="o">+</span><span
class="c1">---------------+--------------------+------------+------------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="k">public</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">t</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">BASE</span><span
class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span
class="o" [...]
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">information_schema</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">tables</span><span
class="w"> </span><span class="o">|</span><span class="w"> </span><span
class="k">VIEW</span><span class="w"> </span><span
class="o">|</span><span class="w"></span>
-<span class="o">+</span><span
class="c1">---------------+--------------------+------------+------------+</span>
-
-<span class="o">></span><span class="w"> </span><span
class="k">select</span><span class="w"> </span><span class="o">*</span><span
class="w"> </span><span class="k">from</span><span class="w"> </span><span
class="n">information_schema</span><span class="p">.</span><span
class="n">tables</span><span class="p">;</span><span class="w"></span>
-
-<span class="o">+</span><span
class="c1">---------------+--------------------+------------+--------------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="n">table_catalog</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">table_schema</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="k">table_name</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">table_type</span><span class="w"> </span><span
class="o">|</span><span class="w"></span>
-<span class="o">+</span><span
class="c1">---------------+--------------------+------------+--------------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="k">public</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">t</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">BASE</span><span
class="w"> </span><span class="k">TABLE</span><span class="w"> </span><span
class=" [...]
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">information_schema</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">TABLES</span><span
class="w"> </span><span class="o">|</span><span class="w"> </span><span
class="k">SYSTEM</span><span class="w"> </span><span
class="k">TABLE</span><span class="w"> </span><span class=" [...]
-<span class="o">+</span><span
class="c1">---------------+--------------------+------------+--------------+</span>
+<p>DataFusion supports showing metadata about the tables and views available.
This information can be accessed using the
+views of the ISO SQL <code class="docutils literal notranslate"><span
class="pre">information_schema</span></code> schema or the DataFusion specific
<code class="docutils literal notranslate"><span class="pre">SHOW</span> <span
class="pre">TABLES</span></code> and <code class="docutils literal
notranslate"><span class="pre">SHOW</span> <span
class="pre">COLUMNS</span></code> commands.</p>
+<p>To show tables in the DataFusion catalog, use the <code class="docutils
literal notranslate"><span class="pre">SHOW</span> <span
class="pre">TABLES</span></code> command or the <code class="docutils literal
notranslate"><span class="pre">information_schema.tables</span></code> view:</p>
+<div class="highlight-text notranslate"><div
class="highlight"><pre><span></span>> show tables;
+or
+> select * from information_schema.tables;
++---------------+--------------------+------------+------------+
+| table_catalog | table_schema | table_name | table_type |
++---------------+--------------------+------------+------------+
+| datafusion | public | t | BASE TABLE |
+| datafusion | information_schema | tables | VIEW |
+| datafusion | information_schema | views | VIEW |
+| datafusion | information_schema | columns | VIEW |
++---------------+--------------------+------------+------------+
</pre></div>
</div>
-<p>To show the schema of a table in DataFusion, use the <code class="docutils
literal notranslate"><span class="pre">SHOW</span> <span
class="pre">COLUMNS</span></code> command or the or <code class="docutils
literal notranslate"><span class="pre">information_schema.columns</span></code>
view:</p>
-<div class="highlight-sql notranslate"><div
class="highlight"><pre><span></span><span class="o">></span><span class="w">
</span><span class="k">show</span><span class="w"> </span><span
class="n">columns</span><span class="w"> </span><span
class="k">from</span><span class="w"> </span><span class="n">t</span><span
class="p">;</span><span class="w"></span>
-<span class="o">+</span><span
class="c1">---------------+--------------+------------+-------------+-----------+-------------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="n">table_catalog</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">table_schema</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="k">table_name</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="k">column_name</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">da [...]
-<span class="o">+</span><span
class="c1">---------------+--------------+------------+-------------+-----------+-------------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="k">public</span><span
class="w"> </span><span class="o">|</span><span class="w"> </span><span
class="n">t</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">a</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">In [...]
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="k">public</span><span
class="w"> </span><span class="o">|</span><span class="w"> </span><span
class="n">t</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n">b</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">Ut [...]
-<span class="o">|</span><span class="w"> </span><span
class="n">datafusion</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="k">public</span><span
class="w"> </span><span class="o">|</span><span class="w"> </span><span
class="n">t</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="k">c</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">Fl [...]
-<span class="o">+</span><span
class="c1">---------------+--------------+------------+-------------+-----------+-------------+</span>
-
-<span class="o">></span><span class="w"> </span><span
class="k">select</span><span class="w"> </span><span
class="k">table_name</span><span class="p">,</span><span class="w">
</span><span class="k">column_name</span><span class="p">,</span><span
class="w"> </span><span class="n">ordinal_position</span><span
class="p">,</span><span class="w"> </span><span
class="n">is_nullable</span><span class="p">,</span><span class="w">
</span><span class="n">data_type</span><span class="w"> </span> [...]
-<span class="o">+</span><span
class="c1">------------+-------------+------------------+-------------+-----------+</span>
-<span class="o">|</span><span class="w"> </span><span
class="k">table_name</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="k">column_name</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">ordinal_position</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span
class="n">is_nullable</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n"> [...]
-<span class="o">+</span><span
class="c1">------------+-------------+------------------+-------------+-----------+</span>
-<span class="o">|</span><span class="w"> </span><span class="n">t</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">a</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="mi">0</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="k">NO</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n" [...]
-<span class="o">|</span><span class="w"> </span><span class="n">t</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="n">b</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="mi">1</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="k">NO</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n" [...]
-<span class="o">|</span><span class="w"> </span><span class="n">t</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="k">c</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="mi">2</span><span
class="w"> </span><span class="o">|</span><span class="w">
</span><span class="k">NO</span><span class="w"> </span><span
class="o">|</span><span class="w"> </span><span class="n" [...]
-<span class="o">+</span><span
class="c1">------------+-------------+------------------+-------------+-----------+</span>
+<p>To show the schema of a table in DataFusion, use the <code class="docutils
literal notranslate"><span class="pre">SHOW</span> <span
class="pre">COLUMNS</span></code> command or the <code class="docutils literal
notranslate"><span class="pre">information_schema.columns</span></code>
view:</p>
+<div class="highlight-text notranslate"><div
class="highlight"><pre><span></span>> show columns from t;
+or
+> select table_catalog, table_schema, table_name, column_name, data_type,
is_nullable from information_schema.columns;
++---------------+--------------+------------+-------------+-----------+-------------+
+| table_catalog | table_schema | table_name | column_name | data_type |
is_nullable |
++---------------+--------------+------------+-------------+-----------+-------------+
+| datafusion | public | t1 | Int64(1) | Int64 | NO
|
++---------------+--------------+------------+-------------+-----------+-------------+
</pre></div>
</div>
</section>
diff --git a/datafusion/user-guide/sql/scalar_functions.html
b/datafusion/user-guide/sql/scalar_functions.html
index 2e8850f863..a42d73978c 100644
--- a/datafusion/user-guide/sql/scalar_functions.html
+++ b/datafusion/user-guide/sql/scalar_functions.html
@@ -1268,7 +1268,7 @@
<p><code class="docutils literal notranslate"><span
class="pre">extract(field</span> <span class="pre">FROM</span> <span
class="pre">source)</span></code></p>
<ul class="simple">
<li><p>The <code class="docutils literal notranslate"><span
class="pre">extract</span></code> function retrieves subfields such as year or
hour from date/time values.
-<code class="docutils literal notranslate"><span
class="pre">source</span></code> must be a value expression of type timestamp,
Data32, or Data64. <code class="docutils literal notranslate"><span
class="pre">field</span></code> is an identifier that selects what field to
extract from the source value.
+<code class="docutils literal notranslate"><span
class="pre">source</span></code> must be a value expression of type timestamp,
Date32, or Date64. <code class="docutils literal notranslate"><span
class="pre">field</span></code> is an identifier that selects what field to
extract from the source value.
The <code class="docutils literal notranslate"><span
class="pre">extract</span></code> function returns values of type u32.</p>
<ul>
<li><p><code class="docutils literal notranslate"><span
class="pre">year</span></code> :<code class="docutils literal
notranslate"><span class="pre">extract(year</span> <span
class="pre">FROM</span> <span
class="pre">to_timestamp('2020-09-08T12:00:00+00:00'))</span> <span
class="pre">-></span> <span class="pre">2020</span></code></p></li>
@@ -1306,7 +1306,8 @@ The valid field names for <code class="docutils literal
notranslate"><span class
</section>
<section id="now">
<h3><code class="docutils literal notranslate"><span
class="pre">now</span></code><a class="headerlink" href="#now" title="Permalink
to this heading">¶</a></h3>
-<p>current time</p>
+<p>Returns current time as <code class="docutils literal notranslate"><span
class="pre">Timestamp(Nanoseconds,</span> <span class="pre">UTC)</span></code>.
Returns same value for the function
+wherever it appears in the statement, using a value chosen at planning
time.</p>
</section>
</section>
<section id="other-functions">
diff --git a/datafusion/user-guide/sql/sql_status.html
b/datafusion/user-guide/sql/sql_status.html
index 08a0d09588..6e11141661 100644
--- a/datafusion/user-guide/sql/sql_status.html
+++ b/datafusion/user-guide/sql/sql_status.html
@@ -272,16 +272,6 @@
Data Sources
</a>
</li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#extensibility">
- Extensibility
- </a>
- </li>
- <li class="toc-h2 nav-item toc-entry">
- <a class="reference internal nav-link" href="#rust-version-compatibility">
- Rust Version Compatibility
- </a>
- </li>
</ul>
</nav>
@@ -389,9 +379,11 @@
<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> trim</p></li>
</ul>
</li>
-<li><p>Miscellaneous/Boolean functions</p>
+<li><p>Conditional functions</p>
<ul class="contains-task-list">
<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> nullif</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> case</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> coalesce</p></li>
</ul>
</li>
<li><p>Approximation functions</p>
@@ -426,8 +418,9 @@
<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> Schema Queries</p>
<ul class="contains-task-list">
<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> SHOW TABLES</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> SHOW COLUMNS</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox">
information_schema.{tables, columns}</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> SHOW COLUMNS FROM
<table/view></p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> SHOW CREATE TABLE
<view></p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox">
information_schema.{tables, columns, views}</p></li>
<li class="task-list-item"><p><input class="task-list-item-checkbox"
disabled="disabled" type="checkbox"> information_schema other views</p></li>
</ul>
</li>
@@ -474,24 +467,10 @@
<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> CSV</p></li>
<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> Parquet primitive
types</p></li>
<li class="task-list-item"><p><input class="task-list-item-checkbox"
disabled="disabled" type="checkbox"> Parquet nested types</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> JSON</p></li>
+<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> Avro</p></li>
</ul>
</section>
-<section id="extensibility">
-<h2>Extensibility<a class="headerlink" href="#extensibility" title="Permalink
to this heading">¶</a></h2>
-<p>DataFusion is designed to be extensible at all points. To that end, you can
provide your own custom:</p>
-<ul class="contains-task-list simple">
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined Functions
(UDFs)</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined Aggregate
Functions (UDAFs)</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined Table
Source (<code class="docutils literal notranslate"><span
class="pre">TableProvider</span></code>) for tables</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined <code
class="docutils literal notranslate"><span class="pre">Optimizer</span></code>
passes (plan rewrites)</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined <code
class="docutils literal notranslate"><span
class="pre">LogicalPlan</span></code> nodes</p></li>
-<li class="task-list-item"><p><input class="task-list-item-checkbox"
checked="checked" disabled="disabled" type="checkbox"> User Defined <code
class="docutils literal notranslate"><span
class="pre">ExecutionPlan</span></code> nodes</p></li>
-</ul>
-</section>
-<section id="rust-version-compatibility">
-<h2>Rust Version Compatibility<a class="headerlink"
href="#rust-version-compatibility" title="Permalink to this heading">¶</a></h2>
-<p>This crate is tested with the latest stable version of Rust. We do not
currently test against other, older versions of the Rust compiler.</p>
-</section>
</section>
