This is an automated email from the ASF dual-hosted git repository.
yamamuro pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 308a8fd [SPARK-31326][SQL][DOCS] Create Function docs structure for
SQL Reference
308a8fd is described below
commit 308a8fd3c67704b7fce0067a199707f46c6e6f1e
Author: Huaxin Gao <[email protected]>
AuthorDate: Fri Apr 3 14:36:03 2020 +0900
[SPARK-31326][SQL][DOCS] Create Function docs structure for SQL Reference
### What changes were proposed in this pull request?
Create Function docs structure for SQL Reference...
### Why are the changes needed?
so the Function docs can be added later, also want to get a consensus about
what to document for Functions in SQL Reference.
### Does this PR introduce any user-facing change?
Yes
<img width="1050" alt="Screen Shot 2020-04-02 at 12 09 20 AM"
src="https://user-images.githubusercontent.com/13592258/78220451-68b6e100-7476-11ea-9a21-733b41652785.png";>
<img width="1051" alt="Screen Shot 2020-04-02 at 12 09 44 AM"
src="https://user-images.githubusercontent.com/13592258/78220460-6ce2fe80-7476-11ea-887c-defefd55c19d.png";>
<img width="1051" alt="Screen Shot 2020-04-02 at 12 10 05 AM"
src="https://user-images.githubusercontent.com/13592258/78220463-6f455880-7476-11ea-81fc-fd4137db7c3f.png";>
### How was this patch tested?
Manually build and check
Closes #28099 from huaxingao/function.
Authored-by: Huaxin Gao <[email protected]>
Signed-off-by: Takeshi Yamamuro <[email protected]>
(cherry picked from commit 4e45c07f5dbc4b178c41449320b7405b20aa05e9)
Signed-off-by: Takeshi Yamamuro <[email protected]>
---
docs/_data/menu-sql.yaml | 21 +++++++++++++++++++++
docs/sql-ref-functions-builtin-aggregate.md | 10 +++++-----
...regate.md => sql-ref-functions-builtin-array.md} | 6 +++---
...te.md => sql-ref-functions-builtin-date-time.md} | 6 +++---
docs/sql-ref-functions-builtin.md | 17 +++++++++--------
...n-aggregate.md => sql-ref-functions-udf-hive.md} | 10 +++++-----
docs/sql-ref-functions-udf.md | 17 +++++++++--------
docs/sql-ref-functions.md | 13 +++++--------
8 files changed, 60 insertions(+), 40 deletions(-)
diff --git a/docs/_data/menu-sql.yaml b/docs/_data/menu-sql.yaml
index 6534c50..500895a 100644
--- a/docs/_data/menu-sql.yaml
+++ b/docs/_data/menu-sql.yaml
@@ -225,5 +225,26 @@
url: sql-ref-syntax-aux-resource-mgmt-list-file.html
- text: LIST JAR
url: sql-ref-syntax-aux-resource-mgmt-list-jar.html
+ - text: Functions
+ url: sql-ref-functions.html
+ subitems:
+ - text: Build-in Functions
+ url: sql-ref-functions-builtin.html
+ subitems:
+ - text: Build-in Aggregate Functions
+ url: sql-ref-functions-builtin-aggregate.html
+ - text: Build-in Array Functions
+ url: sql-ref-functions-builtin-array.html
+ - text: Build-in Date Time Functions
+ url: sql-ref-functions-builtin-date-time.html
+ - text: UDFs (User-Defined Functions)
+ url: sql-ref-functions-udf.html
+ subitems:
+ - text: Scalar UDFs (User-Defined Functions)
+ url: sql-ref-functions-udf-scalar.html
+ - text: UDAFs (User-Defined Aggregate Functions)
+ url: sql-ref-functions-udf-aggregate.html
+ - text: Integration with Hive UDFs/UDAFs/UDTFs
+ url: sql-ref-functions-udf-hive.html
- text: Datetime Pattern
url: sql-ref-datetime-pattern.html
diff --git a/docs/sql-ref-functions-builtin-aggregate.md
b/docs/sql-ref-functions-builtin-aggregate.md
index 3fcd782..d595436 100644
--- a/docs/sql-ref-functions-builtin-aggregate.md
+++ b/docs/sql-ref-functions-builtin-aggregate.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Builtin Aggregate Functions
-displayTitle: Builtin Aggregate Functions
+title: Built-in Aggregate Functions
+displayTitle: Built-in Aggregate Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -9,9 +9,9 @@ license: |
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -19,4 +19,4 @@ license: |
limitations under the License.
---
-**This page is under construction**
+Aggregate functions
\ No newline at end of file
diff --git a/docs/sql-ref-functions-builtin-aggregate.md
b/docs/sql-ref-functions-builtin-array.md
similarity index 87%
copy from docs/sql-ref-functions-builtin-aggregate.md
copy to docs/sql-ref-functions-builtin-array.md
index 3fcd782..599d10c 100644
--- a/docs/sql-ref-functions-builtin-aggregate.md
+++ b/docs/sql-ref-functions-builtin-array.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Builtin Aggregate Functions
-displayTitle: Builtin Aggregate Functions
+title: Built-in Array Functions
+displayTitle: Built-in Array Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -19,4 +19,4 @@ license: |
limitations under the License.
---
-**This page is under construction**
+Array Functions
diff --git a/docs/sql-ref-functions-builtin-aggregate.md
b/docs/sql-ref-functions-builtin-date-time.md
similarity index 87%
copy from docs/sql-ref-functions-builtin-aggregate.md
copy to docs/sql-ref-functions-builtin-date-time.md
index 3fcd782..1042132 100644
--- a/docs/sql-ref-functions-builtin-aggregate.md
+++ b/docs/sql-ref-functions-builtin-date-time.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Builtin Aggregate Functions
-displayTitle: Builtin Aggregate Functions
+title: Built-in Date and Time Functions
+displayTitle: Built-in Date and Time Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -19,4 +19,4 @@ license: |
limitations under the License.
---
-**This page is under construction**
+Date-Time Functions
diff --git a/docs/sql-ref-functions-builtin.md
b/docs/sql-ref-functions-builtin.md
index 6c57b0d6..48e5c0e 100644
--- a/docs/sql-ref-functions-builtin.md
+++ b/docs/sql-ref-functions-builtin.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Reference
-displayTitle: Reference
+title: Built-in Functions
+displayTitle: Built-in Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -9,9 +9,9 @@ license: |
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -19,7 +19,8 @@ license: |
limitations under the License.
---
-Spark SQL is Apache Spark's module for working with structured data.
-This guide is a reference for Structured Query Language (SQL) for Apache
-Spark. This document describes the SQL constructs supported by Spark in detail
-along with usage examples when applicable.
+Spark SQL defines built-in functions to use, a complete list of which can be
found [here](api/sql/). Among them, Spark SQL has several special categories of
built-in functions: [Aggregate
Functions](sql-ref-functions-builtin-aggregate.html) to operate on a group of
rows, [Array Functions](sql-ref-functions-builtin-array.html) to operate on
Array columns, and [Date and Time
Functions](sql-ref-functions-builtin-date-time.html) to operate on Date and
Time.
+
+* [Aggregate Functions](sql-ref-functions-builtin-aggregate.html)
+* [Array Functions](sql-ref-functions-builtin-array.html)
+* [Date and Time Functions](sql-ref-functions-builtin-date-time.html)
diff --git a/docs/sql-ref-functions-builtin-aggregate.md
b/docs/sql-ref-functions-udf-hive.md
similarity index 85%
copy from docs/sql-ref-functions-builtin-aggregate.md
copy to docs/sql-ref-functions-udf-hive.md
index 3fcd782..8698be7 100644
--- a/docs/sql-ref-functions-builtin-aggregate.md
+++ b/docs/sql-ref-functions-udf-hive.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Builtin Aggregate Functions
-displayTitle: Builtin Aggregate Functions
+title: Integration with Hive UDFs/UDAFs/UDTFs
+displayTitle: Integration with Hive UDFs/UDAFs/UDTFs
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -9,9 +9,9 @@ license: |
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -19,4 +19,4 @@ license: |
limitations under the License.
---
-**This page is under construction**
+Integration with Hive UDFs/UDAFs/UDTFs
\ No newline at end of file
diff --git a/docs/sql-ref-functions-udf.md b/docs/sql-ref-functions-udf.md
index 6c57b0d6..91c04f1 100644
--- a/docs/sql-ref-functions-udf.md
+++ b/docs/sql-ref-functions-udf.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Reference
-displayTitle: Reference
+title: UDFs (User-Defined Functions)
+displayTitle: UDFs (User-Defined Functions)
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -9,9 +9,9 @@ license: |
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -19,7 +19,8 @@ license: |
limitations under the License.
---
-Spark SQL is Apache Spark's module for working with structured data.
-This guide is a reference for Structured Query Language (SQL) for Apache
-Spark. This document describes the SQL constructs supported by Spark in detail
-along with usage examples when applicable.
+User-Defined Functions (UDFs) are a feature of Spark SQL that allows users to
define their own functions when the system's built-in functions are not enough
to perform the desired task. To use UDFs in Spark SQL, users must first define
the function, then register the function with Spark, and finally call the
registered function. The User-Defined Functions can act on a single row or act
on multiple rows at once. Spark SQL also supports integration of existing Hive
implementations of UDFs, [...]
+
+* [Scalar User-Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html)
+* [User-Defined Aggregate Functions
(UDAFs)](sql-ref-functions-udf-aggregate.html)
+* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)
diff --git a/docs/sql-ref-functions.md b/docs/sql-ref-functions.md
index 6c57b0d6..dabf90b 100644
--- a/docs/sql-ref-functions.md
+++ b/docs/sql-ref-functions.md
@@ -1,7 +1,7 @@
---
layout: global
-title: Reference
-displayTitle: Reference
+title: Functions
+displayTitle: Functions
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -9,9 +9,9 @@ license: |
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@@ -19,7 +19,4 @@ license: |
limitations under the License.
---
-Spark SQL is Apache Spark's module for working with structured data.
-This guide is a reference for Structured Query Language (SQL) for Apache
-Spark. This document describes the SQL constructs supported by Spark in detail
-along with usage examples when applicable.
+Spark SQL defines commonly used functions as built-in functions. It also
provides User-Defined Functions (UDFs) to allow users to define their own
functions when the system’s built-in functions are not enough to perform the
desired task.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/13592258/78220451-68b6e100-7476-11ea-9a21-733b41652785.png"](https://user-images.githubusercontent.com/13592258/78220451-68b6e100-7476-11ea-9a21-733b41652785.png"){kind=link}
![https://user-images.githubusercontent.com/13592258/78220460-6ce2fe80-7476-11ea-887c-defefd55c19d.png"](https://user-images.githubusercontent.com/13592258/78220460-6ce2fe80-7476-11ea-887c-defefd55c19d.png"){kind=link}
![https://user-images.githubusercontent.com/13592258/78220463-6f455880-7476-11ea-81fc-fd4137db7c3f.png"](https://user-images.githubusercontent.com/13592258/78220463-6f455880-7476-11ea-81fc-fd4137db7c3f.png"){kind=link}