This is an automated email from the ASF dual-hosted git repository.
eyal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafu.git
The following commit(s) were added to refs/heads/main by this push:
new 98d7508 Update site for 1.7.0 release
98d7508 is described below
commit 98d7508aa5e62483745375735186173d79c309bb
Author: Eyal Allweil <e...@apache.org>
AuthorDate: Tue Jan 24 15:49:54 2023 +0200
Update site for 1.7.0 release
---
README.md | 2 +-
changes.md | 1 -
.../blog/2023-01-25-datafu-1-7-0-released.markdown | 49 ++++++++++++++++++++++
.../docs/datafu/getting-started.html.markdown.erb | 2 +-
site/source/docs/download.html.markdown.erb | 17 ++++----
.../docs/spark/getting-started.html.markdown.erb | 2 +-
site/source/docs/spark/guide.html.markdown.erb | 4 +-
site/source/layouts/_docs_nav.erb | 2 +-
site/source/layouts/_footer.erb | 4 +-
9 files changed, 66 insertions(+), 17 deletions(-)
diff --git a/README.md b/README.md
index 4dffd77..eea8ab8 100644
--- a/README.md
+++ b/README.md
@@ -18,7 +18,7 @@
[](https://www.apache.org/licenses/LICENSE-2.0)
[](https://issues.apache.org/jira/projects/DATAFU/)
-[](http://search.maven.org/#search|gav|1|g:"org.apache.datafu")
+[](http://search.maven.org/#search|gav|1|g:"org.apache.datafu")
[](https://github.com/apache/datafu/actions/workflows/tests.yml)
diff --git a/changes.md b/changes.md
index ec33cc9..4a9fb86 100644
--- a/changes.md
+++ b/changes.md
@@ -8,7 +8,6 @@ Additions
Improvements
* Upgrade Log4j version (DATAFU-162)
-* Make dedupWithCombiner support multiple columns (DATAFU-171)
* Added count filtering option to broadcastJoinSkewed
Fixes
diff --git a/site/source/blog/2023-01-25-datafu-1-7-0-released.markdown
b/site/source/blog/2023-01-25-datafu-1-7-0-released.markdown
new file mode 100644
index 0000000..51313c6
--- /dev/null
+++ b/site/source/blog/2023-01-25-datafu-1-7-0-released.markdown
@@ -0,0 +1,49 @@
+---
+title: Apache DataFu-Spark 1.7.0 Released
+author: Eyal Allweil
+license: >
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+---
+
+I'd like to announce the release of Apache DataFu-Spark 1.7.0. Many thanks to
new contributors Arpit Bhardwaj, Ben Rahamim and Shaked Aharon!
+
+Additions
+
+* Add collectLimitedList and dedupRandomN methods (DATAFU-165)
+* Improve broadcastJoinSkewed function performance and allow all join types
(DATAFU-170)
+
+Improvements
+
+* Upgrade Log4j version (DATAFU-162)
+* Added count filtering option to broadcastJoinSkewed
+
+Fixes
+
+* explodeArray method not exposed in Python (DATAFU-163)
+
+Breaking changes
+
+* Spark 2.1.x no longer supported
+
+The source release can be obtained from:
+
+http://www.apache.org/dyn/closer.cgi/datafu/apache-datafu-1.7.0/
+
+Artifacts for DataFu are published in Apache's Maven Repository:
+
+https://repository.apache.org/content/groups/public/org/apache/datafu/
+
+Please visit the [Download](/docs/download.html) page for instructions on
building from source or retrieving the artifacts in your build system.
diff --git a/site/source/docs/datafu/getting-started.html.markdown.erb
b/site/source/docs/datafu/getting-started.html.markdown.erb
index 915edd7..5a4ca61 100644
--- a/site/source/docs/datafu/getting-started.html.markdown.erb
+++ b/site/source/docs/datafu/getting-started.html.markdown.erb
@@ -122,4 +122,4 @@ This produces the expected output:
## Next Steps
-Check out the [Guide](/docs/datafu/guide.html) for more information on what
you can do with DataFu Pig.
\ No newline at end of file
+Check out the [Guide](/docs/datafu/guide.html) for more information on what
you can do with DataFu Pig.
diff --git a/site/source/docs/download.html.markdown.erb
b/site/source/docs/download.html.markdown.erb
index d65db6f..455ca07 100644
--- a/site/source/docs/download.html.markdown.erb
+++ b/site/source/docs/download.html.markdown.erb
@@ -1,7 +1,7 @@
---
title: Download - Apache DataFu
section_name: Getting Started
-version: 1.6.1
+version: 1.7.0
license: >
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -21,7 +21,7 @@ license: >
# Download
-Apache DataFu is available for download as a source release and as compiled
artifacts stored in a Maven repository.
+Apache DataFu is available for download as a source release and as compiled
artifacts stored in a Maven repository. Please note that the latest version of
datafu-pig and datafu-hourglass is not the same as the latest version of
datafu-spark.
## Source Releases
@@ -31,7 +31,7 @@ The latest source release can be found here:
Previous releases:
-* <%= archived_source_release_link("1.6.0") %>
+* <%= archived_source_release_link("1.6.1") %>
### Validation
@@ -84,21 +84,22 @@ Assuming your local maven repository is at `~/.m2`, you
should see the DataFu ar
The latest release can be found in [Apache's Maven Repository for
DataFu](https://repository.apache.org/content/groups/public/org/apache/datafu):
* [datafu-spark_2.11-<%= current_page.data.version
%>](https://repository.apache.org/content/groups/public/org/apache/datafu/datafu-spark_2.11/<%=
current_page.data.version %>/)
-* [datafu-pig-<%= current_page.data.version
%>](https://repository.apache.org/content/groups/public/org/apache/datafu/datafu-pig/<%=
current_page.data.version %>/)
-* [datafu-hourglass-<%= current_page.data.version
%>](https://repository.apache.org/content/groups/public/org/apache/datafu/datafu-hourglass/<%=
current_page.data.version %>/)
+* [datafu-spark_2.12-<%= current_page.data.version
%>](https://repository.apache.org/content/groups/public/org/apache/datafu/datafu-spark_2.12/<%=
current_page.data.version %>/)
+*
[datafu-pig-1.6.1](https://repository.apache.org/content/groups/public/org/apache/datafu/datafu-pig/1.6.1/)
+*
[datafu-hourglass-1.6.1](https://repository.apache.org/content/groups/public/org/apache/datafu/datafu-hourglass/1.6.1/)
You can also use a dependency management system to download the DataFu
artifacts and all their dependencies. Some examples appear below.
SBT:
```scala
-libraryDependencies += "org.apache.datafu" %% "datafu-spark" % "1.6.1"
intransitive
+libraryDependencies += "org.apache.datafu" %% "datafu-spark" % "<%=
current_page.data.version %>" intransitive
```
Gradle:
```groovy
-compile "org.apache.datafu:datafu-pig:<%= current_page.data.version %>"
+compile "org.apache.datafu:datafu-spark:<%= current_page.data.version %>"
```
Maven:
@@ -106,7 +107,7 @@ Maven:
```xml
<dependency>
<groupId>org.apache.datafu</groupId>
- <artifactId>datafu-pig</artifactId>
+ <artifactId>datafu-spark</artifactId>
<version><%= current_page.data.version %></version>
</dependency>
```
diff --git a/site/source/docs/spark/getting-started.html.markdown.erb
b/site/source/docs/spark/getting-started.html.markdown.erb
index 31f3b86..47f70af 100644
--- a/site/source/docs/spark/getting-started.html.markdown.erb
+++ b/site/source/docs/spark/getting-started.html.markdown.erb
@@ -1,6 +1,6 @@
---
title: Apache DataFu Spark - Getting Started
-version: 1.6.1
+version: 1.7.0
section_name: Getting Started
license: >
Licensed to the Apache Software Foundation (ASF) under one or more
diff --git a/site/source/docs/spark/guide.html.markdown.erb
b/site/source/docs/spark/guide.html.markdown.erb
index 03a25fd..8cc593c 100644
--- a/site/source/docs/spark/guide.html.markdown.erb
+++ b/site/source/docs/spark/guide.html.markdown.erb
@@ -1,6 +1,6 @@
---
title: Guide - Apache DataFu Spark
-version: 1.6.1
+version: 1.7.0
section_name: Apache DataFu Spark
license: >
Licensed to the Apache Software Foundation (ASF) under one or more
@@ -26,7 +26,7 @@ It has a number of useful functions available. This guide
will provide examples
## Spark Compatibility
-The current version of DataFu has been tested against Spark versions 2.1.x -
2.4.x, in Scala 2.10, 2.11 and 2.12 (where applicable). The JAR for Scala 2.11
has been published to the [Apache Maven
Repository](https://repository.apache.org/content/groups/public/org/apache/datafu/).
Other versions can be built by [downloading the source](/docs/download.html)
and following the build instructions.
+The current version of DataFu has been tested against Spark versions 2.2.x -
2.4.x, in Scala 2.11 and 2.12 (where applicable). The jars have been published
to the [Apache Maven
Repository](https://repository.apache.org/content/groups/public/org/apache/datafu/).
Other versions can be built by [downloading the source](/docs/download.html)
and following the build instructions.
## Calling DataFu Spark functions from PySpark
diff --git a/site/source/layouts/_docs_nav.erb
b/site/source/layouts/_docs_nav.erb
index 4aba9a5..8514750 100644
--- a/site/source/layouts/_docs_nav.erb
+++ b/site/source/layouts/_docs_nav.erb
@@ -29,7 +29,7 @@
<ul class="nav nav-pills nav-stacked">
<li><a href="/docs/spark/guide.html">Guide</a></li>
- <li><a href="https://datafu.apache.org/docs/spark/1.6.1/";>Scaladocs</a></li>
+ <li><a href="https://datafu.apache.org/docs/spark/1.7.0/";>Scaladocs</a></li>
</ul>
<h4>DataFu Pig Docs</h4>
diff --git a/site/source/layouts/_footer.erb b/site/source/layouts/_footer.erb
index b8f5bed..bb3d816 100644
--- a/site/source/layouts/_footer.erb
+++ b/site/source/layouts/_footer.erb
@@ -24,7 +24,7 @@
</div>
<div class="copyright">
- Copyright © 2011-2021 The Apache Software Foundation, Licensed under
the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache License,
Version 2.0</a>.<br>
+ Copyright © 2011-2023 The Apache Software Foundation, Licensed under
the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache License,
Version 2.0</a>.<br>
Apache DataFu, DataFu, Apache Pig, Apache Hadoop, Hadoop, Apache, and the
Apache feather logo are either registered trademarks or trademarks of the <a
href="http://www.apache.org/";>Apache Software Foundation</a> in the United
States and other countries.
</div>
-</div>
\ No newline at end of file
+</div>
