Author: glauesppen
Date: Sun Sep 3 17:37:12 2023
New Revision: 63755
Log:
Staging of rc3 of Incubator Apache Wayang 0.7.1
Added:
dev/incubator/wayang/0.7.1/rc3/
dev/incubator/wayang/0.7.1/rc3/README.md
dev/incubator/wayang/0.7.1/rc3/RELEASE_NOTES
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip
(with props)
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.asc
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.sha512
Added: dev/incubator/wayang/0.7.1/rc3/README.md
==============================================================================
--- dev/incubator/wayang/0.7.1/rc3/README.md (added)
+++ dev/incubator/wayang/0.7.1/rc3/README.md Sun Sep 3 17:37:12 2023
@@ -0,0 +1,208 @@
+# Apache Wayang (incubating) <img align="right" width="128px"
src="https://wayang.apache.org/assets/img/logo/logo_400x160.png"; alt="Wayang
logo">
+
+## The first open-source cross-platform data processing system
+
+[](https://img.shields.io/maven-central/v/org.apache.wayang/wayang-core.svg)
+[](http://www.apache.org/licenses/LICENSE-2.0)
+[]()
+
+
+
+
+[](https://twitter.com/intent/tweet?text=Apache%20Wayang%20enables%20cross%20platform%20data%20processing,%20star%20it%20via:%20&url=https://github.com/apache/incubator-wayang&via=apachewayang&hashtags=dataprocessing,bigdata,analytics,hybridcloud,developers)
[](https://www.reddit.com/r/ApacheWayang/)
+## Table of contents
+ * [Description](#description)
+ * [Quick Guide for Running Wayang](#quick-guide-for-running-wayang)
+ * [Quick Guide for Developing with
Wayang](#quick-guide-for-developing-with-wayang)
+ * [Installing Wayang](#installing-wayang)
+ + [Requirements at Runtime](#requirements-at-runtime)
+ + [Validating the installation](#validating-the-installation)
+ * [Getting Started](#getting-started)
+ + [Prerequisites](#prerequisites)
+ + [Building](#building)
+ * [Running the tests](#running-the-tests)
+ * [Example Applications](#example-applications)
+ * [Built With](#built-with)
+ * [Contributing](#contributing)
+ * [Authors](#authors)
+ * [License](#license)
+
+## Description
+
+In contrast to traditional data processing systems that provide one dedicated
execution engine, Apache Wayang (incubating) can transparently and seamlessly
integrate multiple execution engines and use them to perform a single task. We
call this *cross-platform data processing*. In Wayang, users can specify any
data processing application using one of Wayang's APIs and then Wayang will
choose the data processing platform(s), e.g., Postgres or Apache Spark, that
best fits the application. Finally, Wayang will perform the execution, thereby
hiding the different platform-specific APIs and coordinating inter-platform
communication.
+
+Apache Wayang (incubating) aims at freeing data engineers and software
developers from the burden of learning all different data processing systems,
their APIs, strengths and weaknesses; the intricacies of coordinating and
integrating different processing platforms; and the inflexibility when trying a
fixed set of processing platforms. As of now, Wayang has built-in support for
the following processing platforms:
+- [Java
Streams](https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html)
+- [Apache Spark](https://spark.apache.org/)
+- [Apache Flink](https://flink.apache.org/)
+- [Apache Giraph](https://giraph.apache.org/)
+- [GraphChi](https://github.com/GraphChi/graphchi-java)
+- [Postgres](http://www.postgresql.org)
+- [SQLite](https://www.sqlite.org/)
+
+Apache Wayang (incubating) can be used via the following APIs:
+- Java native
+- Java scala-like
+- Scala
+- SQL (limited support of simple select-project queries for now)
+
+## Quick Guide for Running Wayang
+
+For a quick guide on how to run WordCount see [here](guides/tutorial.md).
+
+## Quick Guide for Developing with Wayang
+
+For a quick guide on how to use Wayang in your Java/Scala project see
[here](guides/develop-with-Wayang.md).
+
+## Installing Wayang
+
+You first have to build the binaries as shown [here](guides/tutorial.md).
+Once you have the binaries built, follow these steps to install Wayang:
+
+```shell
+tar -xvf wayang-0.6.1-snapshot.tar.gz
+cd wayang-0.6.1-SNAPSHOT
+```
+
+In linux
+```shell
+echo "export WAYANG_HOME=$(pwd)" >> ~/.bashrc
+echo "export PATH=${PATH}:${WAYANG_HOME}/bin" >> ~/.bashrc
+source ~/.bashrc
+```
+In MacOS
+```shell
+echo "export WAYANG_HOME=$(pwd)" >> ~/.zshrc
+echo "export PATH=${PATH}:${WAYANG_HOME}/bin" >> ~/.zshrc
+source ~/.zshrc
+```
+
+### Requirements at Runtime
+
+Since Apache Wayang (incubating) is not an execution engine itself but rather
manages the execution engines for you, it is important to have the necessary
requirements installed.
+
+- Apache Wayang supports Java versions 8 and above. However, the Wayang team
recommends using Java version 11. Donât forget to set the `JAVA_HOME`
environment variable.
+- You need to install Apache Spark version 3 or higher. Donât forget to set
the `SPARK_HOME` environment variable.
+- You need to install Apache Hadoop version 3 or higher. Donât forget to set
the `HADOOP_HOME` environment variable.
+
+### Validating the installation
+
+To execute your first application with Apache Wayang, you need to execute your
program with the 'wayang-submit' command:
+
+```shell
+bin/wayang-submit org.apache.wayang.apps.wordcount.Main java
file://$(pwd)/README.md
+```
+
+## Getting Started
+
+Wayang is available via Maven Central. To use it with Maven, include the
following code snippet into your POM file:
+```xml
+<dependency>
+ <groupId>org.apache.wayang</groupId>
+ <artifactId>wayang-***</artifactId>
+ <version>0.6.0</version>
+</dependency>
+```
+Note the `***`: Wayang ships with multiple modules that can be included in
your app, depending on how you want to use it:
+* `wayang-core`: provides core data structures and the optimizer (required)
+* `wayang-basic`: provides common operators and data types for your apps
(recommended)
+* `wayang-api-scala-java_2.12`: provides an easy-to-use Scala and Java API to
assemble Wayang plans (recommended)
+* `wayang-java`, `wayang-spark`, `wayang-graphchi`, `wayang-sqlite3`,
`wayang-postgres`: adapters for the various supported processing platforms
+* `wayang-profiler`: provides functionality to learn operator and UDF cost
functions from historical execution data
+
+> **NOTE:** The module `wayang-api-scala-java_2.12` is intended to be used
with Java 11 and Scala 2.12. If you have the Java 8 version, you need to use
the `wayang-api-scala-java_2.11` module.
+
+
+For the sake of version flexibility, you still have to include in the POM file
your Hadoop (`hadoop-hdfs` and `hadoop-common`) and Spark (`spark-core` and
`spark-graphx`) version of choice.
+
+In addition, you can obtain the most recent snapshot version of Wayang via
Sonatype's snapshot repository. Just include:
+```xml
+<repositories>
+ <repository>
+ <id>apache-snapshots</id>
+ <name>Apache Foundation Snapshot Repository</name>
+ <url>https://repository.apache.org/content/repositories/snapshots</url>
+ </repository>
+</repositories>
+```
+
+### Prerequisites
+Apache Wayang (incubating) is built with Java 11 and Scala 2.12. However, to
run Apache Wayang it is sufficient to have just Java 11 installed. Please also
consider that processing platforms employed by Wayang might have further
requirements.
+```
+Java 11
+[Scala 2.12]
+```
+
+> **NOTE:** Wayang also works with Java 8 and Scala 2.11. If you want to use
these versions, you will have to re-build Wayang (see below).
+
+> **NOTE:** In windows, you need to define the variable `HADOOP_HOME` with the
winutils.exe, an not official option to obtain [this
repository](https://github.com/steveloughran/winutils), or you can generate
your winutils.exe following the instructions in the repository. Also, you may
need to install
[msvcr100.dll](https://www.microsoft.com/en-us/download/details.aspx?id=26999)
+
+> **NOTE:** Make sure that the JAVA_HOME environment variable is set correctly
to either Java 8 or Java 11 as the prerequisite checker script currently
supports up to Java 11 and checks the latest version of Java if you have higher
version installed. In Linux, it is preferably to use the export JAVA_HOME
method inside the project folder. It is also recommended running './mvnw clean
install' before opening the project using IntelliJ.
+
+
+### Building
+
+If you need to rebuild Wayang, e.g., to use a different Scala version, you can
simply do so via Maven:
+
+1. Adapt the version variables (e.g., `spark.version`) in the main `pom.xml`
file.
+2. Build Wayang with the adapted versions.
+ ```shell
+ git clone https://github.com/apache/incubator-wayang.git
+ cd incubator-wayang
+ ./mvnw clean install -DskipTests
+ ```
+> **NOTE:** If you receive an error about not finding `MathExBaseVisitor`,
then the problem might be that you are trying to build from IntelliJ, without
Maven. MathExBaseVisitor is generated code, and a Maven build should generate
it automatically.
+
+> **NOTE:** In the current Maven setup, the version of scala is tied to the
Java version, you can compile the profile `scala-11` with Java 8 and profile
`scala-12` with Java 11.
+
+> **NOTE:** For compiling and testing the code it is required to have Hadoop
installed on your machine.
+
+> **NOTE:** the `standalone` profile to fix Hadoop and Spark versions, so
that Wayang apps do not explicitly need to declare the corresponding
dependencies.
+>
+> Also, note the `distro` profile, which assembles a binary Wayang
distribution.
+To activate these profiles, you need to specify them when running maven, i.e.,
+
+```shell
+./mvnw clean install -DskipTests -P<profile name>
+```
+
+## Running the tests
+In the incubator-wayang root folder run:
+```shell
+./mvnw test
+```
+
+## Example Applications
+You can see examples on how to start using Wayang
[here](guides/wayang-examples.md)
+
+## Built With
+
+* [Java
11](https://www.oracle.com/de/java/technologies/javase/jdk11-archive-downloads.html)
+* [Scala 2.12](https://www.scala-lang.org/download/2.12.0.html)
+* [Maven](https://maven.apache.org/)
+
+## Contributing
+Before submitting a PR, please take a look on how to contribute with Apache
Wayang contributing guidelines [here](CONTRIBUTING.md).
+
+## Authors
+The list of
[contributors](https://github.com/apache/incubator-wayang/graphs/contributors).
+
+## License
+All files in this repository are licensed under the Apache Software License 2.0
+
+Copyright 2020 - 2023 The Apache Software Foundation.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+## Acknowledgements
+The
[Logo](http://wayang.apache.org/assets/img/logo/Apache_Wayang/Apache_Wayang.pdf)
was donated by Brian Vera.
Added: dev/incubator/wayang/0.7.1/rc3/RELEASE_NOTES
==============================================================================
--- dev/incubator/wayang/0.7.1/rc3/RELEASE_NOTES (added)
+++ dev/incubator/wayang/0.7.1/rc3/RELEASE_NOTES Sun Sep 3 17:37:12 2023
@@ -0,0 +1,101 @@
+==============================================================
+Apache Wayang (incubating) 0.7.1
+==============================================================
+
+This is a patch/bugfix release of Apache Wayang (incubating).
+
+This release supports the following frameworks:
+- This patch version fixes a problem in the artifact gpg encrypting key.
+
+Previous updates that are included in this release:
+- Apache Flink v1.7.1
+- Apache Giraph v1.2.0-hadoop2
+- GraphChi v0.2.2 (only available with scala 11.x)
+- Java Streams (version depends on the java version)
+- JDBC-Template
+- Postgres v9.4.1208 (Implementation JDBC-Template)
+- Apache Spark v3.1.2 (scala 12.x) and v2.4.8 (scala 11.x)
+- SQLite3 v3.8.11.2 (implementation JDBC-Template)
+
+NOTE: depending on the scala version the list of the supported platforms
available
+could be different.
+
+
+New Features
+------------
+[WAYANG-34] Terasort
+[WAYANG-IDE] Recognition of Scala code in IDE
+[WAYANG-8] Inclusion of Python as platform of processing
+[WAYANG-211] JVM-platform inside of Python-API
+Grep Benchmark
+Test classes and operators for Flink
+Support for S3 files locations
+Initial version of the SQL API and suppport for joins
+
+
+Incompatible changes
+--------------------
+Removing [WAYANG-273] - WebUI Draft from release
+
+Miscellaneous changes
+---------------------
+
+Updates in documentation files:
+ - README.md
+ - NOTICE
+ - DISCLAIMER.md
+ - Usage instructions
+ - Add EPL and PD license
+
+CI updates:
+ - Protoc
+ - Skywalking-eyes check
+ - Deadlink check
+ - CodeQl Check
+ - UT check
+ - Auto Build And Install
+ - Create Issues from the TODO in the code
+ - Removed Travis, added Apache Calcite
+
+Update maven-wrapper to 3.1.0, maven to 3.8.4 for performance improvements
+Update .dlc.json
+Updates for monitoring the runtime metrics of underlying platform (Spark)
+[WAYANG-47] Properties Spark 2.11 updated with Spark version
+
+
+Known Issues
+------------
+Wayang Doc tests need further improvements.
+
+Bug Fixes
+---------
+Delete paranamer-2.3 from dependecies (PR#76)
+Code scanning alert (PR#80)
+Correction in DistinctOperator for Apache Flink (PR#226)
+Fix of broken links
+Fix on WayangJoinVisitor
+Concat strings vulnerability
+[WAYANG-assembly] correction in the folder structure (PR#259)
+[WAYANG-48] Fix errors when running WordCount
+
+
+New Contributors
+---------
+
+* @chrisdutz made their first contribution in
https://github.com/apache/incubator-wayang/pull/1
+* @2pk03 made their first contribution in
https://github.com/apache/incubator-wayang/pull/8
+* @ggevay made their first contribution in
https://github.com/apache/incubator-wayang/pull/27
+* @CalvinKirs made their first contribution in
https://github.com/apache/incubator-wayang/pull/44
+* @hboutemy made their first contribution in
https://github.com/apache/incubator-wayang/pull/77
+* @regaleo605 made their first contribution in
https://github.com/apache/incubator-wayang/pull/78
+* @zkaoudi made their first contribution in
https://github.com/apache/incubator-wayang/pull/230
+* @JorgeQuiane made their first contribution in
https://github.com/apache/incubator-wayang/pull/231
+* @kbeedkar made their first contribution in
https://github.com/apache/incubator-wayang/pull/284
+* @bgeng777 made their first contribution in
https://github.com/apache/incubator-wayang/pull/301
+* @damik3 made their first contribution in
https://github.com/apache/incubator-wayang/pull/304
+* @kmoltke made their first contribution in
https://github.com/apache/incubator-wayang/pull/309
+* @michellesackmann made their first contribution in
https://github.com/apache/incubator-wayang/pull/310
+* @AdeelAslamUnimore made their first contribution in
https://github.com/apache/incubator-wayang/pull/311
+* @ichbinrich made their first contribution in
https://github.com/apache/incubator-wayang/pull/315
+
+
Added:
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip
==============================================================================
Binary file - no diff available.
Propchange:
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.asc
==============================================================================
---
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.asc
(added)
+++
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.asc
Sun Sep 3 17:37:12 2023
@@ -0,0 +1,14 @@
+-----BEGIN PGP SIGNATURE-----
+
+iQGzBAABCgAdFiEE9nc6mPu9+AOWh3SZ6mGJAn+aO/EFAmT0wXYACgkQ6mGJAn+a
+O/G7hQv/UMf/Tv9ViVlSTwkyMBxv+4P6ZQX8xp9PcNG7zRwQaRkLTQ+jV0CWuW4O
+p0LzyyFz5TdiyjASZ8JSvzlwXZIrAyIJ4bpTF5i6WoWp4Vu3QOHSQ5auKOjn6yop
+0fnNxvwgcZH7FARbFISlIOvYa2677PPCmaKSdrKgYV/Ml/eW2i4wDSrnqb05U2z9
+P/+B8GroxUoElUckpnqjH0etgVNw4gr/cRWnH2I3QHKn/IcZm4fUwtZ9i53CKYmi
+aPA8ru1sFwvFN94TH8UkjeoI/8KQjtcuza5GTf7UWjzDUcQfwkWkO03LPUBc0Dpq
+I3l+8Bh5CPbyyRJ3vbuAU2BhXQBL2aTkTchydoPB33yhN0Lo0D8G0CeopMgcTFuE
+1xtPUpaZIU/VQh0jHA1PBhYOOseRl1p+uifflBc+BJTr4TrEvwGqbM7OTsPwme+W
+lZcKbk2GD/6T5FqZ6fi4xr7y2gCLTb03g9mhw9WIUNt8ds3JZDhnF/Gt4I0bUCnF
+S2Zw46I+
+=5XM1
+-----END PGP SIGNATURE-----
Added:
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.sha512
==============================================================================
---
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.sha512
(added)
+++
dev/incubator/wayang/0.7.1/rc3/apache-wayang-incubating-0.7.1-source-release.zip.sha512
Sun Sep 3 17:37:12 2023
@@ -0,0 +1 @@
+89660514ef2db7b28403028d31cfbdc22e147fef59d02afb8435c95f79c796d4ef0b0afc06d4c13ae29ee2913c161ee951c99cd64eb2b1b7e797e8ef02acf6db
\ No newline at end of file
![https://wayang.apache.org/assets/img/logo/logo_400x160.png"](https://wayang.apache.org/assets/img/logo/logo_400x160.png"){kind=link}
