This is an automated email from the ASF dual-hosted git repository.
sejer pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-wayang-website.git
The following commit(s) were added to refs/heads/main by this push:
new 38de7d43 Refine introduction and architecture descriptions (#101)
38de7d43 is described below
commit 38de7d43cc4661cfb5a4873bb2354b507ea85710
Author: Zoi Kaoudi <[email protected]>
AuthorDate: Tue Nov 25 14:17:39 2025 +0100
Refine introduction and architecture descriptions (#101)
Revised descriptions for clarity and consistency, emphasizing the unifying
aspects of Apache Wayang and its cross-platform capabilities.
---
docs/introduction/about.md | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/docs/introduction/about.md b/docs/introduction/about.md
index ba2de65d..1a3d1270 100644
--- a/docs/introduction/about.md
+++ b/docs/introduction/about.md
@@ -21,11 +21,17 @@ sidebar_position: 1
limitations under the License.
-->
-#### *A unified data processing framework that seamlessly integrates and
orchestrates multiple data platforms to deliver unparalleled performance and
flexibility.*
+#### *A unifying data processing framework that seamlessly integrates and
orchestrates multiple data processing systems to deliver performance and
flexibility.*
-Apache Wayang's three-layer architecture provides a strategic *abstraction*
between user applications and underlying data processing platforms, ensuring
seamless integration and optimization. The application layer encapsulates
application-specific logic, while the core layer acts as an intermediary,
translating application logic into a standardized intermediate representation
(WayangPlan). This standardized representation is then passed to the platform
layer, where it is optimized for ex [...]
+Apache Wayang's three-layer architecture provides a strategic *abstraction*
between user applications and underlying data processing platforms, ensuring
seamless integration among heterogeneous systems. The application layer
encapsulates application-specific logic, while the core layer acts as an
intermediary, translating application logic into an intermediate representation
(Wayang plan). The Wayang plan is then transformed into an execution plan in
the platform layer, where each operat [...]
-Designed with flexibility as a priority, Apache Wayang enables easy
*extensibility* to accommodate new operators and data platforms.
+One of Wayang’s key innovations is its *cross-platform optimizer*, which
automates data system selection and spares users from making complex platform
choices.
+This optimization process ensures that the resulted execution plan is tailored
to the specific strengths and capabilities of each data engine, maximizing
performance and efficiency.
+
+Apache Wayang's core strength lies in its *cross-platform task execution*,
enabling developers to seamlessly combine the strengths of various processing
engines, such as Spark, Flink, and Tensorflow, *in one pipeline*.
+Designed with flexibility as a priority, Apache Wayang enables easy
*extensibility* to accommodate new operators and data systems.
+The platform's extensibility and ease of use makes it a compelling choice for
data engineers and developers seeking a unifying and versatile data processing
solution.
+<br/>
### Architecture and Software stack
Apache Wayang's unique architecture, unlike traditional DBMSs, decouples the
physical planning and execution layers, empowering developers to express their
data processing logic in a platform-agnostic fashion. This separation of
concerns allows developers to focus on the algorithmic aspects of their
applications without being constrained by the intricacies of specific
processing platforms.
@@ -34,11 +40,8 @@ Apache Wayang's unique architecture, unlike traditional
DBMSs, decouples the phy
<img width="75%" alt="wayang stack" src="/img/architecture/wayang-stack.png"
/>
<br/><br/>
-At the bottom layers of the software stack, there are the different data
storage mediums and the supported data processing platforms. On top of these,
Wayang’s core consists of the following main components: the optimizer, the
executor, the monitor, and platform-specific drivers. Wayang currently supports
two main APIs: the Java one and the Scala one. A Python API is currently under
development. Besides using any of the supported languages, users can directly
input SQL queries via the SQ [...]
-
-<br/>
+At the bottom layers of the software stack, there are the different data
storage mediums and the supported data processing platforms. On top of these,
Wayang’s core consists of the following main components: the optimizer, the
executor, the monitor, and platform-specific drivers. Wayang currently supports
two main APIs: the Java one and the Scala one. A Python API is also supported
with limited operator coverage for the moment. Besides using any of the
supported languages, users can dire [...]
-Apache Wayang's core strength lies in its cross-platform task execution,
enabling developers to seamlessly leverage the strengths of various processing
engines, such as Hadoop, Spark, and Flink, without sacrificing performance or
flexibility. The platform's ease of use further enhances its appeal, making it
a compelling choice for data engineers and developers seeking a unified and
versatile data processing solution.
<br/>
Below you can see on the left, a Wayang plan representing the stochastic
gradient descent algorithm, which used in most deep learning tasks. On the
right, you can see how the optimizer decided to execute it. Orange nodes are
the operators that ran on Spark and green the operators executed as a single
Java process.
<br/>
