[
https://issues.apache.org/jira/browse/HUDI-538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019265#comment-17019265
]
vinoyang edited comment on HUDI-538 at 1/20/20 7:14 AM:
--------------------------------------------------------
[~vinoth] OK, another thing we may need to consider. Based on our discussion,
we agreed on put {{hudi-utilities}} aside. However, for both Flink and Spark,
they follow {{source -> transform -> sink}} mode. Currently, the sources host
in {{hudi-utilities}} package and they are not Spark-free. So, it seems we also
need to consider it. WDYT?
was (Author: yanghua):
[~vinoth] OK, another thing we may need to consider. Based on our discussion,
we agreed on put {{hudi-utilities}} aside. However, for both Flink and Spark,
they observe {{source -> transform -> sink}} mode. Currently, the sources host
in {{hudi-utilities}} package and they are not Spark-free. So, it seems we also
need to consider it. WDYT?
> Restructuring hudi client module for multi engine support
> ---------------------------------------------------------
>
> Key: HUDI-538
> URL: https://issues.apache.org/jira/browse/HUDI-538
> Project: Apache Hudi (incubating)
> Issue Type: Wish
> Components: Code Cleanup
> Reporter: vinoyang
> Priority: Major
>
> Hudi is currently tightly coupled with the Spark framework. It caused the
> integration with other computing engine more difficult. We plan to decouple
> it with Spark. This umbrella issue used to track this work.
> Some thoughts wrote here:
> https://docs.google.com/document/d/1Q9w_4K6xzGbUrtTS0gAlzNYOmRXjzNUdbbe0q59PX9w/edit?usp=sharing
> The feature branch is {{restructure-hudi-client}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
