[
https://issues.apache.org/jira/browse/GRIFFIN-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chitral Verma closed GRIFFIN-340.
---------------------------------
Resolution: Won't Fix
The current implementation of Streaming Griffin is based on Spark 1.x and is
very old. Actually, I'm currently working on getting Spark 2.x structured
streaming support in Griffin.
I believe Flink support will be hard, as it will require a complete rewrite of
the measure module. However, structured streaming is quite mature now and
should solve most of your cases.
Closing this ticket, please refer to this ticket for further updates.
Regards.
> Flink in Griffin
> ----------------
>
> Key: GRIFFIN-340
> URL: https://issues.apache.org/jira/browse/GRIFFIN-340
> Project: Griffin
> Issue Type: Wish
> Reporter: XIAOYU YU
> Priority: Major
>
> We have been using Griffin for batch data quality measurement for some time.
> Recently, we are working on streaming data quality. We have investigated and
> tested Griffin, but it can not meet our needs.
> Therefore, we are developing a streaming data quality measurement tool based
> on Apache Flink and reusing Griffin DSL.
> So we have some thoughts:
> 1. Is it possible that the Griffin DSL batch jobs could also be executed by
> Flink? Because Flink works good on batch data as well. We will try to support
> batch data quality measurement (defined by Griffin DSL) with Flink.
> 2. Can we separate the computing engine layer from Griffin, as an optional
> plug-in? So that the Griffin DSL jobs can run on both Spark and Flink.
> Are the above thoughts feasible? We want to know the community's opinions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
