[
https://issues.apache.org/jira/browse/GRIFFIN-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17178179#comment-17178179
]
Chitral Verma commented on GRIFFIN-340:
---------------------------------------
[~xiaoyu.nova] The current implementation of Streaming Griffin is based on
Spark 1.x and is very old. Actually, I'm currently working on getting Spark 2.x
structured streaming support in Griffin.
I believe Flink support will be hard, as it will require a complete rewrite of
the measure module. However, structured streaming is quite mature now and
should solve most of your cases.
Closing this ticket, please refer to [this
ticket|https://issues.apache.org/jira/browse/GRIFFIN-303] for further updates.
Regards.
> Flink in Griffin
> ----------------
>
> Key: GRIFFIN-340
> URL: https://issues.apache.org/jira/browse/GRIFFIN-340
> Project: Griffin
> Issue Type: Wish
> Reporter: XIAOYU YU
> Priority: Major
>
> We have been using Griffin for batch data quality measurement for some time.
> Recently, we are working on streaming data quality. We have investigated and
> tested Griffin, but it can not meet our needs.
> Therefore, we are developing a streaming data quality measurement tool based
> on Apache Flink and reusing Griffin DSL.
> So we have some thoughts:
> 1. Is it possible that the Griffin DSL batch jobs could also be executed by
> Flink? Because Flink works good on batch data as well. We will try to support
> batch data quality measurement (defined by Griffin DSL) with Flink.
> 2. Can we separate the computing engine layer from Griffin, as an optional
> plug-in? So that the Griffin DSL jobs can run on both Spark and Flink.
> Are the above thoughts feasible? We want to know the community's opinions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
