Flink is not really well suited for interactive / adhoc processing.
What could work is to use some local tool to identify the transformation
rules and apply them with Flink to a large data set.
But that's probably not what you are looking for, right?

Best, Fabian

2017-06-15 3:11 GMT+02:00 qi cui <cuiqi1...@gmail.com>:

> Hi Andrew,
> That will be great if you can come up with something to show the idea.
> There are lots of wiki pages on the github you can refer to(including the
> server side architecture and client side architecture). The unique feature
> of the OpenRefine is its ability to have the user to interact with the
> system and do the point-and-click to wangle the data set. But this is also
> I think hardest part to migrate/refactoring to another system like flink
> and spark. Or you can say OpenRefine deals with finite data set but
> Flink/Spark deal with infinite data set with high velocity and variation.
> They have some intersection in between of course. While we adopt a new
> framework like Flink, we may have to consider what to give up and what
> should be kept.
>
> Jacky
>
> On Wed, Jun 14, 2017 at 2:25 PM, Andrew Psaltis <psaltis.and...@gmail.com>
> wrote:
>
> > Thad,
> > Based on your description that OpenRefine uses similar techniques as
> > Zeeplin then I *think* the reading and writing will work.
> >
> > The Undo/Redo I am fuzzy on as.
> >
> > I will try over the next couple of days and see if I can make something
> > like this work (at lest a trivial use case). Personally I think it would
> be
> > cool to allow business users to wrangle data with OpenRefine with the
> power
> > of Flink behind it.
> >
> >
> >
> > On Wed, Jun 14, 2017 at 7:53 PM, Thad Guidry <thadgui...@gmail.com>
> wrote:
> >
> >> Andrew,
> >>
> >> So you idea is that Flink could be used as a storage abstraction layer
> for
> >> OpenRefine ?  Where OpenRefine would use TableSources for reading and
> >> TableSinks for writing ?
> >> And would that still work with our concept of Undo/Redo in OpenRefine to
> >> use Flink's Savepoints in concert with TableSources and TableSinks ?
> That
> >> last part is where I am reading Flink docs now and still seeing a lot of
> >> fuzzyness, which worries me.
> >>
> >> -Thad
> >> +ThadGuidry <https://www.google.com/+ThadGuidry>
> >>
> >> >
> >> >
> >>
> >
> >
> >
> > --
> > Thanks,
> > Andrew
> >
> > Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> > <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> > twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
> >
>

Reply via email to