Hi,
Yes, I saw that and already provided some info.
The RestIO I have works as follow:
- Producer Read (working)
pipeline.apply(RestIO.read().from("http://localhost:8080/rest";))
It provides a PCollection<String> where String is JSON or XML. It's also
possible to provide a data binding with withDataBinding() method.
It's basically a DoFn as it's not possible to do any splitting.
- Producer Write (working)
pipeline.apply()....apply(RestIO.write().to("http://localhost:8080/rest";))
Here, it's basically the same DoFn as producer using input PCollection as REST
request.
- Consumer Read (work in progress)
pipeline.apply(RestIO.read().bind("http://localhost:9090/foo";))
Here it's an unbounded source that bind a CXF connector (the initialSplit()
method defines where the connector will be bind).
I'm experiment this as it's more "ambitious" than the producer mode.
Regards
JB
On 04/12/2017 08:16 AM, Tang Jijun(上海_技术部_数据平台_唐觊隽) wrote:
Hi Stephen,
We need restIO module. I am looking forward to your pr about it. For restIO ,I
create a jira https://issues.apache.org/jira/browse/BEAM-1946.
-----邮件原件-----
发件人: Jean-Baptiste Onofré [mailto:[email protected]]
发送时间: 2017年4月12日 14:10
收件人: [email protected]
主题: Re: Public in-progress I/O Transform list
Hi Stephen,
Thanks !
I would add Socket IO and REST IO that I have on local branches (I'm waiting
to close some of my PRs before opening new ones). I also have (not fully ready
for
PR) CamelIO, IronMQIO, and others (I will update the website on the fly ;)).
MQTT IO is missing in the list. Gonna provide a website PR about that.
I also created ExecIO while ago but we didn't get a consensus about what we
should do with it.
Regards
JB
On 04/12/2017 02:01 AM, Stephen Sisk wrote:
Hi!
We occasionally get questions about whether or not an I/O is planned
to be added to Beam. I've added a list of known in-progress I/O
Transforms to the "Built-in Transforms" page (where in-progress is
defined as "has a JIRA issue"). The Built-In Transforms page is the
publicly visible list of I/O transforms, so it's a natural place to let users
know what's also in-progress.
You can find the current list here:
https://beam.apache.org/documentation/io/built-in/
I may have missed one or two I/Os (there's no good way to find them in
JIRA as far as I can tell), so if you're working on an I/O Transform
that you intend to contribute to beam that's not listed, feel free to send a PR
adding it to the list.
Thanks,
Stephen
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
