Re: DataflowRunner | Cross-language

2020-05-26 Thread Sergei Sokolenko
small correction. While the new runner will be available with Beam 2.21,
the Cross-Language support will be available in 2.22.
There will be limitations in the initial set of connectors you can use with
Cross-Lang. But at least you will have something to test with, starting in
2.22

On Tue, May 26, 2020 at 11:23 AM Sergei Sokolenko  wrote:

> More info will be forthcoming after Beam 2.21 is out. There will be a docs
> page describing how it all works.
>
> On Thu, May 21, 2020 at 11:18 PM Paweł Urbanowicz <
> pawel.urbanow...@polidea.com> wrote:
>
>> Hello, community,
>>
>> I found information that Google is working on supporting Dataflow runner
>> for cross-language
>> (https://beam.apache.org/roadmap/connectors-multi-sdk/)
>>
>> Is there any more information about the expected release of this feature?
>>
>> Thanks
>>
>>
>>


Re: DataflowRunner | Cross-language

2020-05-26 Thread Sergei Sokolenko
More info will be forthcoming after Beam 2.21 is out. There will be a docs
page describing how it all works.

On Thu, May 21, 2020 at 11:18 PM Paweł Urbanowicz <
pawel.urbanow...@polidea.com> wrote:

> Hello, community,
>
> I found information that Google is working on supporting Dataflow runner
> for cross-language
> (https://beam.apache.org/roadmap/connectors-multi-sdk/)
>
> Is there any more information about the expected release of this feature?
>
> Thanks
>
>
>


Re: [ANNOUNCEMENT] Common Pipeline Patterns - new section in the documentation + contributions welcome

2019-06-07 Thread Sergei Sokolenko
and now the news in on the twitterwebs
https://twitter.com/datancoffee/status/1137160729386074113

On Fri, Jun 7, 2019 at 5:52 PM Reza Rokni  wrote:

> +1 on the pattern Tim!
>
> Please raise a Jira with the label pipeline-patterns, details are here:
>
> https://beam.apache.org/documentation/patterns/overview/#contributing-a-pattern
>
>
>
> On Sat, 8 Jun 2019 at 05:04, Tim Robertson 
> wrote:
>
>> This is great. Thanks Pablo and all
>>
>> I've seen several folk struggle with writing avro to dynamic locations
>> which I think might be a good addition. If you agree I'll offer a PR unless
>> someone gets there first - I have an example here:
>>
>> https://github.com/gbif/pipelines/blob/master/pipelines/export-gbif-hbase/src/main/java/org/gbif/pipelines/hbase/beam/ExportHBase.java#L81
>>
>>
>> On Fri, Jun 7, 2019 at 10:52 PM Pablo Estrada  wrote:
>>
>>> Hello everyone,
>>> A group of community members has been working on gathering and providing
>>> common pipeline patterns for pipelines in Beam. These are examples on how
>>> to perform certain operations, and useful ways of using Beam in your
>>> pipelines. Some of them relate to processing of files, use of side inputs,
>>> sate/timers, etc. Check them out[1].
>>>
>>> These initial patterns have been chosen based on evidence gathered from
>>> StackOverflow, and from talking to users of Beam.
>>>
>>> It would be great if this section could grow, and be useful to many Beam
>>> users. For that reason, we invite anyone to share patterns, and pipeline
>>> examples that they have used in the past. If you are interested in
>>> contributing, please submit a pull request, or get in touch with Cyrus
>>> Maden, Reza Rokni, Melissa Pashniak or myself.
>>>
>>> Thanks!
>>> Best
>>> -P.
>>>
>>> [1] https://beam.apache.org/documentation/patterns/overview/
>>>
>>
>
> --
>
> This email may be confidential and privileged. If you received this
> communication by mistake, please don't forward it to anyone else, please
> erase all copies and attachments, and please let me know that it has gone
> to the wrong person.
>
> The above terms reflect a potential business arrangement, are provided
> solely as a basis for further discussion, and are not intended to be and do
> not constitute a legally binding obligation. No legally binding obligations
> will be created, implied, or inferred until an agreement in final form is
> executed in writing by all parties involved.
>