Re: pipeline steps

2019-02-07 Thread Yi Pan
Shouldn't this apply to more generic scenario for any BeamIO? For example, I am using KafkaIO and wanted to get the topic and partition from which the message was received. Some IOContext associated with each data unit from BeamIO may be useful here? -Yi On Thu, Feb 7, 2019 at 6:29 AM Kenneth

Re: pipeline steps

2019-02-12 Thread Yi Pan
”* was not used before - > in this case it will return simple *KV*. > > In the same time, I agree that it would be useful to have some general way > to obtain meta information of records across all Beam IOs. > > On 7 Feb 2019, at 18:25, Yi Pan wrote: > > Shouldn't this apply to

Re: pipeline steps

2019-02-14 Thread Yi Pan
dAll transforms should be more flexible and we > should be able to adapt them to support returning file-names (and they'll > have the full power of Read transforms after SDF). > > Thanks, > Cham > > > On Tue, Feb 12, 2019 at 12:42 PM Yi Pan wrote: > >> The &quo

Re: Example project configuration (maven or gradle) for projects depending on BeamSQL sdk extensions

2019-01-31 Thread Yi Pan
https://issues.apache.org/jira/browse/BEAM-6558 to figure this out. > > Kenn > > On Wed, Jan 30, 2019 at 7:01 PM Yi Pan wrote: > >> Hi, all, >> >> Newbie here trying to figure out how to use published >> beam-sdks-java-extensions-sql-2.9.0 in my own project. &g

Example project configuration (maven or gradle) for projects depending on BeamSQL sdk extensions

2019-01-30 Thread Yi Pan
Hi, all, Newbie here trying to figure out how to use published beam-sdks-java-extensions-sql-2.9.0 in my own project. I tried to create a gradle project to use BeamSQL sdk libraries. Here is the build.gradle I have: {code} plugins { id 'java' } group 'com.mycompany.myproject' version

JDK11 support?

2019-04-09 Thread Yi Pan
Hi, everyone, I checked out the latest Beam code and seems that the document still says "JDK8". Is there any plan to officially support JDK11? Thanks a lot! -Yi

Re: JDK11 support?

2019-04-09 Thread Yi Pan
ul, e.g. automatic staging of pipeline jars >>> (https://issues.apache.org/jira/browse/BEAM-5495). >>> >>> I hope we can achieve proper Java 11 compatibility in the next releases. >>> Perhaps somebody else can give further insight into how much work is >

Re: JDK11 support?

2019-04-09 Thread Yi Pan
That said, there is still code that needs a major >> overhaul, e.g. automatic staging of pipeline jars >> (https://issues.apache.org/jira/browse/BEAM-5495). >> >> I hope we can achieve proper Java 11 compatibility in the next releases. >> Perhaps somebody else can give

Re: [ANNOUNCE] New committer: Ke Wu

2022-05-28 Thread Yi Pan
Congrats, Ke! -Yi On Sat, May 28, 2022 at 6:57 PM Robert Burke wrote: > Congratulations! > Another place that runs the Go SDK ;) > > On Fri, May 27, 2022, 3:49 PM Ahmet Altay wrote: > >> Hi all, >> >> Please join me and the rest of the Beam PMC in welcoming a new committer: >> Ke Wu (kw2542@)