contributor permission for Beam Jira tickets

Hi, This is Hai from LinkedIn. I'm closely working with Xinyu on portable API for Samza runner. Can someone add me as a contributor for Beam's Jira issue tracker? I would like to create/assign tickets for my work. Thanks, Hai

Re: contributor permission for Beam Jira tickets

n Fri, Oct 12, 2018 at 2:16 PM Hai Lu wrote: > >> Hi, >> >> This is Hai from LinkedIn. I'm closely working with Xinyu on portable API >> for Samza runner. Can someone add me as a contributor for Beam's Jira issue >> tracker? I would like to create/assign tickets for my work. >> >> Thanks, >> Hai >> >

Performance of BeamFnData between Python and Java

Hi, This is Hai from LinkedIn. I'm currently working on Portable API for Samza Runner. I was able to make Python work with Samza container reading from Kafka. However, I'm seeing severe performance issue with my set up, achieving only ~200KB throughput between the Samza runner in the Java side

Re: Beam Samza Runner status update

n :). Again, this work >>>> is not possible without the great Beam portability framework, and the >>>> developers like Luke and Ahmet, just to name a few, behind it. The >>>> ReferenceRunner has been extremely useful to us to figure out what's needed >>>

flink portable runner usage

Hi Thomas, This is Hai who works on portable runner for Samza. I have a few minor question that I would like to get clarification on from you. We chatted briefly at last beam meetup and you mention your flink portable runner (Python) is going into production. So today are you using Beam Python

Re: Enable security for data channels in portability

Hi Lukasz and Ankur, Here is the PR that implements the idea: https://github.com/apache/beam/pull/8597 Would appreciate it if you could take a look. Thanks, Hai On Tue, Apr 30, 2019 at 9:13 AM Hai Lu wrote: > One thing to clarify is that we do not use docker. I don't have too m

Re: Enable security for data channels in portability

tainer/process/external/...) makes sense. > > How would the SDK and runner support storing/sharing this secret? (For > example, in the docker container, how would the secret get there?) > > On Mon, Apr 29, 2019 at 9:23 AM Hai Lu wrote: > >> Hi Lukasz and Ankur, >> &g

Re: Enable security for data channels in portability

;> around channel/call[2] credentials? >>> >>> 1: >>> https://github.com/apache/beam/blob/476e17ed6badd4d5c06c4caf8a824805f40a8e7a/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L139 >>> 2: https://grpc.io/docs/guides/auth/ >>> >>&

Enable security for data channels in portability

leveraging those codes? I don't see on the Java side any work is done, though. Thanks, Hai Lu

IllegalStateException: TimestampCombiner moved element from to earlier time in Python

Hi, This is Hai from LinkedIn. I'm looking into a bug I found internally when using Beam portable API (Python) on our own Samza runner. The pipeline looks something like this: (p | 'read' >> ReadFromKafka(cluster="tracking", topic="PageViewEvent") | 'transform' >> beam.Map(lambda

Re: IllegalStateException: TimestampCombiner moved element from to earlier time in Python

I did a simple fix for this issue here: https://github.com/apache/beam/pull/9364 Tested locally and it fixes the problem. Can someone help take a look? Thanks, Hai On Thu, Aug 15, 2019 at 9:16 AM Hai Lu wrote: > Hi, > > This is Hai from LinkedIn. > > I'm looking into a bug I f

Embedding expansion service for cross language in the runner

Hi, We're looking into leveraging the cross language pipeline feature in our Beam pipelines on Samza runner. While the feature seems to work well, the PTransform expansion as a standalone service isn't very convenient. Particularly that the Python pipeline needs to specify the address of the

Re: Embedding expansion service for cross language in the runner

t; > > > Thomas > > > > On Mon, Nov 4, 2019 at 12:03 PM Robert Bradshaw > wrote: > >> > >> On Mon, Nov 4, 2019 at 11:54 AM Chamikara Jayalath < > chamik...@google.com> wrote: > >> > > >> > On Mon, Nov 4, 2019 at 11:01 A

Re: Interactive Beam Example Failing [BEAM-8451]

Hi Robert, We're trying out iBeam at LinkedIn for Python. As Igor mentioned, there seems to be some inconsistency in the behavior of interactive beam. We can suggest some fixes from our end but we would need some support from the community. Also, is there a plan to support iBeam for streaming