Hi all,
My name is Dmytro, I'm a Google engineer working on making interactive Beam
more useful for Google's engineers (which should also hopefully help to use
Dataflow runner interactively). Can someone please add me to Jira contributors,
so that I could assign some issues to myself? My Jira
On Mon, Jan 11, 2021 at 11:00 AM Robert Bradshaw
wrote:
> On Mon, Jan 11, 2021 at 10:38 AM Brian Hulette
> wrote:
>
>> I spoke with Gris and Agnieszka about this on Friday. I should probably
>> fill in the background a bit.
>>
>> The strategy we've adopted ro review the new designs so far is
Hi,
I proposed to make runner-issue checkpoint frequency configurable for a
pipeline author here:
https://docs.google.com/document/d/18jNLtTyyApx0N2ytp1ytOMmUPLouj2h08N3-4SyWGgQ/edit?usp=sharing.
I believe it will also be helpful for the performance issue. Please feel
free to drop any comments
Never mind - I think I found it.
On Tue, Jan 12, 2021 at 12:31 PM Reuven Lax wrote:
> If I try and run something from Java examples, e.g. the below:
>
> ./gradlew integrationTest -p examples/java --tests
> org.apache.beam.examples.cookbook.BigQueryTornadoes ...
>
>
>
> I now get gradle errors
Hi all,
I'm currently in the process of adding some metrics to an existing pipeline
that runs on Google Dataproc via Spark and I'm trying to determine how to
access these metrics and eventually expose them to Stackdriver (to be used
downstream in Grafana dashboards).
The metrics themselves
Yes, a PTansform can have no sub-transforms, as long as it only returns its
inputs. Updating the docs would be a good idea.
On Tue, Jan 12, 2021 at 1:04 PM Brian Hulette wrote:
> A recent bug with SqlTransform on Dataflow Runner V2 [1] revealed an
> interesting ambiguity in the Beam model: it's
I see. Yes, I meant that things like (or some level) do not appear in
the right nav. I see, if content is the next round I will save more
comments. I think that once we have a style, including what levels are
shown where, then it will be time to fix content issues.
Kenn
On Mon, Dec 21, 2020 at
FYI Emily fixed the bug with unintended license pulling. So bugs like the
one Reuven originally reported should be bypassed by default.
On Fri, Jan 8, 2021 at 5:01 PM Kyle Weaver wrote:
> > $0.02 can we make it a separate target rather than an existing target
> with flags?
>
> Something like
A recent bug with SqlTransform on Dataflow Runner V2 [1] revealed an
interesting ambiguity in the Beam model: it's not clear if a composite
transform is allowed to have zero sub-transforms [2]. This may sound like
an academic concern, but it can happen if a PTransform returns its own
input, making
If I try and run something from Java examples, e.g. the below:
./gradlew integrationTest -p examples/java --tests
org.apache.beam.examples.cookbook.BigQueryTornadoes ...
I now get gradle errors that look like this. Does anyone know if something
changed?
2021-01-12T12:28:51.428-0800 [ERROR]
Hi Alexey,
The short answer is, I hope the code in my sample project makes its way
back into Beam, but it may take some time. Hopefully until then people who
are in a similar position have a demo of a workaround they can work with.
The longer answer is:
The history of this is a bit lengthy, let
Hi Kenn,
thank you for your answers!
We'll update the Capability Matrix UI according to your design related
suggestions. It will be implemented in a way that allows you to rather
easily change its text content, remove/add columns and rows etc. :)
Kind regards,
Agnieszka
On Wed, Jan 6, 2021 at
Hi Cristian,
Great!
Would you be interested to add this functionality into original
ConfluentSchemaRegistryDeserializerProvider (along with Json support maybe) [1]
?
Though, it seems that it will require an update of Confluent deps to at least
version 5.5 [2] and it can be blocked by Beam
13 matches
Mail list logo