For this kind of decisions, I'd write a short doc with pros and cons and
suggest an option. We can further discuss in the doc or dev list if needed.
If there's a significant disagreement we could even go for a vote in the
dev list but usually we do not get to that (and go by lazy consensus [1]).
Yeah, I don't think either finalized or documented (in the Website) the
previous iteration. This doc seems to contain details from the documents
shared in the previous iteration.
Thanks,
Cham
On Mon, Dec 12, 2022 at 6:49 PM Robert Burke wrote:
> I think ultimately: until the docs a clearly
I think ultimately: until the docs a clearly available on the Beam site
itself, it's not documentation. See also, design docs, previous emails, and
similar.
On Mon, Dec 12, 2022, 6:07 PM Andrew Pilloud via dev
wrote:
> I believe the previous iteration was here:
>
I believe the previous iteration was here:
https://lists.apache.org/thread/3o8glwkn70kqjrf6wm4dyf8bt27s52hk
The associated docs are:
https://s.apache.org/beam-io-api-standard-documentation
https://s.apache.org/beam-io-api-standard
This is missing all the relational stuff that was in those docs,
Hi,
"As for the pipeline update feature, we've long discussed
having "pick-your-implementation" transforms that specify
alternative, equivalent implementations."
Could someone point me to where this was discussed please? I seem to have
missed that whole topic. Is it like a dependency injection
Saving up all the breaking changes until a major release definitely
has its downsides (look at Python 3). The migration path is often as
important (if not more so) than the final destination.
As for this particular change, I would question how the benefit (it's
unclear what the exact benefit
Hello Everyone,
*Even if this is your first day learning Beam, please feel welcome to vote.*
*Please cast your single question answer on your preference* for Kubernetes
[1] versus Docker Compose [2] in local development of the Beam Playground
[3]. The form provides short and long versioned
I agree with Mortiz. To answer a few specifics in my own words:
- It is a perfectly sensible refactor, but as a counterpoint without
file-based IO the SDK isn't functional so it is also a reasonable design
point to have this included. There are other things in the core SDK that
are far less
Hi folks,
I'd like to solicit feedback on the notion of using
PubsubMessageWithAttributesAndMessageIdAndOrderingKeyCoder[1] as the
default coder for Pubsub messages instead of the current default of
PubsubMessageWithAttributesCoder.
Not long ago, support for reading and writing Pubsub messages
Thanks for writing this!
IIRC, the similar design doc was sent for review here a while ago. Is this just
an updated version and a new one?
—
Alexey
> On 11 Dec 2022, at 15:16, Herman Mak via dev wrote:
>
> Hello Everyone,
>
> *TLDR*
>
> Should we adopt a set of standards that Connector
Hi everyone. I'm having trouble making a performance test work for the
Debezium connector. This test reads the events from a PostgreSQL database
produced by a number of operations (inserts, deletes, updates) done for ~20
min.
When running in DirectRunner the pipeline reads the messages, stops, and
Hi Damon,
I fear the current release / versioning strategy of Beam doesn’t lend itself
well for such breaking changes. Alexey and I have spent quite some time
discussing how to proceed with the problematic Avro dependency in core (and
respectively AvroIO, of course).
Such changes essentially
This is your daily summary of Beam's current high priority issues that may need
attention.
See https://beam.apache.org/contribute/issue-priorities for the meaning and
expectations around issue priorities.
Unassigned P1 Issues:
https://github.com/apache/beam/issues/24389 [Failing Test]:
Thanks so much! Great to see this to be picked up again with some good progress.
/ Moritz
On 11.12.22, 15:17, "Herman Mak via dev" wrote:
Hello Everyone, *TLDR* Should we adopt a set of standards that Connector I/Os
should adhere to? Attached is a first version of a Beam I/O Standards
14 matches
Mail list logo
