Re: SqlTransform translation deficiencies

2022-12-15 Thread Moritz Mack
Hi all, Just pumping this up again. Would anyone familiar with Beam SQL be able to have a look at this potential bug in Beam SQL. That help would be much appreciated! Thanks so much, Moritz On 23.11.22, 09:58, "Moritz Mack" wrote: Hi all, Not sure who’s best to ping. I spend some time

Re: [Proposal] Adopt a Beam I/O Standard

2022-12-15 Thread Herman Mak via dev
Hey all, Firstly apologies for the confusion. The scope of this effort is to *finalize and have this added to the Beam public documentation* to be used as a PR reference once we have resolved the comments. YES this document is a continuation of the below docs with some additional components such

Re: A Declarative API for Apache Beam

2022-12-15 Thread Ahmet Altay via dev
+1 to both of these proposals. In the past 12 months I have heard of at least 3 YAML implementations built on top of Beam in large production systems. Unfortunately, none of those were open sourced. Having these out of the box would be great, and it will clearly have used demand. Thank you all!

Re: A Declarative API for Apache Beam

2022-12-15 Thread Robert Bradshaw via dev
On Thu, Dec 15, 2022 at 3:37 AM Steven van Rossum wrote: > > This is great! I developed a similar template a year or two ago as a > reference for a customer to speed up their development process and > unsurprisingly it did speed up their development. > Here's an example of the config layout I

Beam Dependency Check Report (2022-12-15)

2022-12-15 Thread Apache Jenkins Server
<<< text/html; charset=UTF-8: Unrecognized >>>

Re: [Proposal] Adopt a Beam I/O Standard

2022-12-15 Thread Chamikara Jayalath via dev
On Thu, Dec 15, 2022, 8:33 AM Alexey Romanenko wrote: > Cham, do you remember what was a reason to not finalise that doc? > I think this is a continuation of those docs (so we are trying to finalize) but probably Herman can explain better. > Personally, I find having such standards very

Re: [Proposal] Adopt a Beam I/O Standard

2022-12-15 Thread Alexey Romanenko
Cham, do you remember what was a reason to not finalise that doc? Personally, I find having such standards very useful (if they are flexible during a time, of course), especially for new developers and PR reviewers, and it’d be great to finally have such doc as a part of contribution guide. —

Re: [Proposal] | Move FileIO and TextIO from :sdks:java:core to :sdks:java:io:file

2022-12-15 Thread Cristian Constantinescu
Counter argument to the "in one box" thing. I would like to point out that "having things in one box" is not a reason to have the code residing in the same module/project. What the user sees and how the code is structured are two very different things in my opinion. Beam can certainly have

Re: A Declarative API for Apache Beam

2022-12-15 Thread Steven van Rossum via dev
This is great! I developed a similar template a year or two ago as a reference for a customer to speed up their development process and unsurprisingly it did speed up their development. Here's an example of the config layout I came up with at the time: options: runner: DirectRunner pipeline: #

Beam High Priority Issue Report (37)

2022-12-15 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/24655 [Bug]: Pipeline