Lakehouse Formats with IO/Integration --> Hudi? Iceberg?

2023-11-06 Thread Austin Bennett
Beam Devs, I was looking through GH Issue and online more generally and hadn't seen much... Has anyone written any Beam IO or other integration for writing to [ or reading from ] either Hudi or Iceberg? Any experience that can be shared [ on list, else feel free to message me off list and I'll

Re: Embeddings generation in MLTransform

2023-11-06 Thread Anand Inguva via dev
Hi all, After the initial email, I went ahead and added a few more things as per the comments on the doc . Please take a look and let me know what you think. Thanks, Anand On

[PR] Publish docs for 2.52.0 release [beam-site]

2023-11-06 Thread via GitHub
damccorm opened a new pull request, #652: URL: https://github.com/apache/beam-site/pull/652 Content generated from https://github.com/apache/beam/tree/v2.52.0-RC1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Beam High Priority Issue Report (49)

2023-11-06 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/29214 [Failing Test]:

Re: [PR] Publish docs for 2.52.0 release [beam-site]

2023-11-06 Thread via GitHub
damccorm closed pull request #651: Publish docs for 2.52.0 release URL: https://github.com/apache/beam-site/pull/651 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: Beam 2.52.0 Release

2023-11-06 Thread Danny McCormick via dev
Update: all issues mentioned in my last email are resolved and I was able to get all artifacts for RC published except for the PyPi artifacts (which I was about to do). However, during my validation before sending out the voting email I discovered a breaking change in the Datastore IO caused by