Thank you so much, Ahmed. I look forward to reading your helpful comments!
On Thu, Nov 17, 2022 at 7:36 AM Ahmed Abualsaud via dev <dev@beam.apache.org> wrote: > Thanks for drafting this Damon! I left some comments on the doc. It's > really cool that users can go to one source with a specified file format > (json, avro, xml, csv, parquet) and retrieve the relevant file writing > PTransform. I also like how the same configuration can be re-used for > different file formats. Looking forward to seeing its implementation :) > > Ahmed > > On Mon, Nov 14, 2022 at 11:53 AM Damon Douglas <douglas.da...@gmail.com> > wrote: > >> Hello Everyone, >> >> I hope you are doing well. The following design document proposes, via a >> single configuration, a producer of a Beam File writing transform >> supporting multiple formats. >> bit.ly/fileioschematransformwriteprovider >> >> For those new to Beam and Schema, I've added a final section of >> suggested pre-requisite reading. It's important that everyone can >> participate in the conversation at any level of experience, even if this is >> the first day learning Beam. *Please feel invited to let me know >> anything that isn't clear so this document can strive to include everyone.* >> >> *My personal thoughts on the proposal's value* >> >> I've witnessed many smart people and teams argue and divide over the >> subject of a programming language. Beam multi-language support allows us >> to join transforms written in various languages, currently Java, Python, >> Go, and experimentally TypeScript into a single unified pipeline. It's >> Beam's schema and processing of these objects, called Rows, that allow this >> unification possible. The aforementioned proposal continues this vision >> for producing file and object system sinks via a single language agnostic >> configuration and supporting provider. >> >> Ada Lovelace dreamed of a machine that processed objects instead of just >> numbers, so that they might produce music and the human things of life. >> Through Beam schema awareness, let us live Ada's dream and join our >> multiple languages so that we may end our strife and produce the valuable >> stuff of life. >> >> Sincerely, >> >> Damon >> >
