I'm glad to know that the article has been well-received. In the second article, I will allocate a dedicated section to summarize the various challenges encountered when using Arrow for this type of project.
@Matt, I want to express my gratitude for your continuous support throughout this project. Your contributions and refinements to the Arrow Go library have enabled me to make significant progress with minimal obstacles. Best Regards, Laurent On Thu, Mar 30, 2023 at 2:24 PM Matt Topol <zotthewiz...@gmail.com> wrote: > +1 (non -binding) > > I'm glad others on here are finding this as useful and interesting as I > did. > > Great job Laurent! > > --Matt > > On Thu, Mar 30, 2023, 3:26 PM Raphael Taylor-Davies > <r.taylordav...@googlemail.com.invalid> wrote: > > > Hi Laurent, > > > > I gave the first blog post a read and I also really like it and would be > > +1 on publishing it, nice work. > > > > I would also like to echo Will's sentiment that getting real-world case > > studies for the more complex Arrow schemas is invaluable and will help > > drive improvements in this space, so thank you for driving this forward. > > > > Kind Regards, > > > > Raphael > > > > On 30/03/2023 19:52, Will Jones wrote: > > > Hi Laurent, > > > > > > I have read the first post and I really like it. I'd be +1 on > publishing > > > these to the blog. I'm interested to read the second one when it's > > finished. > > > > > > IMO the blog could use more examples of using Arrow that's not > building a > > > data frame library / query engine, and I appreciate that this blog > > provides > > > advice for some of the trickier parts of working with complex Arrow > > > schemas. I think this will also provide a good concrete use case for us > > to > > > think about improving the ecosystem's support for nested data. > > > > > > Best, > > > > > > Will Jones > > > > > > On Thu, Mar 30, 2023 at 10:56 AM Laurent Quérel < > > laurent.que...@gmail.com> > > > wrote: > > > > > >> Hello everyone, > > >> > > >> I was wondering if the Apache Arrow community would be interested in > > >> featuring a two-part article series on their blog, discussing the > > >> experiences and insights gained from an experimental version of the > > >> OpenTelemetry protocol (OTLP) utilizing Apache Arrow. As the main > > author of > > >> the OTLP Arrow specification > > >> < > > > https://github.com/lquerel/oteps/blob/main/text/0156-columnar-encoding.md > > >>> , > > >> the reference implementation otlp-arrow-adapter > > >> <https://github.com/f5/otel-arrow-adapter>, and the two articles (see > > >> links > > >> below), I believe that fostering collaboration between open-source > > projects > > >> like these is essential and mutually beneficial. > > >> > > >> These articles would serve as a fitting complement to the three > > >> introductory articles that Andrew Lamb and Raphael Taylor-Davies > > >> co-authored. They delve into the practical aspects of integrating > Apache > > >> Arrow into an existing project, as well as the process of converting a > > >> hierarchical data model into its Arrow representation. The first > article > > >> examines various mapping techniques for aligning an existing data > model > > >> with the corresponding Arrow representation, while the second article > > >> explores an adaptive schema technique that I implemented in the > > library's > > >> final version in greater depth. Although the second article is still > > under > > >> development, the core framework description is already in place. > > >> > > >> What are your thoughts on this proposal? > > >> > > >> Article 1: > > >> > > >> > > > https://docs.google.com/document/d/11lG7Go2IgKOyW-RReBRW6r7HIdV1X7lu5WrDGlW5LbQ/edit?usp=sharing > > >> > > >> Article 2 (WIP): > > >> > > >> > > > https://docs.google.com/document/d/1K2CqAtF4pZjpiVts8BOcq34sOcNgozvZ9ZZw-_zTv6I/edit?usp=sharing > > >> > > >> > > >> Best regards, > > >> > > >> Laurent Quérel > > >> > > >> -- > > >> Laurent Quérel > > >> > > > -- Laurent Quérel
