Hi Laurent,
I gave the first blog post a read and I also really like it and would be
+1 on publishing it, nice work.
I would also like to echo Will's sentiment that getting real-world case
studies for the more complex Arrow schemas is invaluable and will help
drive improvements in this space, so thank you for driving this forward.
Kind Regards,
Raphael
On 30/03/2023 19:52, Will Jones wrote:
Hi Laurent,
I have read the first post and I really like it. I'd be +1 on publishing
these to the blog. I'm interested to read the second one when it's finished.
IMO the blog could use more examples of using Arrow that's not building a
data frame library / query engine, and I appreciate that this blog provides
advice for some of the trickier parts of working with complex Arrow
schemas. I think this will also provide a good concrete use case for us to
think about improving the ecosystem's support for nested data.
Best,
Will Jones
On Thu, Mar 30, 2023 at 10:56 AM Laurent Quérel <laurent.que...@gmail.com>
wrote:
Hello everyone,
I was wondering if the Apache Arrow community would be interested in
featuring a two-part article series on their blog, discussing the
experiences and insights gained from an experimental version of the
OpenTelemetry protocol (OTLP) utilizing Apache Arrow. As the main author of
the OTLP Arrow specification
<https://github.com/lquerel/oteps/blob/main/text/0156-columnar-encoding.md
,
the reference implementation otlp-arrow-adapter
<https://github.com/f5/otel-arrow-adapter>, and the two articles (see
links
below), I believe that fostering collaboration between open-source projects
like these is essential and mutually beneficial.
These articles would serve as a fitting complement to the three
introductory articles that Andrew Lamb and Raphael Taylor-Davies
co-authored. They delve into the practical aspects of integrating Apache
Arrow into an existing project, as well as the process of converting a
hierarchical data model into its Arrow representation. The first article
examines various mapping techniques for aligning an existing data model
with the corresponding Arrow representation, while the second article
explores an adaptive schema technique that I implemented in the library's
final version in greater depth. Although the second article is still under
development, the core framework description is already in place.
What are your thoughts on this proposal?
Article 1:
https://docs.google.com/document/d/11lG7Go2IgKOyW-RReBRW6r7HIdV1X7lu5WrDGlW5LbQ/edit?usp=sharing
Article 2 (WIP):
https://docs.google.com/document/d/1K2CqAtF4pZjpiVts8BOcq34sOcNgozvZ9ZZw-_zTv6I/edit?usp=sharing
Best regards,
Laurent Quérel
--
Laurent Quérel
