Hi Ryan, Thanks for the clarification. Will do.
Casey Lucas Director, Engineering dynata.com The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. From: Ryan Blue <rb...@netflix.com.INVALID> Date: Tuesday, January 5, 2021 at 4:31 PM To: Iceberg Dev List <dev@iceberg.apache.org> Subject: [EXT] Re: implementation in other languages CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Hi Casey, We wrote the spec for Iceberg intending for it to be implemented in other languages, so we purposely avoided anything that would tie it to Java, like using Java `hashCode` implementations. It should be portable, but please let us know if you run into any issues. I'm not aware of an existing go implementation. If you start one, please let the community know! rb On Sun, Jan 3, 2021 at 1:20 PM Driesprong, Fokko <fo...@driesprong.frl> wrote: Hi Casey, 1. Personally I don't know any other implementation than in the repository itself, so JVM and Python. 2. There aren't any limitations. Most of the underlying tech used is compatible with many languages. One thing to keep in mind is that you should be able to read Avro files, which are also available in many languages. Apart from that, the data itself is written in Avro/Parquet/ORC which is also widely used and supported. I've been working on an Apache Beam implementation (very early stage), which also has a Go API. But that will take some time to get it to a certain maturity level. Hope this helps! Cheers, Fokko Op zo 3 jan. 2021 om 19:39 schreef Casey Lucas <mailto:casey.lu...@dynata.com>: I'm just learning about Iceberg. Thanks for the work that has gone into it thus far. It seems like a promising technology. I understand why the core implementation is Java based and saw that there are some aspects of the spec implemented in Python. 1. Is anyone aware of implementations in other languages such as go? 2. Is there anything that would fundamentally limit the use of other implementations being able to read and write Iceberg data that is also being manipulated by the JVM implementation? I assume no based on reading the spec, but any insight is appreciated. I'm curious if reading and writing Iceberg data from a spec-compliant implementation without necessarily involving Presto or Spark is an expected use-case. Thanks, Casey Lucas Director, Engineering https://nam10.safelinks.protection.outlook.com/?url=http://dynata.com/&data=04|01|casey.lu...@dynata.com|cd8b44683a464b2b755208d8b1c99d86|f0ff917dab8c4129b13f33be267a153b|0|0|637454826780075160|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0=|1000&sdata=EO6zIyEgnNMC0Tw1OyMuq6K8A/7VMZc/9CbRScvG2Ko=&reserved=0 The information contained in this e-mail message is intended for the use of the recipient(s) named above and is privileged and confidential. If you are not the intended recipient, you are formally notified that you have received this message in error and that any review, dissemination, distribution, or copying of the message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. -- Ryan Blue Software Engineer Netflix
