Re: supporting a "unit" field for avro schema

2019-06-29 Thread Erik Erlandson
Regarding schema, my proposal for fingerprints would be that units are fingerprinted based on their canonical form, as defined here . Any two unit expressions having the same canonical form (including the corresponding

Re: supporting a "unit" field for avro schema

2019-06-29 Thread Niels Basjes
I think we should approach this idea in two parts: 1) The schema. Things like does a different unit mean a different schema fingerprint even though the bytes remain the same. What does a different unit mean for schema evolution. 2) Language specifics. Scala has different possibilities than Java.

Re: supporting a "unit" field for avro schema

2019-06-29 Thread Erik Erlandson
I've been puzzling over what can be done to support this in more widely-used languages. The dilemma relative to the current language ecosystem is that languages with "modern" type systems (Haskell, Rust, Scala, etc) capable of supporting compile-time unit checking, in the particular style I've

[jira] [Comment Edited] (AVRO-2429) Avro 1.9.0 fails when reading logical types other than "decimal"

2019-06-29 Thread Kengo Seki (JIRA)
[ https://issues.apache.org/jira/browse/AVRO-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875513#comment-16875513 ] Kengo Seki edited comment on AVRO-2429 at 6/29/19 1:40 PM: --- [~Fokko] The file I

[jira] [Commented] (AVRO-2429) Avro 1.9.0 fails when reading logical types other than "decimal"

2019-06-29 Thread Kengo Seki (JIRA)
[ https://issues.apache.org/jira/browse/AVRO-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875513#comment-16875513 ] Kengo Seki commented on AVRO-2429: -- [~Fokko] The file I attached is a valid avro file, as follows:

[jira] [Updated] (AVRO-2429) Avro 1.9.0 fails when reading logical types other than "decimal"

2019-06-29 Thread Kengo Seki (JIRA)
[ https://issues.apache.org/jira/browse/AVRO-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kengo Seki updated AVRO-2429: - Attachment: uuid.avro > Avro 1.9.0 fails when reading logical types other than "decimal" >

Re: supporting a "unit" field for avro schema

2019-06-29 Thread Niels Basjes
Hi, I attended your talk in Berlin and at the end I thought "too bad this is only Scala". I think it's a good idea to have this in Avro. The details will be tricky: How to encode the units in the schema for example. Especially because of the automatic conversion you spoke about. Niels On Fri,