Re: supporting a "unit" field for avro schema

Ismaël Mejía pointed out that the AEP for this feature should be voted on, which I'm in complete agreement with. How should this proceed? Cheers, Erik On Wed, Sep 18, 2019 at 3:30 PM Erik Erlandson wrote: > > I drafted an AEP for unit metadata on schema: > >

Re: supporting a "unit" field for avro schema

To move this idea forward, and start exploring what an implementation of unit analysis in native Avro would really look like, I submitted a PR that implements unit analysis for python Avro schema: https://github.com/apache/avro/pull/841 I've also been playing around with a JSON-Schema that

Re: supporting a "unit" field for avro schema

I drafted an AEP for unit metadata on schema: https://docs.google.com/document/d/1IeVAtf6YcAAn35D4jmFQJjPpEMgEu79wWVMW37KvNps/ On Tue, Jul 16, 2019 at 1:35 PM Erik Erlandson wrote: > Hi Ryan, > Those are all great questions. They're all issues I have ideas about but > I'd want Avro community

Re: supporting a "unit" field for avro schema

Hi Ryan, Those are all great questions. They're all issues I have ideas about but I'd want Avro community input for as well. For that reason I answered them all on AVRO-2474 Cheers! E On Tue, Jul 16, 2019 at 3:13 AM Ryan Skraba wrote: > Hello!

Re: supporting a "unit" field for avro schema

Hello! I've been thinking about this and I generally like the idea of stronger types with units :D I have some questions about what you are thinking of when you say "first class concept" in Avro: - Would you expect a writer schema that wrote a Fahrenheit field and a reader schema that reads

Re: supporting a "unit" field for avro schema

If I'm interpreting the situation correctly, there is an "Avro Enhancement Proposal", but none have been filed in nearly a decade: https://cwiki.apache.org/confluence/display/AVRO/Avro+Enhancement+Proposals As a start, I submitted a jira to track this idea:

Re: supporting a "unit" field for avro schema

What should I do to move this forward? Does Avro have a PIP process? On Sat, Jun 29, 2019 at 3:26 PM Erik Erlandson wrote: > > Regarding schema, my proposal for fingerprints would be that units are > fingerprinted based on their canonical form, as defined here >

Re: supporting a "unit" field for avro schema

Regarding schema, my proposal for fingerprints would be that units are fingerprinted based on their canonical form, as defined here . Any two unit expressions having the same canonical form (including the corresponding

Re: supporting a "unit" field for avro schema

I think we should approach this idea in two parts: 1) The schema. Things like does a different unit mean a different schema fingerprint even though the bytes remain the same. What does a different unit mean for schema evolution. 2) Language specifics. Scala has different possibilities than Java.

Re: supporting a "unit" field for avro schema

I've been puzzling over what can be done to support this in more widely-used languages. The dilemma relative to the current language ecosystem is that languages with "modern" type systems (Haskell, Rust, Scala, etc) capable of supporting compile-time unit checking, in the particular style I've

Re: supporting a "unit" field for avro schema

Hi, I attended your talk in Berlin and at the end I thought "too bad this is only Scala". I think it's a good idea to have this in Avro. The details will be tricky: How to encode the units in the schema for example. Especially because of the automatic conversion you spoke about. Niels On Fri,

supporting a "unit" field for avro schema

Hi Avro community, Recently I have been experimenting with avro schema that are extended with a "unit" field. By "unit" I mean expressions like "second", or "megabyte" - that is "units of measure". I delivered a short talk on my experiments at Berlin Buzzwords, which can be viewed here: