Re: (Default) values for logical types in human-readable form

2017-10-19 Thread Philip Zeyliger
I'm shaky on the details here, but shouldn't humans be using the *.avdl form of specifying schemas? On Thu, Oct 19, 2017 at 9:18 AM, Doug Cutting wrote: > On Thu, Oct 19, 2017 at 8:49 AM, Zoltan Ivanfi wrote: > > > > So then if an older reader reads a

Re: Standardizing char and varchar logical types

2017-10-19 Thread Ryan Blue
I don't think this is necessary. For char and varchar, the underlying storage shouldn't actually do anything differently. For example, what should Avro do if the user writes a long string to a VARCHAR(16) field? I think the last thing Avro should do is drop the extra bytes, so we're forced to do

Re: (Default) values for logical types in human-readable form

2017-10-19 Thread Doug Cutting
On Thu, Oct 19, 2017 at 8:49 AM, Zoltan Ivanfi wrote: > > So then if an older reader reads a schema field with "default-as-string" > > used instead of "default", it will decide that field has no default? I > > don't really like that, but it's better than using the wrong value

Re: (Default) values for logical types in human-readable form

2017-10-19 Thread Zoltan Ivanfi
Hi, On Thu, Oct 19, 2017 at 7:16 AM, Bridger Howell wrote: > So then if an older reader reads a schema field with "default-as-string" > used instead of "default", it will decide that field has no default? I > don't really like that, but it's better than using the wrong value

Standardizing char and varchar logical types

2017-10-19 Thread Zoltan Ivanfi
Hi, Apparently, when saving char or varchar columns to Avro, Hive and Spark add non-standard logical type annotations: {"type":"string","logicalType":"char","maxLength":42} {"type":"string","logicalType":"varchar","maxLength":42} Considering that probably these two SQL engines are the creators