[
https://issues.apache.org/jira/browse/AVRO-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael A. Smith updated AVRO-2611:
-----------------------------------
Description:
The spec says
bq. Language implementations must ignore unknown logical types when reading,
and should use the underlying Avro type. If a logical type is invalid, for
example a decimal with scale greater than its precision, then implementations
should ignore the logical type and use the underlying Avro type.
Work to adhere to this is in AVRO-2429. However, users debugging their avro
usage may wish to get more feedback about logical and literal schema
mismatches. So I propose that we use the Python [warnings
module|https://docs.python.org/2/library/warnings.html] to provide warnings
about schema mismatches, which will natively give the user control over what
warnings they see. Here's an excerpt from the Python warnings documentation:
bq. Warning messages are normally written to {{sys.stderr}}, but their
disposition can be changed flexibly, from ignoring all warnings to turning them
into exceptions. The disposition of warnings can vary based on the warning
category (see below), the text of the warning message, and the source location
where it is issued. Repetitions of a particular warning for the same source
location are typically suppressed.
The logging module also supports [easily transforming
warnings|https://docs.python.org/2/library/logging.html#logging.captureWarnings]
into logger.warn calls.
We should create a subclass of {{UserWarning}} from which all future avro
warnings should extend, and then create an {{IgnoredLogicalType}} warning from
that for this case.
I think the warning should be enabled in "once" mode by default.
was:
The spec says
bq. Language implementations must ignore unknown logical types when reading,
and should use the underlying Avro type. If a logical type is invalid, for
example a decimal with scale greater than its precision, then implementations
should ignore the logical type and use the underlying Avro type.
Work to adhere to this is in AVRO-2429. However, users debugging their avro
usage may wish to get more feedback about logical and literal schema
mismatches. So I propose that we use the Python [warnings
module|https://docs.python.org/2/library/warnings.html] to provide warnings
about schema mismatches, which will natively give the user control over what
warnings they see. Here's an excerpt from the Python warnings documentation:
bq. Warning messages are normally written to {{sys.stderr}}, but their
disposition can be changed flexibly, from ignoring all warnings to turning them
into exceptions. The disposition of warnings can vary based on the warning
category (see below), the text of the warning message, and the source location
where it is issued. Repetitions of a particular warning for the same source
location are typically suppressed.
The logging module also supports [easily transforming
warnings](https://docs.python.org/2/library/logging.html#logging.captureWarnings)
into logger.warn calls.
We should create a subclass of {{UserWarning}} from which all future avro
warnings should extend, and then create an {{IgnoredLogicalType}} warning from
that for this case.
I think the warning should be enabled in "once" mode by default.
> Provide warnings for mismatched logical/literal schema
> ------------------------------------------------------
>
> Key: AVRO-2611
> URL: https://issues.apache.org/jira/browse/AVRO-2611
> Project: Apache Avro
> Issue Type: Improvement
> Components: python
> Reporter: Michael A. Smith
> Priority: Minor
>
> The spec says
> bq. Language implementations must ignore unknown logical types when reading,
> and should use the underlying Avro type. If a logical type is invalid, for
> example a decimal with scale greater than its precision, then implementations
> should ignore the logical type and use the underlying Avro type.
> Work to adhere to this is in AVRO-2429. However, users debugging their avro
> usage may wish to get more feedback about logical and literal schema
> mismatches. So I propose that we use the Python [warnings
> module|https://docs.python.org/2/library/warnings.html] to provide warnings
> about schema mismatches, which will natively give the user control over what
> warnings they see. Here's an excerpt from the Python warnings documentation:
> bq. Warning messages are normally written to {{sys.stderr}}, but their
> disposition can be changed flexibly, from ignoring all warnings to turning
> them into exceptions. The disposition of warnings can vary based on the
> warning category (see below), the text of the warning message, and the source
> location where it is issued. Repetitions of a particular warning for the same
> source location are typically suppressed.
> The logging module also supports [easily transforming
> warnings|https://docs.python.org/2/library/logging.html#logging.captureWarnings]
> into logger.warn calls.
> We should create a subclass of {{UserWarning}} from which all future avro
> warnings should extend, and then create an {{IgnoredLogicalType}} warning
> from that for this case.
> I think the warning should be enabled in "once" mode by default.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
