[
https://issues.apache.org/jira/browse/AVRO-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vegard Solberg updated AVRO-3218:
---------------------------------
Description:
I get an error when parsing a schema which contains a field with
`logicalType=decimal` in Python. Spesifically, I am using avro.schema.parse()
on a schema which have a field on the following format:
{code:java}
{ "name": "myField", "type": [ "null", { "type": "bytes", "logicalType":
"decimal", "precision": 12, "scale": 2 } ]}
{code}
Eventually, make_bytes_decimal_schema() is called. At this point, the variable
other_props looks like this (and is fine):
{code:java}
other_props = {'logicalType': 'decimal', 'precision': 12, 'scale': 2}{code}
Unlike the other logical types, the variable other_props is not passed on to
BytesDecimalSchema. The result is that the resulting logical_schema is missing
the key-value pair 'logicalType': 'decimal'. This blocks me from writing to a
kafka topic, because the schema is parsed incorrectly. The resulting field
looks like this:
{code:java}
{"type": ["null", {"type": "bytes", "precision": 12, "scale": 2}], "name":
"myField"}
{code}
The suggested fix is to simply pass other_props as an argument into
BytesDecimalSchema(). The attached file [^issue.py]contains code to reproduce
the bug
This commit contains the suggested changes: [AVRO-3218 Fix parsing of
logicalType = decimal · vegraux/avro@b5c4a78
(github.com)|https://github.com/vegraux/avro/commit/b5c4a78bc314bdecb6e1b7b32ba5d02ab8670c72]
was:
I get an error when parsing a schema which contains a field with
`logicalType=decimal` in Python. Spesifically, I am using avro.schema.parse()
on a schema which have a field on the following format:
{code:java}
{ "name": "myField", "type": [ "null", { "type": "bytes", "logicalType":
"decimal", "precision": 12, "scale": 2 } ]}
{code}
Eventually, make_bytes_decimal_schema() is called. At this point, the variable
other_props looks like this (and is fine):
{code:java}
other_props = {'logicalType': 'decimal', 'precision': 12, 'scale': 2}{code}
Unlike the other logical types, the variable other_props is not passed on to
BytesDecimalSchema. The result is that the resulting logical_schema is missing
the key-value pair 'logicalType': 'decimal'. This blocks me from writing to a
kafka topic, because the schema is parsed incorrectly. The resulting field
looks like this:
{code:java}
{"type": ["null", {"type": "bytes", "precision": 12, "scale": 2}], "name":
"myField"}
{code}
The suggested fix is to simply pass other_props as an argument into
BytesDecimalSchema().
This commit contains the suggested changes: [AVRO-3218 Fix parsing of
logicalType = decimal · vegraux/avro@b5c4a78
(github.com)|https://github.com/vegraux/avro/commit/b5c4a78bc314bdecb6e1b7b32ba5d02ab8670c72]
> Pass LogicalType to BytesDecimalSchema in Python
> ------------------------------------------------
>
> Key: AVRO-3218
> URL: https://issues.apache.org/jira/browse/AVRO-3218
> Project: Apache Avro
> Issue Type: Bug
> Components: python
> Affects Versions: 1.10.2
> Environment: I am using avro 1.10.2. Example file to reproduce the
> problem.
> [^issue.py]
> Reporter: Vegard Solberg
> Priority: Major
> Attachments: issue.py
>
>
> I get an error when parsing a schema which contains a field with
> `logicalType=decimal` in Python. Spesifically, I am using avro.schema.parse()
> on a schema which have a field on the following format:
> {code:java}
> { "name": "myField", "type": [ "null", { "type": "bytes", "logicalType":
> "decimal", "precision": 12, "scale": 2 } ]}
> {code}
> Eventually, make_bytes_decimal_schema() is called. At this point, the
> variable other_props looks like this (and is fine):
>
> {code:java}
> other_props = {'logicalType': 'decimal', 'precision': 12, 'scale': 2}{code}
>
> Unlike the other logical types, the variable other_props is not passed on to
> BytesDecimalSchema. The result is that the resulting logical_schema is
> missing the key-value pair 'logicalType': 'decimal'. This blocks me from
> writing to a kafka topic, because the schema is parsed incorrectly. The
> resulting field looks like this:
> {code:java}
> {"type": ["null", {"type": "bytes", "precision": 12, "scale": 2}], "name":
> "myField"}
> {code}
>
> The suggested fix is to simply pass other_props as an argument into
> BytesDecimalSchema(). The attached file [^issue.py]contains code to reproduce
> the bug
>
> This commit contains the suggested changes: [AVRO-3218 Fix parsing of
> logicalType = decimal · vegraux/avro@b5c4a78
> (github.com)|https://github.com/vegraux/avro/commit/b5c4a78bc314bdecb6e1b7b32ba5d02ab8670c72]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
