[jira] [Commented] (AVRO-2070) Tolerate any Number when writing primitive values in Java in GenericDatumWriter
[
https://issues.apache.org/jira/browse/AVRO-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113304#comment-17113304
]
Hudson commented on AVRO-2070:
--
SUCCESS: Integrated in Jenkins build AvroJava #881 (See
[https://builds.apache.org/job/AvroJava/881/])
AVRO-2070 tolerate any numbers in GenericDatumWriter while writing (dan:
[https://github.com/apache/avro/commit/2c238747755a19fe6acce107b6fb1ce7d1984563])
* (edit)
lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumWriter.java
AVRO-2070 Added test (dan:
[https://github.com/apache/avro/commit/74ad244741a391450610f4d273172707ae2b9a80])
* (edit)
lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
AVRO-2070 Make import order same as master (dan:
[https://github.com/apache/avro/commit/97e05457f55e3f51dd8279df1247be65125d767a])
* (edit)
lang/java/avro/src/test/java/org/apache/avro/generic/TestGenericDatumWriter.java
> Tolerate any Number when writing primitive values in Java in
> GenericDatumWriter
> ---
>
> Key: AVRO-2070
> URL: https://issues.apache.org/jira/browse/AVRO-2070
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniil Gitelson
>Assignee: Rabi Kumar K C
>Priority: Major
> Fix For: 1.10.0
>
>
> Tolerating any Number (instead of concrete Long, Double, Float) makes
> possible to use mutable Number implmentation for performance reasons
> (specially for primitive collection iterations)
> Currently, this only works for int only:
> {code:java}
> // Here it works
> case INT: out.writeInt(((Number)datum).intValue()); break;
> // This should be replaced with ((Number)datum).longValue() etc
> case LONG:out.writeLong((Long)datum); break;
> case FLOAT: out.writeFloat((Float)datum); break;
> case DOUBLE: out.writeDouble((Double)datum); break;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2070) Tolerate any Number when writing primitive values in Java in GenericDatumWriter
[
https://issues.apache.org/jira/browse/AVRO-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113274#comment-17113274
]
ASF subversion and git services commented on AVRO-2070:
---
Commit 74ad244741a391450610f4d273172707ae2b9a80 in avro's branch
refs/heads/master from ravowlga123
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=74ad244 ]
AVRO-2070 Added test
> Tolerate any Number when writing primitive values in Java in
> GenericDatumWriter
> ---
>
> Key: AVRO-2070
> URL: https://issues.apache.org/jira/browse/AVRO-2070
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniil Gitelson
>Assignee: Rabi Kumar K C
>Priority: Major
>
> Tolerating any Number (instead of concrete Long, Double, Float) makes
> possible to use mutable Number implmentation for performance reasons
> (specially for primitive collection iterations)
> Currently, this only works for int only:
> {code:java}
> // Here it works
> case INT: out.writeInt(((Number)datum).intValue()); break;
> // This should be replaced with ((Number)datum).longValue() etc
> case LONG:out.writeLong((Long)datum); break;
> case FLOAT: out.writeFloat((Float)datum); break;
> case DOUBLE: out.writeDouble((Double)datum); break;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2070) Tolerate any Number when writing primitive values in Java in GenericDatumWriter
[
https://issues.apache.org/jira/browse/AVRO-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113275#comment-17113275
]
ASF subversion and git services commented on AVRO-2070:
---
Commit 97e05457f55e3f51dd8279df1247be65125d767a in avro's branch
refs/heads/master from ravowlga123
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=97e0545 ]
AVRO-2070 Make import order same as master
> Tolerate any Number when writing primitive values in Java in
> GenericDatumWriter
> ---
>
> Key: AVRO-2070
> URL: https://issues.apache.org/jira/browse/AVRO-2070
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniil Gitelson
>Assignee: Rabi Kumar K C
>Priority: Major
>
> Tolerating any Number (instead of concrete Long, Double, Float) makes
> possible to use mutable Number implmentation for performance reasons
> (specially for primitive collection iterations)
> Currently, this only works for int only:
> {code:java}
> // Here it works
> case INT: out.writeInt(((Number)datum).intValue()); break;
> // This should be replaced with ((Number)datum).longValue() etc
> case LONG:out.writeLong((Long)datum); break;
> case FLOAT: out.writeFloat((Float)datum); break;
> case DOUBLE: out.writeDouble((Double)datum); break;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2070) Tolerate any Number when writing primitive values in Java in GenericDatumWriter
[
https://issues.apache.org/jira/browse/AVRO-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17113273#comment-17113273
]
ASF subversion and git services commented on AVRO-2070:
---
Commit 2c238747755a19fe6acce107b6fb1ce7d1984563 in avro's branch
refs/heads/master from ravowlga123
[ https://gitbox.apache.org/repos/asf?p=avro.git;h=2c23874 ]
AVRO-2070 tolerate any numbers in GenericDatumWriter while writing primitives
in java
> Tolerate any Number when writing primitive values in Java in
> GenericDatumWriter
> ---
>
> Key: AVRO-2070
> URL: https://issues.apache.org/jira/browse/AVRO-2070
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniil Gitelson
>Assignee: Rabi Kumar K C
>Priority: Major
>
> Tolerating any Number (instead of concrete Long, Double, Float) makes
> possible to use mutable Number implmentation for performance reasons
> (specially for primitive collection iterations)
> Currently, this only works for int only:
> {code:java}
> // Here it works
> case INT: out.writeInt(((Number)datum).intValue()); break;
> // This should be replaced with ((Number)datum).longValue() etc
> case LONG:out.writeLong((Long)datum); break;
> case FLOAT: out.writeFloat((Float)datum); break;
> case DOUBLE: out.writeDouble((Double)datum); break;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2070) Tolerate any Number when writing primitive values in Java in GenericDatumWriter
[
https://issues.apache.org/jira/browse/AVRO-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040540#comment-17040540
]
jason mathews commented on AVRO-2070:
-
This issue is a bug not an improvement. See the previous comment.
Please reclassify the issue as a bug against the java implementation. Thanks.
> Tolerate any Number when writing primitive values in Java in
> GenericDatumWriter
> ---
>
> Key: AVRO-2070
> URL: https://issues.apache.org/jira/browse/AVRO-2070
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniil Gitelson
>Assignee: Rabi Kumar K C
>Priority: Major
>
> Tolerating any Number (instead of concrete Long, Double, Float) makes
> possible to use mutable Number implmentation for performance reasons
> (specially for primitive collection iterations)
> Currently, this only works for int only:
> {code:java}
> // Here it works
> case INT: out.writeInt(((Number)datum).intValue()); break;
> // This should be replaced with ((Number)datum).longValue() etc
> case LONG:out.writeLong((Long)datum); break;
> case FLOAT: out.writeFloat((Float)datum); break;
> case DOUBLE: out.writeDouble((Double)datum); break;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AVRO-2070) Tolerate any Number when writing primitive values in Java in GenericDatumWriter
[
https://issues.apache.org/jira/browse/AVRO-2070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008942#comment-17008942
]
jason mathews commented on AVRO-2070:
-
This issue should be categorized as a Bug not an Improvement.
I'm running into this issue and have to create a custom GenericDatumWriter
class to allow for mixed number type instances as this fix would elminate doing
so.
Using a mix number types in Java (Short, Integer, Long, Float) when type is
Double results in a ClassCastException.
Java Example:
{code:java}
Schema doubleType = Schema.create(Schema.Type.DOUBLE);
Schema.Field field = new Schema.Field("d", doubleType);
List fields = Collections.singletonList(field);
Schema schema = Schema.createRecord("test", "doc", "", false, fields);
// serialize
GenericDatumWriter datumWriter = new
GenericDatumWriter<>(schema);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try(DataFileWriter dataWriter = new
DataFileWriter<>(datumWriter)) {
dataWriter.create(schema, bos);
GenericData.Record r = new GenericData.Record(schema);
r.put("d", 123.456);
dataWriter.append(r);
r = new GenericData.Record(schema);
r.put("d", 123); // try as Integer
dataWriter.append(r); // throws exception
{code}
Output:
{noformat}
Exception in thread "main"
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.ClassCastException: java.lang.Integer cannot be cast to
java.lang.Double
{noformat}
But having mixed numeric types in Python fastavro implementation has no such
number restriction and a double schema type for example can contain a mix of
floating point or integers.
Python Example:
{code:java}
from fastavro import json_writer, json_reader, parse_schema
schema = {
"namespace": "",
"type": "record",
"name": "record",
"fields": [
{ "name": "d", "type": "double" }
]
}
parsed_schema = parse_schema(schema)
records = [
{ u'd': 1.2345 },
{ u'd': 12345 }
]
with open('test.avro', 'w') as out:
#fastavro.schemaless_writer(out, parsed_schema, { u'd': 1.2345 } )
#fastavro.schemaless_writer(out, parsed_schema, { u'd': 12345 } )
json_writer(out, parsed_schema, records)
with open('test.avro', 'r') as fo:
avro_reader = json_reader(fo, schema)
for record in avro_reader:
print(record)
"""
output:
{'d': 1.2345}
{'d': 12345}
"""
{code}
> Tolerate any Number when writing primitive values in Java in
> GenericDatumWriter
> ---
>
> Key: AVRO-2070
> URL: https://issues.apache.org/jira/browse/AVRO-2070
> Project: Apache Avro
> Issue Type: Improvement
> Components: java
>Reporter: Daniil Gitelson
>Priority: Major
>
> Tolerating any Number (instead of concrete Long, Double, Float) makes
> possible to use mutable Number implmentation for performance reasons
> (specially for primitive collection iterations)
> Currently, this only works for int only:
> {code:java}
> // Here it works
> case INT: out.writeInt(((Number)datum).intValue()); break;
> // This should be replaced with ((Number)datum).longValue() etc
> case LONG:out.writeLong((Long)datum); break;
> case FLOAT: out.writeFloat((Float)datum); break;
> case DOUBLE: out.writeDouble((Double)datum); break;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
