[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15634677#comment-15634677
]
Leonid Granovsky commented on AVRO-1341:
IMO AvroAlias should be applicable to field as well as to the type.
> Allow controlling avro via java annotations when using reflection.
> ---
>
> Key: AVRO-1341
> URL: https://issues.apache.org/jira/browse/AVRO-1341
> Project: Avro
> Issue Type: New Feature
> Components: java
>Reporter: Vincenz Priesnitz
>Assignee: Vincenz Priesnitz
> Fix For: 1.7.5
>
> Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch,
> AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch
>
>
> It would be great if one could control avro with java annotations. As of now,
> it is already possible to mark fields as Nullable or classes being encoded as
> a String. I propose a bigger set of annotations to control the behavior of
> avro on fields and classes. Such annotations have proven useful with jacksons
> json serialization and morphias mongoDB serialization.
> I propose the following additional annotations:
> @AvroName("alternativeName")
> @AvroAlias(alias="alias", space="space")
> @AvroIgnore
> @AvroMeta(key="K", value="V")
> @AvroEncode(using=CustomEncoding.class)
> Java fields with the @AvroName("alternativeName") annotation will be renamed
> in the induced schema. When reading an avro file via reflection, the
> reflection reader will look for fields in the schema with "alternativeName".
> For example:
> {code}
>@AvroName("foo")
>int bar;
> {code}
> is serialized as
> {code}
> { "name" : "foo", "type" : "int" }
> {code}
> The @AvroAlias annotation will add a new alias to the induced schema of a
> record, enum or field. The space parameter is optional and defaults to the
> namespace of the named schema the alias is added to.
> Fields with the @AvroIgnore annotation will be treated as if they had a
> transient modifier, i.e. they will not be written to or read from avro files.
> The @AvroMeta(key="K", value="V") annotation allows you to store an arbitrary
> key : value pair at every node in the schema.
> {code}
>@AvroMeta(key="fieldKey", value="fieldValue")
>int foo;
> {code}
> will create the following schema
> {code}
> {"name" : "foo", "type" : "int", "fieldKey" : "fieldValue" }
> {code}
> Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
> annotation. This annotation is a generalization of the @Stringable
> annotation. The @Stringable annotation is limited to classes with string
> argument constructors. Some classes can be similarly reduced to a smaller
> class or even a single primitive, but dont fit the requirements for
> @Stringable. A prominent example is java.util.Date, which instances can
> essentially be described with a single long. Such classes can now be encoded
> with a CustomEncoding, which reads and writes directly from the
> encoder/decoder.
> One simply extends the abstract CustomEncodings class by implementing a
> schema, a read method and a write method. A java field can then be annotated
> like this:
> {code}
> @AvroEncode(using=DateAslongEncoding.class)
> Date date;
> {code}
> The custom encoding implementation would look like
> {code}
> public class DateAsLongEncoding extends CustomEncoding {
> {
> schema = Schema.create(Schema.Type.LONG);
> schema.addProp("CustomEncoding", "DateAsLongEncoding");
> }
>
> @Override
> public void write(Object datum, Encoder out) throws IOException {
> out.writeLong(((Date)datum).getTime());
> }
>
> @Override
> public Date read(Object reuse, Decoder in) throws IOException {
> if (reuse != null) {
> ((Date)reuse).setTime(in.readLong());
> return (Date)reuse;
> }
> else return new Date(in.readLong());
> }
> }
> {code}
> I implemented said annotations and a custom encoding for java.util.Date as a
> proof of concept and also extended the @Stringable annotations to fields.
> This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394802#comment-14394802
]
Zhaonan Sun commented on AVRO-1341:
---
[~rdblue] Added here https://issues.apache.org/jira/browse/AVRO-1658
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Fix For: 1.7.5
Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch,
AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroAlias(alias=alias, space=space)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
The @AvroAlias annotation will add a new alias to the induced schema of a
record, enum or field. The space parameter is optional and defaults to the
namespace of the named schema the alias is added to.
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393138#comment-14393138
]
Ryan Blue commented on AVRO-1341:
-
[~sunzhaonan], there isn't an @AvroDoc annotation, but its a great idea. Could
you open an issue to add it? It shouldn't be too much work, either.
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Fix For: 1.7.5
Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch,
AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroAlias(alias=alias, space=space)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
The @AvroAlias annotation will add a new alias to the induced schema of a
record, enum or field. The space parameter is optional and defaults to the
namespace of the named schema the alias is added to.
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393091#comment-14393091
]
Zhaonan Sun commented on AVRO-1341:
---
Looks like @AvroMeta can't add reserved fields, like @AvroMeta(doc, some
doc) will have exceptions.
Do we have a @AvroDoc or something similar to this?
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Fix For: 1.7.5
Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch,
AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroAlias(alias=alias, space=space)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
The @AvroAlias annotation will add a new alias to the induced schema of a
record, enum or field. The space parameter is optional and defaults to the
namespace of the named schema the alias is added to.
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13732988#comment-13732988
]
Hudson commented on AVRO-1341:
--
SUCCESS: Integrated in AvroJava #390 (See
[https://builds.apache.org/job/AvroJava/390/])
AVRO-1341. Java: Add reflection annotations @AvroName, @AvroIgnore, @AvroMeta,
@AvroAlias and @AvroEncode. Contributed by Vincenz Priesnitz. (cutting: rev
1511531)
* /avro/trunk/CHANGES.txt
* /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/Schema.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/generic/GenericDatumReader.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/AvroAlias.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/AvroEncode.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/AvroIgnore.java
* /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/AvroMeta.java
* /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/AvroName.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/CustomEncoding.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/DateAsLongEncoding.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/FieldAccessReflect.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/FieldAccessUnsafe.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/FieldAccessor.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectData.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectDatumReader.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectDatumWriter.java
*
/avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/Stringable.java
* /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/reflect/package.html
*
/avro/trunk/lang/java/avro/src/test/java/org/apache/avro/reflect/TestReflect.java
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Fix For: 1.7.5
Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch,
AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroAlias(alias=alias, space=space)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
The @AvroAlias annotation will add a new alias to the induced schema of a
record, enum or field. The space parameter is optional and defaults to the
namespace of the named schema the alias is added to.
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719893#comment-13719893
]
Doug Cutting commented on AVRO-1341:
On second thought, I am now concerned that this could hurt performance.
AVRO-1282 did a lot to improve reflect performance and we wouldn't want to harm
that.
This patch adds calls to isAnnotationPresent to readField() and writeField(),
which are on the critical path.
Can you please run Perf.java with and without this patch? If performance is
affected then I suggest we should make isCustomEncoded and isStringable boolean
fields in FieldAccessor that are set when the field is constructed so that
their access cost in readField() and writeField() is negligible.
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Fix For: 1.7.5
Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch,
AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroAlias(alias=alias, space=space)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
The @AvroAlias annotation will add a new alias to the induced schema of a
record, enum or field. The space parameter is optional and defaults to the
namespace of the named schema the alias is added to.
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13700188#comment-13700188
]
Vincenz Priesnitz commented on AVRO-1341:
-
Attached is a new patch with more JavaDocs, including warnings for using custom
encodings.
I also moved the @AvroAlias annotation from issue AVRO-1347 here, but excluded
the controversial writer aliases and added unit tests. It is now also possible
to add an alias without a namespace.
This patch still contains the @AvroIgnore annotation, which I would really like
to see committed.
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Attachments: AVRO-1341.patch, AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699189#comment-13699189
]
Vincenz Priesnitz commented on AVRO-1341:
-
Thanks for reviewing the Patch.
I think it is clearer if there was an extra Ignore Annotation for Avro. Also,
the general java Annotations might collide with other Services that use
Annotations.
The AvroEncoder Annotation should indded only used by Experts; I will add such
warnings.
I'd like to mention Issue AVRO-1347 here, which adds another Annotation for
adding Aliases.
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Attachments: AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699250#comment-13699250
]
Doug Cutting commented on AVRO-1341:
Perhaps the annotation from AVRO-1347 should be included here instead?
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Attachments: AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-1341) Allow controlling avro via java annotations when using reflection.
[
https://issues.apache.org/jira/browse/AVRO-1341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13699458#comment-13699458
]
Vincenz Priesnitz commented on AVRO-1341:
-
If noone objects, i will include Avro-1347 into this issue tomorrow.
Allow controlling avro via java annotations when using reflection.
---
Key: AVRO-1341
URL: https://issues.apache.org/jira/browse/AVRO-1341
Project: Avro
Issue Type: New Feature
Components: java
Reporter: Vincenz Priesnitz
Assignee: Vincenz Priesnitz
Attachments: AVRO-1341.patch, AVRO-1341.patch
It would be great if one could control avro with java annotations. As of now,
it is already possible to mark fields as Nullable or classes being encoded as
a String. I propose a bigger set of annotations to control the behavior of
avro on fields and classes. Such annotations have proven useful with jacksons
json serialization and morphias mongoDB serialization.
I propose the following additional annotations:
@AvroName(alternativeName)
@AvroIgnore
@AvroMeta(key=K, value=V)
@AvroEncode(using=CustomEncoding.class)
Java fields with the @AvroName(alternativeName) annotation will be renamed
in the induced schema. When reading an avro file via reflection, the
reflection reader will look for fields in the schema with alternativeName.
For example:
{code}
@AvroName(foo)
int bar;
{code}
is serialized as
{code}
{ name : foo, type : int }
{code}
Fields with the @AvroIgnore annotation will be treated as if they had a
transient modifier, i.e. they will not be written to or read from avro files.
The @AvroMeta(key=K, value=V) annotation allows you to store an arbitrary
key : value pair at every node in the schema.
{code}
@AvroMeta(key=fieldKey, value=fieldValue)
int foo;
{code}
will create the following schema
{code}
{name : foo, type : int, fieldKey : fieldValue }
{code}
Fields can be custom encoded with the AvroEncode(using=CustomEncoding.class)
annotation. This annotation is a generalization of the @Stringable
annotation. The @Stringable annotation is limited to classes with string
argument constructors. Some classes can be similarly reduced to a smaller
class or even a single primitive, but dont fit the requirements for
@Stringable. A prominent example is java.util.Date, which instances can
essentially be described with a single long. Such classes can now be encoded
with a CustomEncoding, which reads and writes directly from the
encoder/decoder.
One simply extends the abstract CustomEncodings class by implementing a
schema, a read method and a write method. A java field can then be annotated
like this:
{code}
@AvroEncode(using=DateAslongEncoding.class)
Date date;
{code}
The custom encoding implementation would look like
{code}
public class DateAsLongEncoding extends CustomEncodingDate {
{
schema = Schema.create(Schema.Type.LONG);
schema.addProp(CustomEncoding, DateAsLongEncoding);
}
@Override
public void write(Object datum, Encoder out) throws IOException {
out.writeLong(((Date)datum).getTime());
}
@Override
public Date read(Object reuse, Decoder in) throws IOException {
if (reuse != null) {
((Date)reuse).setTime(in.readLong());
return (Date)reuse;
}
else return new Date(in.readLong());
}
}
{code}
I implemented said annotations and a custom encoding for java.util.Date as a
proof of concept and also extended the @Stringable annotations to fields.
This issue is a followup of AVRO-1328 and AVRO-1330.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
