[
https://issues.apache.org/jira/browse/AVRO-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505952#comment-13505952
]
Scott Carey commented on AVRO-1105:
-----------------------------------
@Quinn: based on my glance on github, it seems that the implementation is
code-gen based and wraps the existing Java implementation for most of its work.
Is that correct? That is fine, code gen is a common use case with Avro (along
with two other common cases, I'll discuss shortly). As you indicate there are
Scala devs who would like to use it. We don't have to start out with all use
cases available, or with a pure Scala implementation.
h6. Common Avro use patterns
There are three common patterns for interacting with Avro data from code:
* "Schema First" (e.g. code gen) : Schemas are managed outside of the code, and
shared across products / languages. These generally represent business objects
and result in pure-data classes available to the programmer.
* "Code First" (e.g. reflection) : The canonical representation for data is in
code, and Avro schemas are generated based on that code for persistence of data
and schema evolution.
* "Dynamic" (e.g. Java generic API) : Code has no a priori knowledge of schemas
and programs interpret Avro data dynamically based on inputs or directives.
- Schema first patterns work well with long-lived data types and applications
built to directly work with those data types, or exchange them with other
applications. These applications often want to expose the data types to the
programmer directly (e.g. make record 'Foo' appear as class "Foo" with field
accessors name the same as the fields for compile time safety).
- Code first patterns have low programming overhead and fit well with agile use
cases, prototypes, or situations where a single language can host the canonical
representation of a long living data type.
- Dynamic patterns are required for general data processing and storage,
generic data access and transformation tools, or any other use case where a
priori knowledge of the schemas passing through the system by the programmer is
impossible or a burden.
If this patch only addresses one of the three use cases, that is OK with me, we
simply need to be clear what it does not do, and encourage others to contribute
work that completes other use cases. This is really a Scala code gen wrapper
around the Java implementation, we need it to be clear that this is not a full
language implementation -- maybe it is simply a module within the Java
implementation.
On the other hand, if there are ways to improve this work and achieve the same
use cases then that is something to consider now, especially if it improves
buy-in from the Scala community.
Typically, once a language has all three use case types, much of the
implementation overlaps on the back-end.
@John: This patch does not appear to address the dynamic use cases where
macros and type level programming would really shine, nor any code first style.
That would require a different contribution effort. However, for a schema
first style, are Scala 2.10 macros truly an alternative to code generation? I
believe they can generate classes conforming to types defined at compile time
from a schema, but are they powerful enough to inject type and field names that
correspond to the schema record and field names? I want to make sure we are
talking about solving the same use cases.
On the idiomatic Scala objection, I see a few things in the implementation that
are a result of using the Avro Java implementation's APIs for encoding,
decoding, and Schemas; changing that does not make sense for a Scala wrapper
around the Java API, I am more concerned about things that are exposed to users.
> Scala API for Avro
> ------------------
>
> Key: AVRO-1105
> URL: https://issues.apache.org/jira/browse/AVRO-1105
> Project: Avro
> Issue Type: New Feature
> Reporter: Christophe Taton
> Attachments: avro-scala.patch
>
>
> Umbrella issue.
> Goal is to provide Scala friendly APIs for Avro records and protocols (RPCs).
> Related project: http://code.google.com/p/avro-scala-compiler-plugin/ looks
> dead (no change since Sep 2010).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
