[jira] [Commented] (AVRO-784) SpecificCompiler should generate accessors
[ https://issues.apache.org/jira/browse/AVRO-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222130#comment-13222130 ] Scott Carey commented on AVRO-784: -- I think there were some details that did not make it into AVRO-839. In particular, all accessors in AVRO-839 appear to be boxed, whether the value is nullable or not. I have not moved to using the API there because of the boxing overhead incurred. These issues will need to go into another ticket for later. > SpecificCompiler should generate accessors > -- > > Key: AVRO-784 > URL: https://issues.apache.org/jira/browse/AVRO-784 > Project: Avro > Issue Type: Improvement > Components: java >Affects Versions: 1.5.0 >Reporter: E. Sammer > Labels: features > Attachments: avro-784.diff, avro-784.diff > > > Avro's Java SpecificCompiler should generate java bean style accessors. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-816) Schema Comparison Utils
[
https://issues.apache.org/jira/browse/AVRO-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222127#comment-13222127
]
Scott Carey commented on AVRO-816:
--
Sorry for the (far too long) delay.
{quote}
In the case of unions, maximum fidelity is simple, but how would you propose
"unifying" a type: "long" and type: "int" field that aren't in a union? Are you
proposing that the composite type would be the union of "long" and "int"?
{quote}
Yes, I am suggesting that an operation that creates a union of two schemas
would result in all possible branches existing and not just a compatible
subset. A separate operation would take care of the latter, but I'm not sure
what a good name for that would be.
The use case for each of these differs. In one case the need is for an exact
superset schema in order to read all variations with maximum information
retention -- for example this may be useful when reading over a large set of
archived data with a long schema evolution history.
In another situation, reading all of the data with possible loss of precision
is ok, and instead the goal is to generate the simplest schema that can read
all of the data within an accepted level of precision loss.
My contention is not that we must create both of these in this ticket, but that
if we only have one of them we leave space in the API conceptually for the
other (in documentation and class/method names).
{quote}These rules are explicitly about reading data. Perhaps we could name
them 'resolves' and 'resolve'?{quote}
Perhaps in another ticket we can talk about how to change the language in the
spec to be about about the operations we apply to the schema rather than what
we are using those operations for. I think at this point that is relatively
low priority until we start adding other features besides reading that rely on
schema resolution, and I don't want to tie that discussion to this ticket.
{quote}
So do you think these methods don't belong on Schema, but rather on some
utility class in the io package
{quote}
It makes sense to keep pure schema operations in the Schema class or with other
Schema utilities. If I were to start over from scratch I'd probably have an
org.apache.avro.schema package that had no dependencies on other concepts in
Avro such as serialization or object representations. But that is separate
from this ticket. Lets just be careful with mixing schema
manipulation/transformation concepts unnecessarily with object representation
or serialization concepts.
> Schema Comparison Utils
> ---
>
> Key: AVRO-816
> URL: https://issues.apache.org/jira/browse/AVRO-816
> Project: Avro
> Issue Type: New Feature
> Components: java
>Reporter: Joe Crobak
>Assignee: Joe Crobak
>Priority: Minor
> Fix For: 1.7.0
>
> Attachments: AVRO-816.patch, AVRO-816.patch, AVRO-816.patch,
> AVRO-816.patch
>
>
> From my post on the mailing list, and Doug's response:
> {quote}
> On 05/05/2011 10:29 AM, Joe Crobak wrote:
> > We've recently come across a situation where we have two data files with
> > different schemas that we'd like to process together using
> > GenericDatumReader. One schema is promotable to the other, but not vice
> > versa. We'd like to programmatically determine which of the schemas to
> > use. I did a brief look through javadoc and tests, and I couldn't find
> > any examples of checking if one schema is promotable to the other. Has
> > anyone else come across this?
> >
> > For some context, we're considering patching AvroStorage [1] to remove
> > the assumption that all files have the same schema. In our case, our
> > schema has evolved in that a field that was an int was promoted to a long.
> A boolean method that tells you if one schema is promotable to another
> would work in this case, but would not help in cases where, e.g.,
> different fields had changed in different versions. For example, in
> branched development, two branches might each add a distinct symbol to
> an enum. So I think you might be better off with a method that, given
> two schemas, returns their superset, a schema that can read data written
> by either.
> Such a method does not yet exist in Avro, but should not be difficult to
> add. Please file an issue in Jira if this sounds of interest.
> Doug
> {quote}
> I think it would be useful to have both of the methods that Doug mentioned in
> some sort of schema utils class.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Avro RPC interop
Hi Avro devs, What is the latest on Avro RPC interoperability? Flume 1.x is currently using the Avro Netty implementation as an RPC protocol, however I am not aware of compatible implementations of this transport in languages other than Java [1] and I wonder if this protocol will ever be compatible [2] with a future standard [3]. Would any of you folks recommend running RPC APIs intended for public consumption on top of NettyServer at this time? It seems to be the one with the most active development going on, and appears to have significant interest, which is a good sign. Since there are other RPC transports available as well - SaslSocketServer and HttpServer - I am also wondering what anecdotal experience others can share about these implementations, in terms of speed, interoperability, and maturity. Thanks in advance for your thoughts. Regards, Mike [1] https://mail-archives.apache.org/mod_mbox/avro-dev/201201.mbox/%3c4f1dc9ec.1080...@apache.org%3E [2] https://mail-archives.apache.org/mod_mbox/avro-dev/201201.mbox/%3c4f1a09b9.1010...@apache.org%3E [3] https://issues.apache.org/jira/browse/AVRO-341 - specify avro transport in spec
Re: Mailing list hosting @ MarkMail
You might be interested in using http://search-hadoop.com/?q=&fc_project=Avro Mailing lists, JIRA tickets, javadoc, and the website are all indexed. -Scott On 2/25/12 4:23 AM, "lewis john mcgibbney" wrote: >Hi Avro Devs, > >I'm pretty new to Avro, I'll therefore find myself moving towards the >user@lists to ask questions which I may have found if they were >indexed/archived >and searchable. I know that you currently have the mod_mbox archives, but >these are difficult to browse. > >It's a trivial task of setting this up and I would be happy to get it done >if people agree. > >Thanks for now > >Lewis
Re: [VOTE] Avro release 1.6.3 (rc 0)
+1 Checksums match, src tarball builds and tests pass. (Built on Mac OS X 10.6.8, csharp not built or tested). Maven repository works, my projects build and test successfully against it (Java). -Scott On 3/2/12 2:52 PM, "Doug Cutting" wrote: >I have created a candidate build for Avro release 1.6.3. > >Changes are listed at: > > http://s.apache.org/o6a > >Please download, test, and vote. > > http://people.apache.org/~cutting/avro-1.6.3-rc0/ > >The Maven staging repository is at: > > https://repository.apache.org/content/repositories/orgapacheavro-045/ > >Thanks in advance for voting! > >Doug
[jira] [Commented] (AVRO-987) Make Avro OSGi ready
[ https://issues.apache.org/jira/browse/AVRO-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221862#comment-13221862 ] Claus Ibsen commented on AVRO-987: -- Any update on this? > Make Avro OSGi ready > > > Key: AVRO-987 > URL: https://issues.apache.org/jira/browse/AVRO-987 > Project: Avro > Issue Type: New Feature > Components: java >Reporter: Ioannis Canellos > Attachments: AVRO-987-patch-updated.txt, AVRO-987-patch.txt > > > It would be really nice to be able to use Avro inside OSGi. To achieve this > two things are required: > i) Provide proper MANIFEST.MF. > ii) Deal with potential class loading issues. Avro uses Class.forName a lot > and that is not very OSGi friendly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
