[jira] [Commented] (AVRO-1126) Upgrade to Jackson 2+
[
https://issues.apache.org/jira/browse/AVRO-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805269#comment-13805269
]
Anthony Watkins commented on AVRO-1126:
---
It is great to see this direction. We have adopted Avro at Flurry to support
our ad serving protocol using Avro-C for our iOS sdk and Avro Java for Android
and on our server. It works great, but the 1 significant downside has been the
reliance on the jackson lib. Nothing against Jackson, it's a great lib, however
the size of the jackson-mapper classes were 700-800Kb, which increased our sdk
size about 4x. In mobile, this becomes an issue particularly since we are a 3rd
party lib. We had started some effort to refactor jackson out, but decided it
was too woven into the code and we were concerned upgrading to future version
of Avro would prove difficult because of this.
An abstracted api would great, hopefully one in which we could even plug in the
native Android json parser.
Upgrade to Jackson 2+
-
Key: AVRO-1126
URL: https://issues.apache.org/jira/browse/AVRO-1126
Project: Avro
Issue Type: Task
Components: java
Reporter: James Tyrrell
Priority: Critical
Fix For: 1.8.0
Quite annoyingly with Jackson 2+ the base package name has changed from
org.codehaus.jackson to com.fasterxml.jackson so in addition to changing the
dependencies from:
{code:xml}
dependency
groupIdorg.codehaus.jackson/groupId
artifactIdjackson-core-asl/artifactId
version${jackson.version}/version
/dependency
dependency
groupIdorg.codehaus.jackson/groupId
artifactIdjackson-mapper-asl/artifactId
version${jackson.version}/version
/dependency
{code}
to:
{code:xml}
dependency
groupIdcom.fasterxml.jackson.core/groupId
artifactIdjackson-core/artifactId
version${jackson.version}/version
/dependency
dependency
groupIdcom.fasterxml.jackson.core/groupId
artifactIdjackson-databind/artifactId
version${jackson.version}/version
/dependency
{code}
the base package in the code needs to be updated. More info can be found
[here|http://wiki.fasterxml.com/JacksonUpgradeFrom19To20], I am happy to do
the work just let me know what is preferable i.e. should I just attach a
patch to this issue?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Updated] (AVRO-1384) avro-maven-plugin: Can't import idl files as classpath resources from the same set of sources
[
https://issues.apache.org/jira/browse/AVRO-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Doug Cutting updated AVRO-1384:
---
Attachment: AVRO-1384.patch
Sorry it took so long for me to look at this!
It looks fine except for the inclusion of some unnecessary whitespace-only
changes. Here's a version of the patch without those.
Unless someone objects, I'll commit this soon.
avro-maven-plugin: Can't import idl files as classpath resources from the
same set of sources
--
Key: AVRO-1384
URL: https://issues.apache.org/jira/browse/AVRO-1384
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.7.5
Reporter: Alexandre Normand
Assignee: Alexandre Normand
Attachments: AVRO-1384.patch, AVRO-1384.patch, AVRO-1384-rev1.patch
We have a use case for importing idl files as classpath resources (it makes
possible to re-use idls in downstream artifacts if all imports use classpath
resources as opposed to a file-based reference). To enable this, the imports
in the same artifacts should be able to use classpath resources for other
avro files in the same source structure.
The proposed approach is to just add the source directory of avro files to
the classpath. The avro resources would then follow the directory structure.
This means that if someone configures the avro-maven-plugin with a source
directory like this:
{code}
sourceDirectory${project.basedir}/src/main/resources/sourceDirectory
{code}
And the idl files live under {{project.basedir/src/main/resources/avro}}, the
imports could be referencing other avro files like this:
{code}
import idl avro/otherSchema.avdl;
{code}
Things open for discussion:
* Any documentation required to make this clear to end-users?
* Should I add the resources directories to the classpath as well?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (AVRO-1384) avro-maven-plugin: Can't import idl files as classpath resources from the same set of sources
[
https://issues.apache.org/jira/browse/AVRO-1384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805736#comment-13805736
]
Alexandre Normand commented on AVRO-1384:
-
Thanks! (and sorry about the extra white-spaces)
avro-maven-plugin: Can't import idl files as classpath resources from the
same set of sources
--
Key: AVRO-1384
URL: https://issues.apache.org/jira/browse/AVRO-1384
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.7.5
Reporter: Alexandre Normand
Assignee: Alexandre Normand
Attachments: AVRO-1384.patch, AVRO-1384.patch, AVRO-1384-rev1.patch
We have a use case for importing idl files as classpath resources (it makes
possible to re-use idls in downstream artifacts if all imports use classpath
resources as opposed to a file-based reference). To enable this, the imports
in the same artifacts should be able to use classpath resources for other
avro files in the same source structure.
The proposed approach is to just add the source directory of avro files to
the classpath. The avro resources would then follow the directory structure.
This means that if someone configures the avro-maven-plugin with a source
directory like this:
{code}
sourceDirectory${project.basedir}/src/main/resources/sourceDirectory
{code}
And the idl files live under {{project.basedir/src/main/resources/avro}}, the
imports could be referencing other avro files like this:
{code}
import idl avro/otherSchema.avdl;
{code}
Things open for discussion:
* Any documentation required to make this clear to end-users?
* Should I add the resources directories to the classpath as well?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (AVRO-1387) DataFileWriter must allow files not to be flushed at the end of every block.
[ https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805752#comment-13805752 ] ASF subversion and git services commented on AVRO-1387: --- Commit 1535888 from [~cutting] in branch 'avro/trunk' [ https://svn.apache.org/r1535888 ] AVRO-1387. Java: Add DataFileWriter option to inhibit flush per block. Contributed by Hari Shreedharan. DataFileWriter must allow files not to be flushed at the end of every block. Key: AVRO-1387 URL: https://issues.apache.org/jira/browse/AVRO-1387 Project: Avro Issue Type: Bug Reporter: Hari Shreedharan Assignee: Hari Shreedharan Fix For: 1.7.6 Attachments: AVRO-1387.patch We are considering changes in Flume's file channel to use Avro, one of the requirements is that each event (which maps to one avro record) be checksummed so we know if the data is corrupt. We'd probably have to add a new version for this, since this will change the data format on disk. I can start working on a Java version if there are no objections -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (AVRO-1126) Upgrade to Jackson 2+
[
https://issues.apache.org/jira/browse/AVRO-1126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805774#comment-13805774
]
Doug Cutting commented on AVRO-1126:
Android's json libraries do not implement a standard, abstract API.
The javax.json glassfish implementation looks to be a considerably smaller than
Jackson. So switching to using the javax.json API for parsing printing JSON
should permit one to choose between Jackson and at least one smaller
implementation, but not the implementation built into Android.
Note that this is separate from the representation used for default values in
Avro's public API. For that, I'm currently leaning towards something like
MapString,Object, etc.
Upgrade to Jackson 2+
-
Key: AVRO-1126
URL: https://issues.apache.org/jira/browse/AVRO-1126
Project: Avro
Issue Type: Task
Components: java
Reporter: James Tyrrell
Priority: Critical
Fix For: 1.8.0
Quite annoyingly with Jackson 2+ the base package name has changed from
org.codehaus.jackson to com.fasterxml.jackson so in addition to changing the
dependencies from:
{code:xml}
dependency
groupIdorg.codehaus.jackson/groupId
artifactIdjackson-core-asl/artifactId
version${jackson.version}/version
/dependency
dependency
groupIdorg.codehaus.jackson/groupId
artifactIdjackson-mapper-asl/artifactId
version${jackson.version}/version
/dependency
{code}
to:
{code:xml}
dependency
groupIdcom.fasterxml.jackson.core/groupId
artifactIdjackson-core/artifactId
version${jackson.version}/version
/dependency
dependency
groupIdcom.fasterxml.jackson.core/groupId
artifactIdjackson-databind/artifactId
version${jackson.version}/version
/dependency
{code}
the base package in the code needs to be updated. More info can be found
[here|http://wiki.fasterxml.com/JacksonUpgradeFrom19To20], I am happy to do
the work just let me know what is preferable i.e. should I just attach a
patch to this issue?
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (AVRO-1387) DataFileWriter must allow files not to be flushed at the end of every block.
[ https://issues.apache.org/jira/browse/AVRO-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13805929#comment-13805929 ] Hudson commented on AVRO-1387: -- SUCCESS: Integrated in AvroJava #399 (See [https://builds.apache.org/job/AvroJava/399/]) AVRO-1387. Java: Add DataFileWriter option to inhibit flush per block. Contributed by Hari Shreedharan. (cutting: rev 1535888) * /avro/trunk/CHANGES.txt * /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/file/DataFileStream.java * /avro/trunk/lang/java/avro/src/main/java/org/apache/avro/file/DataFileWriter.java * /avro/trunk/lang/java/avro/src/test/java/org/apache/avro/TestDataFile.java DataFileWriter must allow files not to be flushed at the end of every block. Key: AVRO-1387 URL: https://issues.apache.org/jira/browse/AVRO-1387 Project: Avro Issue Type: Bug Reporter: Hari Shreedharan Assignee: Hari Shreedharan Fix For: 1.7.6 Attachments: AVRO-1387.patch We are considering changes in Flume's file channel to use Avro, one of the requirements is that each event (which maps to one avro record) be checksummed so we know if the data is corrupt. We'd probably have to add a new version for this, since this will change the data format on disk. I can start working on a Java version if there are no objections -- This message was sent by Atlassian JIRA (v6.1#6144)
