[
https://issues.apache.org/jira/browse/AVRO-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12910239#action_12910239
]
Scott Carey commented on AVRO-668:
----------------------------------
As far as performance goes, I have not set up a benchmark. I did note the
String > Utf8 > byte[] > outputStream garbage creation in a profiler of my app.
But reducing garbage almost never does much in a microbenchmark, it is more of
a system level optimization when you've got lots of code churning the GC. It
might be useful for:
http://wiki.github.com/eishay/jvm-serializers/ though. Anything that makes
String > out and in > String more streamlined will help that benchmark.
* The most recent patch compiles. I failed to include it the first time.
* I'll change it to UTF8_CS
I can't run all java tests from ant on trunk. Not sure if its my fault or not
but it hangs forever in:
[junit] Running org.apache.avro.TestProtocolHttp
[junit] 5455 [main] INFO org.apache.avro.ipc.DatagramTransceiver - sent to
/127.0.0.1:15276
The test prior fails too:
[junit] Running org.apache.avro.TestProtocolGenericMeta
[junit] 5427 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketTransceiver - open to /10.0.0.231:65043
[junit] 5429 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - stopping /0.0.0.0
[junit] 5427 [main] INFO org.apache.avro.ipc.SocketTransceiver - open to
0.0.0.0/0.0.0.0:65040
[junit] 5441 [Connection to /10.0.0.231:65043] INFO
org.apache.avro.TestProtocolGeneric - hello: bob
[junit] 5441 [main] INFO org.apache.avro.ipc.SocketTransceiver - closing to
0.0.0.0/0.0.0.0:65040
[junit] Tests run: 6, Failures: 0, Errors: 5, Time elapsed: 0.071 sec
with 5 errors similar to:
Testcase: testEcho took 0 sec
Caused an ERROR
null
java.nio.channels.ClosedChannelException
at
sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:126)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
at
org.apache.avro.ipc.SocketTransceiver.writeLength(SocketTransceiver.java:94)
at
org.apache.avro.ipc.SocketTransceiver.writeBuffers(SocketTransceiver.java:84)
at org.apache.avro.ipc.Transceiver.transceive(Transceiver.java:38)
at org.apache.avro.ipc.Requestor.request(Requestor.java:123)
at
org.apache.avro.generic.GenericRequestor.request(GenericRequestor.java:43)
at
org.apache.avro.TestProtocolGeneric.testEcho(TestProtocolGeneric.java:132)
Either that is my setup, or a different JIRA to make that work. Or its just
more stupid Mac crap causing trouble.
If I run TestBlockIO2 by itself, I can see the error you mention but I'm
puzzled by what that might be at the moment. Maybe its because I switched from
the deprecated Utf8 getLength() to getByteLength()?
I think providing both writeString(Utf8) and writeString(CharSequence) at the
Encoder level would be useful. However thats an API change that would have to
wait until 1.5. That is not so bad I guess this isn't a huge performance
benefit, just something that combined with 4 or 5 other little things would add
up.
> Java: Streamline writing of Strings for Encoders and GenericDatumWriter
> -----------------------------------------------------------------------
>
> Key: AVRO-668
> URL: https://issues.apache.org/jira/browse/AVRO-668
> Project: Avro
> Issue Type: Improvement
> Reporter: Scott Carey
> Assignee: Scott Carey
> Fix For: 1.4.1
>
> Attachments: AVRO-668.patch
>
>
> We can streamline writing of strings to minimize object creation during
> writes.
> We can avoid converting a String into Utf8 for Json output, and for Binary
> output we can avoid a Utf8 (but still create a byte[]).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
