[jira] [Commented] (PARQUET-415) ByteBufferBackedBinary serialization is broken
[
https://issues.apache.org/jira/browse/PARQUET-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433287#comment-16433287
]
ASF GitHub Bot commented on PARQUET-415:
jinxing64 commented on issue #305: PARQUET-415: Fix ByteBuffer Binary
serialization.
URL: https://github.com/apache/parquet-mr/pull/305#issuecomment-380301709
@rdblue
Thanks a lot !
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ByteBufferBackedBinary serialization is broken
> --
>
> Key: PARQUET-415
> URL: https://issues.apache.org/jira/browse/PARQUET-415
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
>Affects Versions: 1.8.1
>Reporter: Ryan Blue
>Assignee: Ryan Blue
>Priority: Blocker
> Fix For: 1.9.0
>
>
> While working on logical types for parquet-avro, I updated the
> {{FromStringBinary}} to work with a {{CharSequence}} instead of a {{String}},
> which broke {{TestFilterApiMethods#testSerializable}}. The underlying problem
> is that when the buffer-backed binary is deserialized, {{length}} and
> {{offset}} are not initialized so the buffer is correct but the apparent
> length of the binary is 0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PARQUET-415) ByteBufferBackedBinary serialization is broken
[
https://issues.apache.org/jira/browse/PARQUET-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432466#comment-16432466
]
ASF GitHub Bot commented on PARQUET-415:
rdblue commented on issue #305: PARQUET-415: Fix ByteBuffer Binary
serialization.
URL: https://github.com/apache/parquet-mr/pull/305#issuecomment-380143377
@jinxing64, this is fixed in 1.9.0. It could result in the trace you pasted,
I think. I'm not sure if that's actually happening though.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ByteBufferBackedBinary serialization is broken
> --
>
> Key: PARQUET-415
> URL: https://issues.apache.org/jira/browse/PARQUET-415
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
>Affects Versions: 1.8.1
>Reporter: Ryan Blue
>Assignee: Ryan Blue
>Priority: Blocker
> Fix For: 1.9.0
>
>
> While working on logical types for parquet-avro, I updated the
> {{FromStringBinary}} to work with a {{CharSequence}} instead of a {{String}},
> which broke {{TestFilterApiMethods#testSerializable}}. The underlying problem
> is that when the buffer-backed binary is deserialized, {{length}} and
> {{offset}} are not initialized so the buffer is correct but the apparent
> length of the binary is 0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PARQUET-415) ByteBufferBackedBinary serialization is broken
[
https://issues.apache.org/jira/browse/PARQUET-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432019#comment-16432019
]
ASF GitHub Bot commented on PARQUET-415:
jinxing64 commented on issue #305: PARQUET-415: Fix ByteBuffer Binary
serialization.
URL: https://github.com/apache/parquet-mr/pull/305#issuecomment-380045863
@rdblue
Thanks for fixing this problem, may I ask a question?
1. What parquet version includes this change? is it 1.9.0?
2. Without this change, what exception will be thrown? is it below?
```
Caused by: java.nio.BufferUnderflowException
at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:151)
at java.nio.ByteBuffer.get(ByteBuffer.java:715)
at
org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.getBytes(Binary.java:405)
at
org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.getBytesUnsafe(Binary.java:414)
at
org.apache.parquet.io.api.Binary$ByteBufferBackedBinary.writeObject(Binary.java:484)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1028)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
..
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> ByteBufferBackedBinary serialization is broken
> --
>
> Key: PARQUET-415
> URL: https://issues.apache.org/jira/browse/PARQUET-415
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
>Affects Versions: 1.8.1
>Reporter: Ryan Blue
>Assignee: Ryan Blue
>Priority: Blocker
> Fix For: 1.9.0
>
>
> While working on logical types for parquet-avro, I updated the
> {{FromStringBinary}} to work with a {{CharSequence}} instead of a {{String}},
> which broke {{TestFilterApiMethods#testSerializable}}. The underlying problem
> is that when the buffer-backed binary is deserialized, {{length}} and
> {{offset}} are not initialized so the buffer is correct but the apparent
> length of the binary is 0.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (PARQUET-415) ByteBufferBackedBinary serialization is broken
[
https://issues.apache.org/jira/browse/PARQUET-415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068891#comment-15068891
]
Ryan Blue commented on PARQUET-415:
---
Fixed in PR #305. Luckily, no other Binary classes failed the new serialization
test.
> ByteBufferBackedBinary serialization is broken
> --
>
> Key: PARQUET-415
> URL: https://issues.apache.org/jira/browse/PARQUET-415
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
>Affects Versions: 1.8.1
>Reporter: Ryan Blue
>Assignee: Ryan Blue
>Priority: Blocker
> Fix For: 1.9.0
>
>
> While working on logical types for parquet-avro, I updated the
> {{FromStringBinary}} to work with a {{CharSequence}} instead of a {{String}},
> which broke {{TestFilterApiMethods#testSerializable}}. The underlying problem
> is that when the buffer-backed binary is deserialized, {{length}} and
> {{offset}} are not initialized so the buffer is correct but the apparent
> length of the binary is 0.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
