[
https://issues.apache.org/jira/browse/ARROW-17896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616463#comment-17616463
]
Matthew Topol edited comment on ARROW-17896 at 10/12/22 2:27 PM:
-----------------------------------------------------------------
[~brupp] Can you try upgrading to the newer versions of the library such as
`github.com/apache/arrow/go/v9/parquet` instead of the old version you're using
and see if you can reproduce the issue?
I'm fairly certain you're hitting a bug that was already addressed and fixed in
one of the earlier versions.
was (Author: zeroshade):
[~brupp] Can you try upgrading to the newer versions of the library such as
`github.com/apache/arrow/go/v9/parquet` instead of the old version you're using
and see if you can reproduce the issue?
> [Go][Parquet] panic: fatal error: found bad pointer in Go heap on
> ByteArrayColumnChunkReader.Skip
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-17896
> URL: https://issues.apache.org/jira/browse/ARROW-17896
> Project: Apache Arrow
> Issue Type: Bug
> Reporter: Ben
> Priority: Major
> Labels: go, parquet
>
> {code:java}
> runtime: pointer 0xc001470001 to unused region of span
> span.base()=0xc000e78000 span.limit=0xc000e7a000 span.state=1
> fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)
> runtime stack:
> runtime.throw({0x15ac9e9?, 0x3?})
> /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc00010fee0
> sp=0xc00010feb0 pc=0x438351
> runtime.badPointer(0x7f16917f4d70, 0xc00010ff80?, 0x0, 0xc0006ef1e0?)
> /usr/local/go/src/runtime/mbitmap.go:368 +0x150 fp=0xc00010ff30
> sp=0xc00010fee0 pc=0x414490
> runtime.findObject(0xc000193a00?, 0xc0006ef1e0?, 0xc00010ffb0?)
> /usr/local/go/src/runtime/mbitmap.go:410 +0xa6 fp=0xc00010ff68
> sp=0xc00010ff30 pc=0x414626
> runtime.wbBufFlush1(0xc000066000)
> /usr/local/go/src/runtime/mwbbuf.go:260 +0xe5 fp=0xc00010ffb0
> sp=0xc00010ff68 pc=0x433525
> runtime.wbBufFlush.func1()
> /usr/local/go/src/runtime/mwbbuf.go:201 +0x25 fp=0xc00010ffc8
> sp=0xc00010ffb0 pc=0x433365
> runtime.systemstack()
> /usr/local/go/src/runtime/asm_amd64.s:469 +0x49 fp=0xc00010ffd0
> sp=0xc00010ffc8 pc=0x468ea9
> goroutine 27 [running]:
> runtime.systemstack_switch()
> /usr/local/go/src/runtime/asm_amd64.s:436 fp=0xc0007790c0
> sp=0xc0007790b8 pc=0x468e40
> runtime.wbBufFlush(0x0?, 0x0?)
> /usr/local/go/src/runtime/mwbbuf.go:200 +0x6c fp=0xc0007790e0
> sp=0xc0007790c0 pc=0x4333ec
> runtime.wbBufFlush(0xc00091e190, 0xc0014870eb)
> <autogenerated>:1 +0x2a fp=0xc000779100 sp=0xc0007790e0 pc=0x46d3aa
> runtime.gcWriteBarrier()
> /usr/local/go/src/runtime/asm_amd64.s:1669 +0xa3 fp=0xc000779180
> sp=0xc000779100 pc=0x46b003
> runtime.gcWriteBarrierDX()
> /usr/local/go/src/runtime/asm_amd64.s:1697 +0x7 fp=0xc000779188
> sp=0xc000779180 pc=0x46b087
> github.com/apache/arrow/go/parquet/internal/encoding.(*ByteArrayDictConverter).Copy(0xc00091ca80?,
> {0x1252500?, 0xc0006c6000?}, {0xc0005e7000?, 0x400?, 0x0?})
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/internal/encoding/typed_encoder.gen.go:1344
> +0xa7 fp=0xc0007791b0 sp=0xc000779188 pc=0xbde387
> github.com/apache/arrow/go/parquet/internal/utils.(*RleDecoder).GetBatchWithDictByteArray(0xc000276500,
> {0x1766bf8, 0xc000505140}, {0xc00091ca80, 0x229, 0x400})
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/internal/utils/typed_rle_dict.gen.go:1168
> +0x1d0 fp=0xc000779238 sp=0xc0007791b0 pc=0xbac590
> github.com/apache/arrow/go/parquet/internal/utils.(*RleDecoder).GetBatchWithDict(0x40a4cd?,
> {0x1766bf8?, 0xc000505140?}, {0x1252500?, 0xc0007792e0?})
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/internal/utils/rle.go:422
> +0x1de fp=0xc000779290 sp=0xc000779238 pc=0xba4bbe
> github.com/apache/arrow/go/parquet/internal/encoding.(*dictDecoder).decode(...)
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/internal/encoding/decoder.go:140
> github.com/apache/arrow/go/parquet/internal/encoding.(*DictByteArrayDecoder).Decode(0xc00114b620?,
> {0xc00091ca80?, 0xc000276500?, 0x41fc10?})
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/internal/encoding/typed_encoder.gen.go:1250
> +0x6d fp=0xc000779308 sp=0xc000779290 pc=0xbdd9cd
> github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).ReadBatch.func1(0x0,
> 0x229)
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/file/column_reader_types.gen.go:263
> +0xa6 fp=0xc000779350 sp=0xc000779308 pc=0xbf0e86
> github.com/apache/arrow/go/parquet/file.(*columnChunkReader).readBatch(0xc000e4aa00,
> 0x229, {0xc00091ca80, 0x3000, 0x3000}, {0xc00091ca80, 0x3000, 0x3000},
> 0xc000779440)
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/file/column_reader.go:485
> +0xcd fp=0xc0007793f8 sp=0xc000779350 pc=0xbeec4d
> github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).ReadBatch(0x0?,
> 0x7f16b85f9108?, {0xc00091ca80?, 0xc000580000?, 0x203000?}, {0xc00091ca80?,
> 0xc000779538?, 0x46b107?}, {0xc00091ca80, 0x3000, ...})
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/file/column_reader_types.gen.go:262
> +0x87 fp=0xc000779478 sp=0xc0007793f8 pc=0xbf0d67
> github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).Skip.func1(0xc001054b40?,
> {0xc00091ca80?, 0x0?, 0x0?})
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/file/column_reader_types.gen.go:244
> +0x1f8 fp=0xc000779570 sp=0xc000779478 pc=0xbf0c78
> github.com/apache/arrow/go/parquet/file.(*columnChunkReader).skipValues(0xc000e4aa00,
> 0x629, 0xc000779608)
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/file/column_reader.go:442
> +0x2a5 fp=0xc0007795f0 sp=0xc000779570 pc=0xbeeac5
> github.com/apache/arrow/go/parquet/file.(*ByteArrayColumnChunkReader).Skip(0x10?,
> 0x176e738?)
>
> /go/pkg/mod/github.com/apache/arrow/go/[email protected]/file/column_reader_types.gen.go:242
> +0x37 fp=0xc000779628 sp=0xc0007795f0 pc=0xbf0a57 {code}
> I occasionally (but frequently) hit the above, +un-recoverable+ panic when
> parsing parquet files.
> It does not look like this has anything to do with the parquet file itself,
> as repeatedly parsing the same file works out fine.
>
> Using *github.com/apache/arrow/go/parquet v0.0.0-20211112161151-bc219186db40*
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
