[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16879051#comment-16879051
]
Tham commented on ARROW-5410:
-
[~pitrou] after disable optimization on arrow build, I don't meet this issue
anymore. It seems there is something wrong with optimizer. I'll tell you if I
can sure about that.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876852#comment-16876852
]
Tham commented on ARROW-5410:
-
Ok. I will.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876849#comment-16876849
]
Antoine Pitrou commented on ARROW-5410:
---
Ok, I'm gonna close this issue. Out of curiosity though, it would be nice to
post here when you finally diagnose the underlying cause.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876847#comment-16876847
]
Tham commented on ARROW-5410:
-
Yes, I think it's highly something with our project build. Thanks so much for
your support.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876845#comment-16876845
]
Antoine Pitrou commented on ARROW-5410:
---
So it seems to have something to do with the Windows CRT, or (perhaps more
likely) the way it is used in your project... Last try: are you using the same
version of the CRT (debug/release, static/dynamic) from both projects? See e.g.
https://docs.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-library?view=vs-2019
(I think a compiler bug is extremely unlikely on such a simple function, and by
extremely I mean something like 0.0001% probability; also the ASM looks sound)
After that, I don't think I'm able to help anymore. You'll probably have to
find a Windows expert in-house...
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876841#comment-16876841
]
Tham commented on ARROW-5410:
-
> can you try passing {{-DARROW_USE_STATIC_CRT=on}} when building the Arrow and
>Parquet static libraries?
I've tried this.
-DARROW_USE_STATIC_CRT=on, it always crash.
-DARROW_USE_STATIC_CRT=off, it crashes on some Windows version.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876840#comment-16876840
]
Antoine Pitrou commented on ARROW-5410:
---
Last idea: can you try passing {{-DARROW_USE_STATIC_CRT=on}} when building the
Arrow and Parquet static libraries?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876837#comment-16876837
]
Tham commented on ARROW-5410:
-
> That's weird. Are you using the same compiler version for Arrow and your Qt
>project?
Yes, they are the same. I've checked compiling log, the same cl.exe at path:
C:/Program Files (x86)/Microsoft Visual
Studio/2017/BuildTools/VC/Tools/MSVC/14.13.26128/bin/HostX64/x64/cl.exe.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876819#comment-16876819
]
Antoine Pitrou commented on ARROW-5410:
---
That's weird. Are you using the same compiler version for Arrow and your Qt
project?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876818#comment-16876818
]
Tham commented on ARROW-5410:
-
> Is it easy for you to try using Arrow shared libraries and see what happens?
I've still got the same error. Any idea?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876791#comment-16876791
]
Tham commented on ARROW-5410:
-
> Is it easy for you to try using Arrow shared libraries and see what happens?
Ok, let me try.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876787#comment-16876787
]
Antoine Pitrou commented on ARROW-5410:
---
Explanation for the asm:
* 33152 is probably the value of _O_CREAT | _O_BINARY | _O_NOINHERIT
* "shl eax, 9" takes the value of "bool truncate" and multiplies it by 512
(left-shift by 9); 512 is the value of O_TRUNC
* "or r8d, 8" adds O_APPEND (value 8)
I cannot say for sure, but the code does look correct...
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876785#comment-16876785
]
Antoine Pitrou commented on ARROW-5410:
---
Also:
> I use visual studio 2017 c++ build tools to build arrow, parquet libraries.
> Then I link these static libraries from my qt project.
Is it easy for you to try using Arrow shared libraries and see what happens?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876783#comment-16876783
]
Tham commented on ARROW-5410:
-
Sure. [^io-util-no-printf.asm] here it is.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util-no-printf.asm,
> io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876777#comment-16876777
]
Antoine Pitrou commented on ARROW-5410:
---
Ah, sorry. Can you remove the printf calls and post the disassembly again?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876696#comment-16876696
]
Tham commented on ARROW-5410:
-
[~pitrou] I uploaded 2 assembly files:
[^io-util.asm] is the one I get the issue with oflag.
I don't understand why "shl" is here:
{code:java}
mov eax, ebx
shl eax, 9
add eax, 33152 ; 8180H
mov ebx, eax
{code}
[^io-util-add-more-log.asm]is the one I added more logs, which I didn't get
issue with oflag.
Since I'm not familiar with assembly language, can you help?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
> Attachments: io-util-add-more-log.asm, io-util.asm
>
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876625#comment-16876625
]
Tham commented on ARROW-5410:
-
> Do you use a "normal" toolchain for compiling? i.e. Visual Studio? Or do you
> use custom proprietary tools or libraries?
I use visual studio 2017 c++ build tools to build arrow, parquet libraries.
Then I link these static libraries from my qt project.
> You could try to disassemble the {{FileOpenWritable}} function.
Let me try, I didn't disassemble a function before.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876113#comment-16876113
]
Antoine Pitrou commented on ARROW-5410:
---
> It seems add more logs affects to results. I has still been investigating,
> but do you have any idea why?
There's no reason why. The oflag computation is very simple and depends only on
local variables...
Do you use a "normal" toolchain for compiling? i.e. Visual Studio? Or do you
use custom proprietary tools or libraries?
You could try to disassemble the {{FileOpenWritable}} function.
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
> `000f ` :
> CortexSync!parquet::arrow::FileWriter::WriteTable+0x6cc
> 001e`10eff410 7ff7`7f18baf3 : 0122`bbec39b0 0122`b24c53f8
> `3f80 ` :
> CortexSync!Cortex::Storage::ParquetStreamWriter::writeRowGroup+0x49{code}
> I tried a lot of ways to find out the root cause, but failed. Can anyone here
> give me some information/advice please, so that I can investigate more?
> Thanks!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-5410) [C++] Crash at arrow::internal::FileWrite
[
https://issues.apache.org/jira/browse/ARROW-5410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876085#comment-16876085
]
Tham commented on ARROW-5410:
-
[~pitrou] Maybe the problem come earlier when opening file, not when writing
table data into file. As I add a log before _wsopen_s:
{code:java}
std::cout << "before _wsopen_s() --> oflag:" << oflag << std::endl;
errno_actual = _wsopen_s(fd, file_name.ToNative().c_str(), oflag, _SH_DENYNO,
pmode);{code}
The output of oflag is not the same (I'm sure that I have the same config on my
code):
{code:java}
before _wsopen_s() --> oflag:99209
before _wsopen_s() --> oflag:103305
before _wsopen_s() --> oflag:107401
before _wsopen_s() --> oflag:111497
before _wsopen_s() --> oflag:115593
before _wsopen_s() --> oflag:119689
...
before _wsopen_s() --> oflag:131977 --> write into file will crash after that
{code}
There numbers have step of 4096???
I wanted to know which steps make oflag is like this, then I added more logs,
then oflag is always the same and is exactly below:
{code:java}
after truncate --> oflag:33664
after append --> oflag:33672
after write_only --> oflag:33673
before _wsopen_s() --> oflag:33673
{code}
It seems add more logs affects to results. I has still been investigating, but
do you have any idea why?
> [C++] Crash at arrow::internal::FileWrite
> -
>
> Key: ARROW-5410
> URL: https://issues.apache.org/jira/browse/ARROW-5410
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Environment: Windows version 10.0.14393.0 (rs1_release.160715-1616)
>Reporter: Tham
>Priority: Major
> Labels: parquet
>
> My application is writing a bunch of parquet files and it often crashes. Most
> of the time it crashes when writing the first file, sometimes it can write
> the first file and crashing at the 2nd file. The file can always be opened.
> It only crashes at writeTable.
> As I tested, my application crashes when build with release mode, but don't
> crash with debug mode. It crashed only on one Windows machine, not others.
> Here is stack trace from dump file:
> {code:java}
> STACK_TEXT:
> 001e`10efd840 7ffc`0333d53f : ` 001e`10efe230
> `0033 7ffc`032dbe21 :
> CortexSync!google_breakpad::ExceptionHandler::HandleInvalidParameter+0x1a0
> 001e`10efe170 7ffc`0333d559 : `ff02 7ffc`032da63d
> `0033 `0033 : ucrtbase!invalid_parameter+0x13f
> 001e`10efe1b0 7ffc`03318664 : 7ff7`7f7c8489 `ff02
> 001e`10efe230 `0033 : ucrtbase!invalid_parameter_noinfo+0x9
> 001e`10efe1f0 7ffc`032d926d : ` `0140
> `0005 0122`bbe61e30 :
> ucrtbase!_acrt_uninitialize_command_line+0x6fd4
> 001e`10efe250 7ff7`7f66585e : 0010`0005 `
> 001e`10efe560 0122`b2337b88 : ucrtbase!write+0x8d
> 001e`10efe2a0 7ff7`7f632785 : 7ff7` 7ff7`7f7bb153
> 0122`bbe890e0 001e`10efe634 :
> CortexSync!arrow::internal::FileWrite+0x5e
> 001e`10efe360 7ff7`7f632442 : `348a `0004
> 733f`5e86f38c 0122`bbe14c40 :
> CortexSync!arrow::io::OSFile::Write+0x1d5
> 001e`10efe510 7ff7`7f71c1b9 : 001e`10efe738 7ff7`7f665522
> 0122`bbffe6e0 ` :
> CortexSync!arrow::io::FileOutputStream::Write+0x12
> 001e`10efe540 7ff7`7f79cb2f : 0122`bbe61e30 0122`bbffe6e0
> `0013 001e`10efe730 :
> CortexSync!parquet::ArrowOutputStream::Write+0x39
> 001e`10efe6e0 7ff7`7f7abbaf : 7ff7`7fd75b78 7ff7`7fd75b78
> 001e`10efe9c0 ` :
> CortexSync!parquet::ThriftSerializer::Serialize+0x11f
> 001e`10efe8c0 7ff7`7f7aaf93 : ` 0122`bbe3f450
> `0002 0122`bc0218d0 :
> CortexSync!parquet::SerializedPageWriter::WriteDictionaryPage+0x44f
> 001e`10efee20 7ff7`7f7a3707 : 0122`bbe3f450 001e`10eff250
> ` 0122`b168 :
> CortexSync!parquet::TypedColumnWriterImpl
> >::WriteDictionaryPage+0x143
> 001e`10efeed0 7ff7`7f710480 : 001e`10eff1c0 `
> 0122`bbe3f540 0122`b2439998 :
> CortexSync!parquet::ColumnWriterImpl::Close+0x47
> 001e`10efef60 7ff7`7f7154da : 0122`bbec3cd0 001e`10eff1c0
> 0122`bbec4bb0 0122`b2439998 :
> CortexSync!parquet::arrow::FileWriter::Impl::`vector deleting
> destructor'+0x100
> 001e`10efefa0 7ff7`7f71619c : ` 001e`10eff1c0
> 0122`bbe89390 ` :
> CortexSync!parquet::arrow::FileWriter::Impl::WriteColumnChunk+0x6fa
> 001e`10eff150 7ff7`7f202de9 : `0001 001e`10eff430
>
