[jira] [Commented] (ARROW-1171) [C++] Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682656#comment-16682656
]
Uwe L. Korn commented on ARROW-1171:
Yes, this is reproducible and probably not fixable.
> [C++] Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
> -
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1171) [C++] Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682604#comment-16682604
]
Wes McKinney commented on ARROW-1171:
-
Is this reproducible?
> [C++] Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
> -
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432580#comment-16432580
]
Uwe L. Korn commented on ARROW-1171:
> I'm assuming the issue is that the libstdc++ Arrow was (statically or not)
> linked with also gets used at runtime to resolve symbols in turbodbc (or the
> reverse)? Am I right?
Yes. There are some symbols that are in the C++ STL that are always statically
linked. We provide a version script and don't export our symbols but sadly
other libraries export theirs and this then leads to a crash in Arrow.
Using {{-Bsymbolic}} (and friends!) would be the path that would ensure that
Arrow actually uses the symbols that were packaged with it. Sadly this has some
nasty side effects. When I have some more time, I try to dive into this issue
again and read also the linked articles. Currently this problem here is no a
high priority for me.
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432484#comment-16432484
]
Antoine Pitrou commented on ARROW-1171:
---
I'm assuming the issue is that the libstdc++ Arrow was (statically or not)
linked with also gets used at runtime to resolve symbols in turbodbc (or the
reverse)? Am I right?
Apparently another solution would be to use something called a "version script"
to control which symbols are exported from the Arrow library, so that the
static libstdc++ doesn't participate in dynamic symbol resolution:
[https://www.technovelty.org/c/what-exactly-does-bsymblic-do.html]
[http://anadoxin.org/blog/control-over-symbol-exports-in-gcc.html]
(I may be speaking cluelessly here)
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432457#comment-16432457
]
Uwe L. Korn commented on ARROW-1171:
It will not end the issues but help us to move to a more modern compiler for
the {{manylinux1}} packages. We will still get issues when the wrong symbols
are pulled in, probably for a lot less symbols than currently.
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432452#comment-16432452
]
Antoine Pitrou commented on ARROW-1171:
---
As a sidenote, a new manylinux standard is being developed with a more recent
baseline (CentOS 6) :
https://www.python.org/dev/peps/pep-0571/
The PEP is still being discussed but chances are it will be accepted soon:
https://mail.python.org/pipermail/distutils-sig/2018-March/032102.html
Perhaps it will end the libstdc++ issues (or perhaps not :-)).
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>Priority: Major
> Labels: pull-request-available
> Fix For: 0.10.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315429#comment-16315429
]
ASF GitHub Bot commented on ARROW-1171:
---
xhochy commented on issue #1464: [WIP] ARROW-1171: Statically link libstdc++,
use -BSymbolic
URL: https://github.com/apache/arrow/pull/1464#issuecomment-355844955
This should actually fix most of the compiler/runtime problems we are seeing
with the manylinux packages, sadly it crashes in the Decimal code (C++
exceptions are thrown in various locations). I revisited this from time to time
but cannot find the root cause.
@wesm @pcmoritz feel free to take a shot at this, I have no ideas anymore
what‘s going wrong here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315428#comment-16315428
]
ASF GitHub Bot commented on ARROW-1171:
---
xhochy opened a new pull request #1464: [WIP] ARROW-1171: Statically link
libstdc++, use -BSymbolic
URL: https://github.com/apache/arrow/pull/1464
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16272357#comment-16272357
]
Uwe L. Korn commented on ARROW-1171:
Moved to 0.9.0 as the decimal code throws exceptions once I add {{-Bsymbolic}}
to the linker options. This seems to take much longer to correctly understand
the varying behaviour.
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
> Fix For: 0.9.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168305#comment-16168305
]
Uwe L. Korn commented on ARROW-1171:
{{-DCMAKE_CXX_FLAGS='-Wl,-Bsymbolic -static-libstdc++'}} seems to solve the
issue, marking this for the next release. I'll not do a direct PR but rather
understand the effect a bit better.
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
> Fix For: 0.8.0
>
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165310#comment-16165310
]
Uwe L. Korn commented on ARROW-1171:
Stock GCC, so there is a version mismatch. I learned a bit more about
`-Bsymbolic` and its friends, giving this a shot to see if that fixes it.
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165289#comment-16165289
]
Wes McKinney commented on ARROW-1171:
-
Is the self-compiling Fedora 24 user using devtoolset or stock gcc?
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16165283#comment-16165283
]
Uwe L. Korn commented on ARROW-1171:
For reference, an import of this mixture (pyarrow-manylinux1 0.6.0 &
turbodbc-self-compiled 2.2.0 on Fedora 24) fails with
{code}
Thread 1 (Thread 0x7f927a5db700 (LWP 21)):
#0 0x7f9278df4110 in vtable for arrow::StringType () from
/venv/lib/python2.7/site-packages/pyarrow/libarrow.so.0
#1 0x7f92789d2069 in
std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() () from
/venv/lib/python2.7/site-packages/pyarrow/libarrow.so.0
#2 0x7f92792cc948 in __run_exit_handlers (status=0, listp=0x7f92796505d8
<__exit_funcs>, run_list_atexit=run_list_atexit@entry=true) at exit.c:82
#3 0x7f92792cc995 in __GI_exit (status=) at exit.c:104
#4 0x7f92792b3738 in __libc_start_main (main=0x563816bf87b0 ,
argc=3, argv=0x7ffc0b05e9f8, init=, fini=,
rtld_fini=,
stack_end=0x7ffc0b05e9e8) at ../csu/libc-start.c:323
#5 0x563816bf87e9 in _start ()
{code}
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070090#comment-16070090
]
Uwe L. Korn commented on ARROW-1171:
No, we already disabled that. With enabled gcc5 ABI, it won't even link.
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (ARROW-1171) C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and self-compiled turbodbc
[
https://issues.apache.org/jira/browse/ARROW-1171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16070087#comment-16070087
]
Wes McKinney commented on ARROW-1171:
-
Seems like disabling the gcc5 ABI may be the solution?
> C++: Segmentation faults on Fedora 24 with pyarrow-manylinux1 and
> self-compiled turbodbc
>
>
> Key: ARROW-1171
> URL: https://issues.apache.org/jira/browse/ARROW-1171
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
>Affects Versions: 0.4.1
>Reporter: Uwe L. Korn
>Assignee: Uwe L. Korn
>
> Original issue: https://github.com/blue-yonder/turbodbc/issues/102
> When using the {{pyarrow}} {{manylinux1}} Wheels to build Turbodbc on Fedora
> 24, the {{turbodbc_arrow}} unittests segfault. The main environment attribute
> here is that the compiler version used for building Turbodbc is newer than
> the one used for Arrow.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
