[
https://issues.apache.org/jira/browse/ARROW-15678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17523302#comment-17523302
]
Weston Pace commented on ARROW-15678:
-------------------------------------
Ok, so I was finally able to track this down. Fortunately (unfortunately?) it
is not really a compiler bug (or maybe it is, I'm not sure). At the very
least, I think we can avoid it.
{{level_comparison.cc}} is compiled with {{-msse4.2}}.
{{level_comparison_avx2.cc}} is compiled with {{-mavx2}}
This is expected and the functions they generate are housed in separate
namespaces so they don't get confused. However, both functions rely on the
function arrow::internal::FirstTimeBitmapWriter::AppendWord. The function is
not templated but it is defined in the header file (and is not marked inline).
I'm not really sure how we aren't getting a duplicate symbol error but some
reading suggests it is implicitly inlined at link time.
In the object file (libparquet.a), there are two identical symbols named
{{__ZN5arrow8internal21FirstTimeBitmapWriter10AppendWordEyx}}. One of them has
{{SHLX}} and one of them has {{SHL}}. This disassembly of the {{SHLX}} version
matches exactly the disassembly in the stack trace that [~jonkeane] posted in
the PR. The two calling functions are
({{parquet::internal::standard::DefLevelsBatchToBitmap}} and
{{parquet::internal::bmi2::DefLevelsBatchToBitmap}}.
So I think, the -O3 version is inlining the functions. The -Os version is not
(-Os seems to discourage inlining in general). The linker is then faced with
two identical symbols and just picks one (again, trying to optimize for size).
It just so happens the version it picked was the one with {{SHLX}}.
So, as a test, we can try splitting the implementation part of
{{bitmap_writer.h}} into {{bitmap_writer.cc}} (at least for
{{FirstTimeBitmapWriter}}). The .cc file should then only be compiled once
(with sse4.2). However, it's very possible we are just hitting the tip of the
iceberg here, as any header file linked in by these avx2 compiled versions
could be a ticking time bomb.
> [C++][CI] a crossbow job with MinRelSize enabled
> ------------------------------------------------
>
> Key: ARROW-15678
> URL: https://issues.apache.org/jira/browse/ARROW-15678
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Continuous Integration
> Reporter: Jonathan Keane
> Priority: Major
> Labels: pull-request-available
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
