Tim Armstrong created IMPALA-9690:
-------------------------------------
Summary: Bump minimum x86-64 CPU requirements
Key: IMPALA-9690
URL: https://issues.apache.org/jira/browse/IMPALA-9690
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Tim Armstrong
Assignee: Tim Armstrong
We still have a minimum CPU requirement of SSSE3 support
https://impala.apache.org/docs/build/html/topics/impala_prereqs.html. I.e. we
don't assume SSE4.2 or AVX or AVX2.
There is a lot of legacy code to support CPUs without SSE4.2 and various other
extensions. As a start, here are all the locations in the code where we branch
based on CPU feature:
:~/impala/impala$ git grep CpuInfo::IsSupport
be/src/benchmarks/int-hash-benchmark.cc: if
(CpuInfo::IsSupported(CpuInfo::SSE4_2)) suite32.BENCH(uint32_t, CRC);
be/src/benchmarks/int-hash-benchmark.cc: if
(CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
be/src/benchmarks/int-hash-benchmark.cc: if
(CpuInfo::IsSupported(CpuInfo::SSE4_1)) {
be/src/benchmarks/int-hash-benchmark.cc: if
(CpuInfo::IsSupported(CpuInfo::AVX2)) {
be/src/benchmarks/string-compare-benchmark.cc: if
(CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
be/src/exec/delimited-text-parser.cc: if
(CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
be/src/exec/delimited-text-parser.cc: if
(CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
be/src/exec/delimited-text-parser.inline.h:
DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/exec/delimited-text-parser.inline.h: if
(LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
be/src/runtime/io/disk-io-mgr.cc: if (!CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::SSSE3)) {
be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(cpu_info_flag)) {
be/src/util/bit-util-test.cc:// CpuInfo::IsSupported() checks. This doesn't
test the bug precisely but is a canary for
be/src/util/bit-util.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
be/src/util/bit-util.cc: } else if
(LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) {
be/src/util/bit-util.cc: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) {
be/src/util/bit-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::POPCNT))) {
be/src/util/bloom-filter.cc: if (CpuInfo::IsSupported(CpuInfo::AVX)) {
be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
be/src/util/cpu-info.cc: if (!CpuInfo::IsSupported(CpuInfo::SSSE3)) {
be/src/util/cpu-info.h: /// // line, CpuInfo::IsSupported(CpuInfo::AVX2)
will return false.
be/src/util/cpu-info.h: : feature_(feature),
reenable_(CpuInfo::IsSupported(feature)) {
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
be/src/util/hash-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
be/src/util/openssl-util.cc: return
(CpuInfo::IsSupported(CpuInfo::PCLMULQDQ)
{noformat}
We also ship two versions of the codegen module, one of which (nosse42) is
essentially never used.
I think it would be uncontroversial to bump the minimum requirement to SSE4.2,
which would allow us to delete some old fallbacks. I think the last time Intel
or AMD shipped a processor without this was 2010 or 2011. Jumping to AVX is
probably almost as uncontroversial, since it looks like that has been universal
for nearly as long:
https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX.
Some older lower-performance cloud instance types don't support AVX, it looks
like, but I think this is an edge case.
It would be very nice to require AVX2 because that could remove a bunch of
conditional code (I think the contributors adding ARM support might want to
keep the scalar fallback code though, potentially). It looks like most Intel
and AMD processors have supported it since 2013 and 2015 respectively, except
some low-end Intel processors:
https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2.
In the past we've been a bit conservative about this - people do sometimes use
very old hardware as a test cluster because it was the hardware that they could
get their hands on (e.g. I remember someone complaining several years back that
they couldn't get Impala running on some old AMD CPUs that were missing SSSE3
support).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
