[
https://issues.apache.org/jira/browse/IMPALA-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell reassigned IMPALA-9690:
-------------------------------------
Assignee: Joe McDonnell
> Bump minimum x86-64 CPU requirements
> ------------------------------------
>
> Key: IMPALA-9690
> URL: https://issues.apache.org/jira/browse/IMPALA-9690
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: incompatibility, performance
>
> We still have a minimum CPU requirement of SSSE3 support
> https://impala.apache.org/docs/build/html/topics/impala_prereqs.html. I.e. we
> don't assume SSE4.2 or AVX or AVX2.
> There is a lot of legacy code to support CPUs without SSE4.2 and various
> other extensions. As a start, here are all the locations in the code where we
> branch based on CPU feature:
> {noformat}
> :~/impala/impala$ git grep CpuInfo::IsSupport
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) suite32.BENCH(uint32_t, CRC);
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_1)) {
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/benchmarks/string-compare-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.inline.h:
> DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/exec/delimited-text-parser.inline.h: if
> (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
> be/src/runtime/io/disk-io-mgr.cc: if
> (!CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::SSSE3)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(cpu_info_flag)) {
> be/src/util/bit-util-test.cc:// CpuInfo::IsSupported() checks. This doesn't
> test the bug precisely but is a canary for
> be/src/util/bit-util.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bit-util.cc: } else if
> (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) {
> be/src/util/bit-util.cc: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3)))
> {
> be/src/util/bit-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::POPCNT)))
> {
> be/src/util/bloom-filter.cc: if (CpuInfo::IsSupported(CpuInfo::AVX)) {
> be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/cpu-info.cc: if (!CpuInfo::IsSupported(CpuInfo::SSSE3)) {
> be/src/util/cpu-info.h: /// // line, CpuInfo::IsSupported(CpuInfo::AVX2)
> will return false.
> be/src/util/cpu-info.h: : feature_(feature),
> reenable_(CpuInfo::IsSupported(feature)) {
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: if
> (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
> be/src/util/openssl-util.cc: return
> (CpuInfo::IsSupported(CpuInfo::PCLMULQDQ)
> {noformat}
> We also ship two versions of the codegen module, one of which (nosse42) is
> essentially never used.
> I think it would be uncontroversial to bump the minimum requirement to
> SSE4.2, which would allow us to delete some old fallbacks. I think the last
> time Intel or AMD shipped a processor without this was 2010 or 2011. Jumping
> to AVX is probably almost as uncontroversial, since it looks like that has
> been universal for nearly as long:
> https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX.
> Some older lower-performance cloud instance types don't support AVX, it looks
> like, but I think this is an edge case.
> It would be very nice to require AVX2 because that could remove a bunch of
> conditional code (I think the contributors adding ARM support might want to
> keep the scalar fallback code though, potentially). It looks like most Intel
> and AMD processors have supported it since 2013 and 2015 respectively, except
> some low-end Intel processors:
> https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2.
> In the past we've been a bit conservative about this - people do sometimes
> use very old hardware as a test cluster because it was the hardware that they
> could get their hands on (e.g. I remember someone complaining several years
> back that they couldn't get Impala running on some old AMD CPUs that were
> missing SSSE3 support).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
