[
https://issues.apache.org/jira/browse/IMPALA-9690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364349#comment-17364349
]
ASF subversion and git services commented on IMPALA-9690:
---------------------------------------------------------
Commit 3076f8a7a92bed21872b3597d24e7ce9b7beedbb in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=3076f8a ]
IMPALA-9690 addendum: Add a legacy AVX mode
Impala 4 introduced a requirement that x86_64 machines
support AVX2. This adds a startup parameter
enable_legacy_avx_support that relaxes the AVX2 requirement
to require only AVX. By default, enable_legacy_avx_support
is false. enable_legacy_avx_support will be removed in a
future release, but it seems like a sensible way to support
older machines for now.
Testing:
- Ran with enable_legacy_avx_support on a pool of machines
that includes machines without AVX2
- Ran a core job on a machine with AVX2
Change-Id: I44051720697a2a497f9017e4b31289be77fb6b05
Reviewed-on: http://gerrit.cloudera.org:8080/17588
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
Reviewed-by: Daniel Becker <[email protected]>
Reviewed-by: Quanlong Huang <[email protected]>
> Bump minimum x86-64 CPU requirements
> ------------------------------------
>
> Key: IMPALA-9690
> URL: https://issues.apache.org/jira/browse/IMPALA-9690
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Tim Armstrong
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: incompatibility, performance
> Fix For: Impala 4.0
>
>
> We still have a minimum CPU requirement of SSSE3 support
> https://impala.apache.org/docs/build/html/topics/impala_prereqs.html. I.e. we
> don't assume SSE4.2 or AVX or AVX2.
> There is a lot of legacy code to support CPUs without SSE4.2 and various
> other extensions. As a start, here are all the locations in the code where we
> branch based on CPU feature:
> {noformat}
> :~/impala/impala$ git grep CpuInfo::IsSupport
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) suite32.BENCH(uint32_t, CRC);
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_1)) {
> be/src/benchmarks/int-hash-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/benchmarks/string-compare-benchmark.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.cc: if
> (CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/exec/delimited-text-parser.inline.h:
> DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/exec/delimited-text-parser.inline.h: if
> (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
> be/src/runtime/io/disk-io-mgr.cc: if
> (!CpuInfo::IsSupported(CpuInfo::SSE4_2)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::SSSE3)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bit-util-test.cc: if (CpuInfo::IsSupported(cpu_info_flag)) {
> be/src/util/bit-util-test.cc:// CpuInfo::IsSupported() checks. This doesn't
> test the bug precisely but is a canary for
> be/src/util/bit-util.cc: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bit-util.cc: } else if
> (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3))) {
> be/src/util/bit-util.cc: if (LIKELY(CpuInfo::IsSupported(CpuInfo::SSSE3)))
> {
> be/src/util/bit-util.h: if (LIKELY(CpuInfo::IsSupported(CpuInfo::POPCNT)))
> {
> be/src/util/bloom-filter.cc: if (CpuInfo::IsSupported(CpuInfo::AVX)) {
> be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/bloom-filter.h: if (CpuInfo::IsSupported(CpuInfo::AVX2)) {
> be/src/util/cpu-info.cc: if (!CpuInfo::IsSupported(CpuInfo::SSSE3)) {
> be/src/util/cpu-info.h: /// // line, CpuInfo::IsSupported(CpuInfo::AVX2)
> will return false.
> be/src/util/cpu-info.h: : feature_(feature),
> reenable_(CpuInfo::IsSupported(feature)) {
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: DCHECK(CpuInfo::IsSupported(CpuInfo::SSE4_2));
> be/src/util/hash-util.h: if
> (LIKELY(CpuInfo::IsSupported(CpuInfo::SSE4_2))) {
> be/src/util/openssl-util.cc: return
> (CpuInfo::IsSupported(CpuInfo::PCLMULQDQ)
> {noformat}
> We also ship two versions of the codegen module, one of which (nosse42) is
> essentially never used.
> I think it would be uncontroversial to bump the minimum requirement to
> SSE4.2, which would allow us to delete some old fallbacks. I think the last
> time Intel or AMD shipped a processor without this was 2010 or 2011. Jumping
> to AVX is probably almost as uncontroversial, since it looks like that has
> been universal for nearly as long:
> https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX.
> Some older lower-performance cloud instance types don't support AVX, it looks
> like, but I think this is an edge case.
> It would be very nice to require AVX2 because that could remove a bunch of
> conditional code (I think the contributors adding ARM support might want to
> keep the scalar fallback code though, potentially). It looks like most Intel
> and AMD processors have supported it since 2013 and 2015 respectively, except
> some low-end Intel processors:
> https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2.
> In the past we've been a bit conservative about this - people do sometimes
> use very old hardware as a test cluster because it was the hardware that they
> could get their hands on (e.g. I remember someone complaining several years
> back that they couldn't get Impala running on some old AMD CPUs that were
> missing SSSE3 support).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
