Issue 241330
Summary [LoopVectorize] Loop over std::complex<float> is not vectorized: <2 x float> loads and stores block it
Labels new issue
Assignees
Reporter JeWaVe
    A loop over std::complex<float> is not vectorized at -O3, while the same loop over std::complex<double>, or over C float _Complex, is. The remark gives no reason. GCC vectorizes all of them.

Reproducer (https://godbolt.org/z/no9qM6Yca), clang++ -std=c++20 -O3 -march=x86-64-v4 -Rpass=loop-vectorize -Rpass-missed=loop-vectorize:

#include <complex>
#include <cstddef>
void add(const std::complex<float>* x, std::complex<float>* y, std::size_t n) {
 for (std::size_t i = 0; i < n; ++i) y[i] = x[i] + x[i];
}
void addd(const std::complex<double>* x, std::complex<double>* y, std::size_t n) {
    for (std::size_t i = 0; i < n; ++i) y[i] = x[i] + x[i];
}

<source>:4:5: remark: loop not vectorized [-Rpass-missed=loop-vectorize]
<source>:7:5: remark: vectorized loop (vectorization width: 4, interleaved count: 2) [-Rpass=loop-vectorize]

Seen with clang trunk and 21.1.8, with both libstdc++ and libc++ (-stdlib=libc++). GCC trunk vectorizes both loops with 64-byte vectors.

Each element is loaded, added and stored as one <2 x float>. The loop is only unrolled, by 8:

%28 = load <2 x float>, ptr %27, align 4
%29 = fadd <2 x float> %28, %28
store <2 x float> %29, ptr %30, align 4

Reading and writing the parts as float makes it vectorize, at width 8 here, and at width 16 with -mprefer-vector-width=512:

const float* in = reinterpret_cast<const float*>(x);
float* out = reinterpret_cast<float*>(y);
for (std::size_t i = 0; i < n; ++i) {
 const std::complex<float> r = a * std::complex<float>(in[2 * i], in[2 * i + 1]) + b;
    out[2 * i] = r.real();
    out[2 * i + 1] = r.imag();
}

It looks like the <2 x float> memory accesses, produced before the loop vectorizer runs, are not widened as interleaved accesses. It matters for signal processing code, where std::complex<float> arrays are common. In a polynomial evaluation (a FIR frequency response) the plain loop stays scalar, while the workaround gives 16-wide AVX-512 code.

Possibly related: #198530 (trivially copyable structs and memcpy).
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs

Reply via email to