arichardson created this revision.
arichardson added reviewers: rsmith, aaron.ballman, theraven, fhahn.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.
This change introduces three new builtins (which work on both pointers
and integers) that can be used instead of common bitwise arithmetic:
__builtin_align_up(x, alignment), __builtin_align_down(x, alignment) and
__builtin_is_aligned(x, alignment).
I originally added these builtins to the CHERI fork of LLVM a few years ago
to handle the slightly different C semantics that we use for CHERI [1].
Until recently these builtins (or sequences of other builtins) were
required to generate correct code. I have since made changes to the default
C semantics so that they are no longer strictly necessary (but using them
does generate slightly more efficient code). However, based on our
experience using them in various projects over the past few years, I
believe that adding these builtins to clang would be useful.
These builtins have the following benefit over bit-manipulation and casts
via uintptr_t:
- The named builtins clearly convey the semantics of the operation. While
checking alignment using __builtin_is_aligned(x, 16) versus ((x & 15) == 0) is
probably not a huge win in readably, I personally find __builtin_align_up(x, N)
a lot easier to read than (x+(N-1))&~(N-1).
- They preserve the type of the argument (including const qualifiers). When
using casts via uintptr_t, it is easy to cast to the wrong type or strip
qualifiers such as const.
- If the alignment argument is a constant value, clang can check that it is a
power-of-two and within the range of the type. Since the semantics of these
builtins is well defined compared to arbitraty bit-manipulation, it is possible
to add a UBSAN checker that the run-time value is a valid power-of-two. I
intend to add this as a follow-up to this change.
- The builtins avoids int-to-pointer casts both in C and LLVM IR. In this patch
I use the new llvm.ptrmask intrinsic to avoid the inttoptr+ptrtoint
instructions that would normally be generated.
- They can be used to round up/down to the next aligned value for both integers
and pointers without requiring two separate macros.
[1] In our CHERI compiler we have compilation mode where all pointers are
implemented as capabilities (essentially unforgeable 128-bit fat pointers).
In our original model, casts from uintptr_t (which is a 128-bit capability)
to an integer value returned the "offset" of the capability (i.e. the
difference between the virtual address and the base of the allocation).
This causes problems for cases such as checking the alignment: for example,
the expression `if ((uintptr_t)ptr & 63) == 0` is generally used to check
if the pointer is aligned to a multiple of 64 bytes. The problem offsets is
that any pointer to the beginning of an allocation will have an offset of
zero, so this check always succeeds in that case (even if the address is
not correctly aligned). The same issues also exist when aligning up or
down. Using the alignment builtins ensures that the address is used instead
of the offset. While I have since changed the default C semantics to return
the address instead of the offset when casting, this offset compilation
mode can still be used by setting with command-line flag.
Repository:
rG LLVM Github Monorepo
https://reviews.llvm.org/D71499
Files:
clang/include/clang/Basic/Builtins.def
clang/include/clang/Basic/DiagnosticSemaKinds.td
clang/lib/AST/ExprConstant.cpp
clang/lib/CodeGen/CGBuiltin.cpp
clang/lib/CodeGen/CodeGenFunction.h
clang/lib/Sema/SemaChecking.cpp
clang/test/CodeGen/builtin-align-array.c
clang/test/CodeGen/builtin-align.c
clang/test/Sema/builtin-align.c
clang/test/SemaCXX/builtin-align-cxx.cpp
Index: clang/test/SemaCXX/builtin-align-cxx.cpp
===================================================================
--- /dev/null
+++ clang/test/SemaCXX/builtin-align-cxx.cpp
@@ -0,0 +1,99 @@
+// C++-specific checks for the alignment builtins
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -std=c++11 -o - %s -fsyntax-only -verify
+
+// Check that we don't crash when using dependent types in __builtin_align:
+template <typename a, a b>
+void *c(void *d) { // expected-note{{candidate template ignored}}
+ return __builtin_align_down(d, b);
+}
+
+struct x {};
+x foo;
+void test(void *value) {
+ c<int, 16>(value);
+ c<struct x, foo>(value); // expected-error{{no matching function for call to 'c'}}
+}
+
+template<typename T, long Alignment, long ArraySize = 16>
+void test_templated_arguments() {
+ T array[ArraySize]; // expected-error{{variable has incomplete type 'fwddecl'}}
+ static_assert(__is_same(decltype(__builtin_align_up(array, Alignment)), T*), // expected-error{{requested alignment is not a power of 2}}
+ "return type should be the decayed array type");
+ static_assert(__is_same(decltype(__builtin_align_down(array, Alignment)), T*),
+ "return type should be the decayed array type");
+ static_assert(__is_same(decltype(__builtin_is_aligned(array, Alignment)), bool),
+ "return type should be bool");
+ T* x1 = __builtin_align_up(array, Alignment);
+ T* x2 = __builtin_align_down(array, Alignment);
+ bool x3 = __builtin_align_up(array, Alignment);
+}
+
+void test() {
+ test_templated_arguments<int, 32>(); // fine
+ test_templated_arguments<struct fwddecl, 16>();
+ // expected-note@-1{{in instantiation of function template specialization 'test_templated_arguments<fwddecl, 16, 16>'}}
+ // expected-note@-2{{forward declaration of 'fwddecl'}}
+ test_templated_arguments<int, 7>(); // invalid alignment value
+ // expected-note@-1{{in instantiation of function template specialization 'test_templated_arguments<int, 7, 16>'}}
+
+}
+
+template<typename T>
+void test_incorrect_alignment_without_instatiation(T value) {
+ int array[32];
+ static_assert(__is_same(decltype(__builtin_align_up(array, 31)), int*), // expected-error{{requested alignment is not a power of 2}}
+ "return type should be the decayed array type");
+ static_assert(__is_same(decltype(__builtin_align_down(array, 7)), int*), // expected-error{{requested alignment is not a power of 2}}
+ "return type should be the decayed array type");
+ static_assert(__is_same(decltype(__builtin_is_aligned(array, -1)), bool), // expected-error{{requested alignment must be 1 or greater}}
+ "return type should be bool");
+ __builtin_align_up(array); // expected-error{{too few arguments to function call, expected 2, have 1}}
+ __builtin_align_up(array, 31); // expected-error{{requested alignment is not a power of 2}}
+ __builtin_align_down(array, 31); // expected-error{{requested alignment is not a power of 2}}
+ __builtin_align_up(array, 31); // expected-error{{requested alignment is not a power of 2}}
+ __builtin_align_up(value, 31); // This shouldn't want since the type is dependent
+ __builtin_align_up(value); // Same here
+}
+
+// The original fix for the issue above broke some legitimate code.
+// Here is a regression test:
+typedef __SIZE_TYPE__ size_t;
+void *allocate_impl(size_t size);
+template <typename T>
+T *allocate() {
+ constexpr size_t allocation_size =
+ __builtin_align_up(sizeof(T), sizeof(void *));
+ return static_cast<T *>(
+ __builtin_assume_aligned(allocate_impl(allocation_size), sizeof(void *)));
+}
+struct Foo {
+ int value;
+};
+void *test2() {
+ return allocate<struct Foo>();
+}
+
+
+// Check that pointers-to-members cannot be used
+class MemPtr {
+public:
+ int data;
+ void func();
+ virtual void vfunc();
+};
+void test_member_ptr() {
+ __builtin_align_up(&MemPtr::data, 64); // expected-error{{operand of type 'int MemPtr::*' where arithmetic or pointer type is required}}
+ __builtin_align_down(&MemPtr::func, 64); // expected-error{{operand of type 'void (MemPtr::*)()' where arithmetic or pointer type is required}}
+ __builtin_is_aligned(&MemPtr::vfunc, 64); // expected-error{{operand of type 'void (MemPtr::*)()' where arithmetic or pointer type is required}}
+}
+
+void test_references(Foo& i) {
+ // Check that the builtins look at the referenced type rather than the reference itself.
+ (void)__builtin_align_up(i, 64); // expected-error{{operand of type 'Foo' where arithmetic or pointer type is required}}
+ (void)__builtin_align_up(static_cast<Foo&>(i), 64); // expected-error{{operand of type 'Foo' where arithmetic or pointer type is required}}
+ (void)__builtin_align_up(static_cast<const Foo&>(i), 64); // expected-error{{operand of type 'const Foo' where arithmetic or pointer type is required}}
+ (void)__builtin_align_up(static_cast<Foo&&>(i), 64); // expected-error{{operand of type 'Foo' where arithmetic or pointer type is required}}
+ (void)__builtin_align_up(static_cast<const Foo&&>(i), 64); // expected-error{{operand of type 'const Foo' where arithmetic or pointer type is required}}
+ (void)__builtin_align_up(&i, 64);
+ __builtin_align_up(&i, 64);
+}
Index: clang/test/Sema/builtin-align.c
===================================================================
--- /dev/null
+++ clang/test/Sema/builtin-align.c
@@ -0,0 +1,133 @@
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -DALIGN_BUILTIN=__builtin_align_down -DRETURNS_BOOL=0 %s -fsyntax-only -verify -Wpedantic
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -DALIGN_BUILTIN=__builtin_align_up -DRETURNS_BOOL=0 %s -fsyntax-only -verify -Wpedantic
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -DALIGN_BUILTIN=__builtin_is_aligned -DRETURNS_BOOL=1 %s -fsyntax-only -verify -Wpedantic
+
+struct Aggregate {
+ int i;
+ int j;
+};
+enum Enum { EnumValue1,
+ EnumValue2 };
+typedef __SIZE_TYPE__ size_t;
+
+void test_parameter_types(char *ptr, size_t size) {
+ struct Aggregate agg;
+ enum Enum e = EnumValue2;
+ _Bool b = 0;
+
+ // first parameter can be any pointer or integer type:
+ (void)ALIGN_BUILTIN(ptr, 4);
+ (void)ALIGN_BUILTIN(size, 2);
+ (void)ALIGN_BUILTIN(12345, 2);
+ (void)ALIGN_BUILTIN(agg, 2); // expected-error {{operand of type 'struct Aggregate' where arithmetic or pointer type is required}}
+ (void)ALIGN_BUILTIN(e, 2); // expected-error {{operand of type 'enum Enum' where arithmetic or pointer type is required}}
+ (void)ALIGN_BUILTIN(b, 2); // expected-error {{operand of type '_Bool' where arithmetic or pointer type is required}}
+ (void)ALIGN_BUILTIN((int)e, 2); // but with a cast it is fine
+ (void)ALIGN_BUILTIN((int)b, 2); // but with a cast it is fine
+
+ // second parameter must be an integer type (but not enum or _Bool)
+ (void)ALIGN_BUILTIN(ptr, size);
+ (void)ALIGN_BUILTIN(ptr, ptr); // expected-error {{used type 'char *' where integer is required}}
+ (void)ALIGN_BUILTIN(ptr, agg); // expected-error {{used type 'struct Aggregate' where integer is required}}
+ (void)ALIGN_BUILTIN(ptr, b); // expected-error {{used type '_Bool' where integer is required}}
+ (void)ALIGN_BUILTIN(ptr, e); // expected-error {{used type 'enum Enum' where integer is required}}
+ (void)ALIGN_BUILTIN(ptr, (int)e); // but with a cast enums are fine
+ (void)ALIGN_BUILTIN(ptr, (int)b); // but with a cast booleans are fine
+
+ (void)ALIGN_BUILTIN(ptr, size);
+ (void)ALIGN_BUILTIN(size, size);
+}
+
+void test_result_unused(int i, int align) {
+ // -Wunused-result does not trigger for macros so we can't use ALIGN_BUILTIN()
+ // but need to call each function here
+ __builtin_align_up(i, align); // expected-warning{{ignoring return value of function declared with const attribute}}
+ __builtin_align_down(i, align); // expected-warning{{ignoring return value of function declared with const attribute}}
+ __builtin_is_aligned(i, align); // expected-warning{{ignoring return value of function declared with const attribute}}
+ ALIGN_BUILTIN(i, align); // no warning here
+}
+
+#define check_same_type(type1, type2) __builtin_types_compatible_p(type1, type2) && __builtin_types_compatible_p(type1*, type2*)
+
+void test_return_type(void *ptr, int i, long l) {
+ char array[32];
+ __extension__ typedef typeof(ALIGN_BUILTIN(ptr, 4)) result_type_ptr;
+ __extension__ typedef typeof(ALIGN_BUILTIN(i, 4)) result_type_int;
+ __extension__ typedef typeof(ALIGN_BUILTIN(l, 4)) result_type_long;
+ __extension__ typedef typeof(ALIGN_BUILTIN(array, 4)) result_type_char_array;
+#if RETURNS_BOOL
+ _Static_assert(check_same_type(_Bool, result_type_ptr), "Should return bool");
+ _Static_assert(check_same_type(_Bool, result_type_int), "Should return bool");
+ _Static_assert(check_same_type(_Bool, result_type_long), "Should return bool");
+ _Static_assert(check_same_type(_Bool, result_type_char_array), "Should return bool");
+#else
+ _Static_assert(check_same_type(void *, result_type_ptr), "Should return void*");
+ _Static_assert(check_same_type(int, result_type_int), "Should return int");
+ _Static_assert(check_same_type(long, result_type_long), "Should return long");
+ // Check that we can use the alignment builtins on on array types (result should decay)
+ _Static_assert(check_same_type(char *, result_type_char_array),
+ "Using the builtins on an array should yield the decayed type");
+#endif
+}
+
+void test_invalid_alignment_values(char *ptr, long * longptr, size_t align) {
+ int x = 1;
+ (void)ALIGN_BUILTIN(ptr, 2);
+ (void)ALIGN_BUILTIN(longptr, 1024);
+ (void)ALIGN_BUILTIN(x, 32);
+
+ (void)ALIGN_BUILTIN(ptr, 0); // expected-error {{requested alignment must be 1 or greater}}
+ (void)ALIGN_BUILTIN(ptr, 1);
+#if RETURNS_BOOL
+ // expected-warning@-2 {{checking whether a value is aligned to 1 byte is always true}}
+#else
+ // expected-warning@-4 {{aligning a value to 1 byte is a no-op}}
+#endif
+ (void)ALIGN_BUILTIN(ptr, 3); // expected-error {{requested alignment is not a power of 2}}
+ (void)ALIGN_BUILTIN(x, 7); // expected-error {{requested alignment is not a power of 2}}
+
+ // check the maximum range for smaller types:
+ __UINT8_TYPE__ c = ' ';
+
+ (void)ALIGN_BUILTIN(c, 128); // this is fine
+ (void)ALIGN_BUILTIN(c, 256); // expected-error {{requested alignment must be 128 or smaller}}
+ (void)ALIGN_BUILTIN(x, 1ULL << 31); // this is also fine
+ (void)ALIGN_BUILTIN(x, 1LL << 31); // this is also fine
+ __INT32_TYPE__ i32 = 3;
+ __UINT32_TYPE__ u32 = 3;
+ // Maximum is the same for int32 and uint32
+ (void)ALIGN_BUILTIN(i32, 1ULL << 32); // expected-error {{requested alignment must be 2147483648 or smaller}}
+ (void)ALIGN_BUILTIN(u32, 1ULL << 32); // expected-error {{requested alignment must be 2147483648 or smaller}}
+ (void)ALIGN_BUILTIN(ptr, ((__int128)1) << 65); // expected-error {{requested alignment must be 9223372036854775808 or smaller}}
+ (void)ALIGN_BUILTIN(longptr, ((__int128)1) << 65); // expected-error {{requested alignment must be 9223372036854775808 or smaller}}
+
+ const int bad_align = 8 + 1;
+ (void)ALIGN_BUILTIN(ptr, bad_align); // expected-error {{requested alignment is not a power of 2}}
+}
+
+// check that it can be used in constant expressions
+void constant_expression(int x) {
+ _Static_assert(__builtin_is_aligned(1024, 512), "");
+ _Static_assert(!__builtin_is_aligned(256, 512ULL), "");
+ _Static_assert(__builtin_align_up(33, 32) == 64, "");
+ _Static_assert(__builtin_align_down(33, 32) == 32, "");
+
+ // but not if one of the arguments isn't constant
+ _Static_assert(ALIGN_BUILTIN(33, x) != 100, ""); // expected-error {{static_assert expression is not an integral constant expression}}
+ _Static_assert(ALIGN_BUILTIN(x, 4) != 100, ""); // expected-error {{static_assert expression is not an integral constant expression}}
+}
+
+// Check that it is a constant expression that can be assigned to globals:
+int global1 = __builtin_align_down(33, 8);
+int global2 = __builtin_align_up(33, 8);
+_Bool global3 = __builtin_is_aligned(33, 8);
+
+extern void test_ptr(char *c);
+char *test_array_and_fnptr(void) {
+ char buf[1024];
+ // The builtins should also work on arrays (decaying the return type)
+ (void)(ALIGN_BUILTIN(buf, 16));
+ // But not on functions and function pointers:
+ (void)(ALIGN_BUILTIN(test_array_and_fnptr, 16)); // expected-error{{operand of type 'char *(void)' where arithmetic or pointer type is required}}
+ (void)(ALIGN_BUILTIN(&test_array_and_fnptr, 16)); // expected-error{{operand of type 'char *(*)(void)' where arithmetic or pointer type is required}}
+}
Index: clang/test/CodeGen/builtin-align.c
===================================================================
--- /dev/null
+++ clang/test/CodeGen/builtin-align.c
@@ -0,0 +1,106 @@
+/// Check the code generation for the alignment builtins
+/// To make the test case easier to read, run SROA after generating IR to remove the alloca instructions.
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_VOID_PTR \
+// RUN: -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN: FileCheck %s -check-prefixes CHECK,POINTER,ALIGNMENT_EXT \
+// RUN: -enable-var-scope '-D$PTRTYPE=i8'
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_FLOAT_PTR \
+// RUN: -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN: FileCheck %s -check-prefixes CHECK,POINTER,NON_I8_POINTER,ALIGNMENT_EXT \
+// RUN: -enable-var-scope '-D$PTRTYPE=f32'
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_LONG \
+// RUN: -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN: FileCheck %s -check-prefixes CHECK,INTEGER,ALIGNMENT_EXT -enable-var-scope
+/// Check that we can handle the case where the alignment parameter is wider
+/// than the source type (generate a trunc on alignment instead of zext)
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -DTEST_USHORT \
+// RUN: -o - -emit-llvm %s -disable-O0-optnone | opt -S -sroa | \
+// RUN: FileCheck %s -check-prefixes CHECK,INTEGER,ALIGNMENT_TRUNC -enable-var-scope
+
+#ifdef TEST_VOID_PTR
+#define TYPE void *
+#elif defined(TEST_FLOAT_PTR)
+#define TYPE float *
+#elif defined(TEST_LONG)
+#define TYPE long
+#elif defined(TEST_CAP)
+#define TYPE void *__capability
+#elif defined(TEST_USHORT)
+#define TYPE unsigned short
+#else
+#error MISSING TYPE
+#endif
+
+/// Check that constant initializers work and are correct
+_Bool aligned_true = __builtin_is_aligned(1024, 512);
+// CHECK: @aligned_true = global i8 1, align 1
+_Bool aligned_false = __builtin_is_aligned(123, 512);
+// CHECK: @aligned_false = global i8 0, align 1
+
+int down_1 = __builtin_align_down(1023, 32);
+// CHECK: @down_1 = global i32 992, align 4
+int down_2 = __builtin_align_down(256, 32);
+// CHECK: @down_2 = global i32 256, align 4
+
+int up_1 = __builtin_align_up(1023, 32);
+// CHECK: @up_1 = global i32 1024, align 4
+int up_2 = __builtin_align_up(256, 32);
+// CHECK: @up_2 = global i32 256, align 4
+
+/// Capture the IR type here to use in the remaining FileCheck captures:
+// CHECK: define {{[^@]+}}@get_type() #0
+// CHECK-NEXT: entry:
+// POINTER-NEXT: ret [[$TYPE:.+]] null
+// INTEGER-NEXT: ret [[$TYPE:.+]] 0
+//
+TYPE get_type(void) {
+ return (TYPE)0;
+}
+
+// CHECK-LABEL: define {{[^@]+}}@is_aligned
+// CHECK-SAME: ([[$TYPE]] {{[^%]*}}[[PTR:%.*]], i32 [[ALIGN:%.*]]) #0
+// CHECK-NEXT: entry:
+// ALIGNMENT_EXT-NEXT: [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to [[ALIGN_TYPE:i64]]
+// ALIGNMENT_TRUNC-NEXT: [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to [[ALIGN_TYPE:i16]]
+// CHECK-NEXT: [[MASK:%.*]] = sub [[ALIGN_TYPE]] [[ALIGNMENT]], 1
+// POINTER-NEXT: [[PTR:%.*]] = ptrtoint [[$TYPE]] %ptr to i64
+// CHECK-NEXT: [[SET_BITS:%.*]] = and [[ALIGN_TYPE]] [[PTR]], [[MASK]]
+// CHECK-NEXT: [[IS_ALIGNED:%.*]] = icmp eq [[ALIGN_TYPE]] [[SET_BITS]], 0
+// CHECK-NEXT: ret i1 [[IS_ALIGNED]]
+//
+_Bool is_aligned(TYPE ptr, unsigned align) {
+ return __builtin_is_aligned(ptr, align);
+}
+
+// CHECK-LABEL: define {{[^@]+}}@align_up
+// CHECK-SAME: ([[$TYPE]] {{[^%]*}}[[PTR:%.*]], i32 [[ALIGN:%.*]]) #0
+// CHECK-NEXT: entry:
+// ALIGNMENT_EXT-NEXT: [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to [[ALIGN_TYPE:i64]]
+// ALIGNMENT_TRUNC-NEXT: [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to [[ALIGN_TYPE:i16]]
+// CHECK-NEXT: [[MASK:%.*]] = sub [[ALIGN_TYPE]] [[ALIGNMENT]], 1
+// NON_I8_POINTER-NEXT: [[PTR:%.*]] = bitcast [[$TYPE]] %ptr to i8*
+// POINTER-NEXT: [[OVER_BOUNDARY:%.*]] = getelementptr i8, i8* [[PTR]], i64 [[MASK]]
+// INTEGER-NEXT: [[OVER_BOUNDARY:%.*]] = add [[$TYPE]] [[PTR]], [[MASK]]
+// CHECK-NEXT: [[NEGATED_MASK:%.*]] = xor [[ALIGN_TYPE]] [[MASK]], -1
+// POINTER-NEXT: [[ALIGNED_RESULT:%.*]] = call [[$TYPE]] @llvm.ptrmask.p0[[$PTRTYPE]].p0i8.i64(i8* [[OVER_BOUNDARY]], [[ALIGN_TYPE]] [[NEGATED_MASK]])
+// INTEGER-NEXT: [[ALIGNED_RESULT:%.*]] = and [[$TYPE]] [[OVER_BOUNDARY]], [[NEGATED_MASK]]
+// CHECK-NEXT: ret [[$TYPE]] [[ALIGNED_RESULT]]
+//
+TYPE align_up(TYPE ptr, unsigned align) {
+ return __builtin_align_up(ptr, align);
+}
+
+// CHECK-LABEL: define {{[^@]+}}@align_down
+// CHECK-SAME: ([[$TYPE]] {{[^%]*}}[[PTR:%.*]], i32 [[ALIGN:%.*]]) #0
+// CHECK-NEXT: entry:
+// ALIGNMENT_EXT-NEXT: [[ALIGNMENT:%.*]] = zext i32 [[ALIGN]] to [[ALIGN_TYPE:i64]]
+// ALIGNMENT_TRUNC-NEXT: [[ALIGNMENT:%.*]] = trunc i32 [[ALIGN]] to [[ALIGN_TYPE:i16]]
+// CHECK-NEXT: [[MASK:%.*]] = sub [[ALIGN_TYPE]] [[ALIGNMENT]], 1
+// CHECK-NEXT: [[NEGATED_MASK:%.*]] = xor [[ALIGN_TYPE]] [[MASK]], -1
+// POINTER-NEXT: [[ALIGNED_RESULT:%.*]] = call [[$TYPE]] @llvm.ptrmask.p0[[$PTRTYPE]].p0[[$PTRTYPE]].i64([[$TYPE]] [[PTR]], [[ALIGN_TYPE]] [[NEGATED_MASK]])
+// INTEGER-NEXT: [[ALIGNED_RESULT:%.*]] = and [[$TYPE]] [[PTR]], [[NEGATED_MASK]]
+// CHECK-NEXT: ret [[$TYPE]] [[ALIGNED_RESULT]]
+//
+TYPE align_down(TYPE ptr, unsigned align) {
+ return __builtin_align_down(ptr, align);
+}
Index: clang/test/CodeGen/builtin-align-array.c
===================================================================
--- /dev/null
+++ clang/test/CodeGen/builtin-align-array.c
@@ -0,0 +1,55 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+/// Check that the alignment builtins handle array-to-pointer decay
+// RUN: %clang_cc1 -triple=x86_64-unknown-unknown -o - -emit-llvm %s | FileCheck %s
+
+extern int func(char *c);
+
+// CHECK-LABEL: define {{[^@]+}}@test_array() #0
+// CHECK-NEXT: entry:
+// CHECK-NEXT: [[BUF:%.*]] = alloca [1024 x i8], align 16
+// CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 44
+// CHECK-NEXT: [[ALIGNED_RESULT:%.*]] = call i8* @llvm.ptrmask.p0i8.p0i8.i64(i8* [[ARRAYIDX]], i64 -16)
+// CHECK-NEXT: [[CALL:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT]])
+// CHECK-NEXT: [[ARRAYIDX1:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 22
+// CHECK-NEXT: [[OVER_BOUNDARY:%.*]] = getelementptr i8, i8* [[ARRAYIDX1]], i64 31
+// CHECK-NEXT: [[ALIGNED_RESULT2:%.*]] = call i8* @llvm.ptrmask.p0i8.p0i8.i64(i8* [[OVER_BOUNDARY]], i64 -32)
+// CHECK-NEXT: [[CALL3:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT2]])
+// CHECK-NEXT: [[ARRAYIDX4:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 16
+// CHECK-NEXT: [[SRC_ADDR:%.*]] = ptrtoint i8* [[ARRAYIDX4]] to i64
+// CHECK-NEXT: [[SET_BITS:%.*]] = and i64 [[SRC_ADDR]], 63
+// CHECK-NEXT: [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
+// CHECK-NEXT: [[CONV:%.*]] = zext i1 [[IS_ALIGNED]] to i32
+// CHECK-NEXT: ret i32 [[CONV]]
+//
+int test_array(void) {
+ char buf[1024];
+ func(__builtin_align_down(&buf[44], 16));
+ func(__builtin_align_up(&buf[22], 32));
+ return __builtin_is_aligned(&buf[16], 64);
+}
+
+// CHECK-LABEL: define {{[^@]+}}@test_array_should_not_mask() #0
+// CHECK-NEXT: entry:
+// CHECK-NEXT: [[BUF:%.*]] = alloca [1024 x i8], align 32
+// CHECK-NEXT: [[ARRAYIDX:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 64
+// CHECK-NEXT: [[ALIGNED_RESULT:%.*]] = call i8* @llvm.ptrmask.p0i8.p0i8.i64(i8* [[ARRAYIDX]], i64 -16)
+// CHECK-NEXT: [[CALL:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT]])
+// CHECK-NEXT: [[ARRAYIDX1:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 32
+// CHECK-NEXT: [[OVER_BOUNDARY:%.*]] = getelementptr i8, i8* [[ARRAYIDX1]], i64 31
+// CHECK-NEXT: [[ALIGNED_RESULT2:%.*]] = call i8* @llvm.ptrmask.p0i8.p0i8.i64(i8* [[OVER_BOUNDARY]], i64 -32)
+// CHECK-NEXT: [[CALL3:%.*]] = call i32 @func(i8* [[ALIGNED_RESULT2]])
+// CHECK-NEXT: [[ARRAYIDX4:%.*]] = getelementptr inbounds [1024 x i8], [1024 x i8]* [[BUF]], i64 0, i64 64
+// CHECK-NEXT: [[SRC_ADDR:%.*]] = ptrtoint i8* [[ARRAYIDX4]] to i64
+// CHECK-NEXT: [[SET_BITS:%.*]] = and i64 [[SRC_ADDR]], 31
+// CHECK-NEXT: [[IS_ALIGNED:%.*]] = icmp eq i64 [[SET_BITS]], 0
+// CHECK-NEXT: [[CONV:%.*]] = zext i1 [[IS_ALIGNED]] to i32
+// CHECK-NEXT: ret i32 [[CONV]]
+//
+int test_array_should_not_mask(void) {
+ /// TODO: these alignment builtins could be omitted since we already know
+ /// that buf is aligned to 32 bytes.
+ _Alignas(32) char buf[1024];
+ func(__builtin_align_down(&buf[64], 16));
+ func(__builtin_align_up(&buf[32], 32));
+ return __builtin_is_aligned(&buf[64], 32);
+}
Index: clang/lib/Sema/SemaChecking.cpp
===================================================================
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -201,6 +201,73 @@
return false;
}
+static bool SemaBuiltinAlignment(Sema &S, CallExpr *TheCall, unsigned ID) {
+ if (checkArgCount(S, TheCall, 2))
+ return true;
+
+ clang::Expr *Source = TheCall->getArg(0);
+ clang::Expr *AlignOp = TheCall->getArg(1);
+ bool IsBooleanAlignBuiltin = ID == Builtin::BI__builtin_is_aligned;
+
+ auto IsValidIntegerType = [](QualType Ty) {
+ return Ty->isIntegerType() && !Ty->isEnumeralType() && !Ty->isBooleanType();
+ };
+ if (!IsValidIntegerType(AlignOp->getType())) {
+ S.Diag(AlignOp->getExprLoc(), diag::err_typecheck_expect_int)
+ << AlignOp->getType();
+ return true;
+ }
+
+ QualType SrcTy = Source->getType();
+ // Should also be able to use it with arrays (but not functions!)
+ bool IsArrayToPointerDecay =
+ SrcTy->canDecayToPointerType() && SrcTy->isArrayType();
+ if ((!SrcTy->isPointerType() && !IsArrayToPointerDecay &&
+ !IsValidIntegerType(SrcTy)) || SrcTy->isFunctionPointerType()) {
+ // XXX: this is not quite the right error message since we don't allow
+ // floating point types, or member pointers
+ S.Diag(AlignOp->getExprLoc(), diag::err_typecheck_expect_scalar_operand)
+ << SrcTy;
+ return true;
+ }
+ Expr::EvalResult AlignResult;
+ unsigned MaxAlignmentBits = S.Context.getIntWidth(SrcTy) - 1;
+ // Can't check validity of alignment if it is type dependent
+ if (!AlignOp->isInstantiationDependent() &&
+ AlignOp->EvaluateAsInt(AlignResult, S.Context,
+ Expr::SE_AllowSideEffects)) {
+ llvm::APSInt AlignValue = AlignResult.Val.getInt();
+ llvm::APSInt MaxValue(
+ llvm::APInt::getOneBitSet(MaxAlignmentBits + 1, MaxAlignmentBits));
+ if (AlignValue < 1) {
+ S.Diag(AlignOp->getExprLoc(), diag::err_alignment_too_small) << 1;
+ return true;
+ } else if (llvm::APSInt::compareValues(AlignValue, MaxValue) > 0) {
+ S.Diag(AlignOp->getExprLoc(), diag::err_alignment_too_big)
+ << MaxValue.toString(10);
+ return true;
+ } else if (AlignValue == 1) {
+ S.Diag(AlignOp->getExprLoc(), diag::warn_alignment_builtin_useless)
+ << IsBooleanAlignBuiltin;
+ } else if (!AlignValue.isPowerOf2()) {
+ S.Diag(AlignOp->getExprLoc(), diag::err_alignment_not_power_of_two);
+ return true;
+ }
+ }
+
+ if (IsBooleanAlignBuiltin) {
+ // __builtin_is_aligned() always returns bool.
+ TheCall->setType(S.Context.BoolTy);
+ } else {
+ // For align_up/align_down, the return type is the same as the argument type
+ // (including qualifiers). However, in the case of arrays, we must ensure
+ // that the return type is the decayed type and not an array type.
+ TheCall->setType(IsArrayToPointerDecay ? S.Context.getDecayedType(SrcTy)
+ : SrcTy);
+ }
+ return false;
+}
+
static bool SemaBuiltinOverflow(Sema &S, CallExpr *TheCall) {
if (checkArgCount(S, TheCall, 3))
return true;
@@ -1354,6 +1421,12 @@
if (SemaBuiltinAddressof(*this, TheCall))
return ExprError();
break;
+ case Builtin::BI__builtin_is_aligned:
+ case Builtin::BI__builtin_align_up:
+ case Builtin::BI__builtin_align_down:
+ if (SemaBuiltinAlignment(*this, TheCall, BuiltinID))
+ return ExprError();
+ break;
case Builtin::BI__builtin_add_overflow:
case Builtin::BI__builtin_sub_overflow:
case Builtin::BI__builtin_mul_overflow:
Index: clang/lib/CodeGen/CodeGenFunction.h
===================================================================
--- clang/lib/CodeGen/CodeGenFunction.h
+++ clang/lib/CodeGen/CodeGenFunction.h
@@ -3731,6 +3731,11 @@
/// Emit IR for __builtin_os_log_format.
RValue emitBuiltinOSLogFormat(const CallExpr &E);
+ /// Emit IR for __builtin_is_aligned
+ RValue EmitBuiltinIsAligned(const CallExpr *E);
+ /// Emit IR for __builtin_align_up/__builtin_align_down
+ RValue EmitBuiltinAlignTo(const CallExpr *E, bool AlignUp);
+
llvm::Function *generateBuiltinOSLogHelperFunction(
const analyze_os_log::OSLogBufferLayout &Layout,
CharUnits BufferAlignment);
Index: clang/lib/CodeGen/CGBuiltin.cpp
===================================================================
--- clang/lib/CodeGen/CGBuiltin.cpp
+++ clang/lib/CodeGen/CGBuiltin.cpp
@@ -3501,6 +3501,13 @@
return EmitBuiltinNewDeleteCall(
E->getCallee()->getType()->castAs<FunctionProtoType>(), E, true);
+ case Builtin::BI__builtin_is_aligned:
+ return EmitBuiltinIsAligned(E);
+ case Builtin::BI__builtin_align_up:
+ return EmitBuiltinAlignTo(E, true);
+ case Builtin::BI__builtin_align_down:
+ return EmitBuiltinAlignTo(E, false);
+
case Builtin::BI__noop:
// __noop always evaluates to an integer literal zero.
return RValue::get(ConstantInt::get(IntTy, 0));
@@ -14248,6 +14255,80 @@
}
}
+struct BuiltinAlignArgs {
+ llvm::Value *Src = nullptr;
+ llvm::Type *SrcType = nullptr;
+ llvm::Value *Alignment = nullptr;
+ llvm::Value *Mask = nullptr;
+ llvm::IntegerType *IntType = nullptr;
+
+ BuiltinAlignArgs(const CallExpr *E, CodeGenFunction &CGF) {
+ QualType AstType = E->getArg(0)->getType();
+ if (AstType->isArrayType()) {
+ Src = CGF.EmitArrayToPointerDecay(E->getArg(0)).getPointer();
+ } else {
+ Src = CGF.EmitScalarExpr(E->getArg(0));
+ }
+ SrcType = Src->getType();
+ if (SrcType->isPointerTy()) {
+ IntType = IntegerType::get(
+ CGF.getLLVMContext(),
+ CGF.CGM.getDataLayout().getIndexTypeSizeInBits(SrcType));
+ } else {
+ assert(SrcType->isIntegerTy());
+ IntType = cast<llvm::IntegerType>(SrcType);
+ }
+ Alignment = CGF.EmitScalarExpr(E->getArg(1));
+ Alignment = CGF.Builder.CreateZExtOrTrunc(Alignment, IntType, "alignment");
+ auto *One = llvm::ConstantInt::get(IntType, 1);
+ Mask = CGF.Builder.CreateSub(Alignment, One, "mask");
+ }
+};
+
+/// Generate (x & (y-1)) == 0
+RValue CodeGenFunction::EmitBuiltinIsAligned(const CallExpr *E) {
+ BuiltinAlignArgs Args(E, *this);
+ llvm::Value *SrcAddress = Args.Src;
+ if (Args.SrcType->isPointerTy())
+ SrcAddress =
+ Builder.CreateBitOrPointerCast(Args.Src, Args.IntType, "src_addr");
+ return RValue::get(Builder.CreateICmpEQ(
+ Builder.CreateAnd(SrcAddress, Args.Mask, "set_bits"),
+ llvm::Constant::getNullValue(Args.IntType), "is_aligned"));
+}
+
+/// Generate (x & ~(y-1)) to align down or ((x+(y-1)) & ~(y-1)) to align up.
+/// Note: For pointer types we avoid ptrtoint/inttoptr pairs by using the
+/// llvm.ptrmask instrinsic (with a GEP before in the align_up case).
+RValue CodeGenFunction::EmitBuiltinAlignTo(const CallExpr *E, bool AlignUp) {
+ BuiltinAlignArgs Args(E, *this);
+ llvm::Value *SrcForMask = Args.Src;
+ if (AlignUp) {
+ // When aligning up we have to first add the mask to ensure we go over the
+ // next alignment value and then align down to the next valid multiple
+ // By adding the mask, we ensure that align_up on an already aligned
+ // value will not change the value.
+ if (SrcForMask->getType()->isPointerTy()) {
+ SrcForMask = EmitCastToVoidPtr(SrcForMask);
+ SrcForMask = Builder.CreateGEP(SrcForMask, Args.Mask, "over_boundary");
+ } else {
+ SrcForMask = Builder.CreateAdd(SrcForMask, Args.Mask, "over_boundary");
+ }
+ }
+ // Negate the mask to only clear the lower bits
+ llvm::Value *NegatedMask = Builder.CreateNot(Args.Mask, "negated_mask");
+ llvm::Value *Result;
+ if (SrcForMask->getType()->isPointerTy()) {
+ Result = Builder.CreateIntrinsic(
+ Intrinsic::ptrmask, {Args.SrcType, SrcForMask->getType(), Args.IntType},
+ {SrcForMask, NegatedMask}, nullptr, "aligned_result");
+ } else {
+ Result = Builder.CreateAnd(SrcForMask, NegatedMask, "aligned_result");
+ }
+ assert(Result->getType() == Args.SrcType);
+ return RValue::get(Result);
+}
+
Value *CodeGenFunction::EmitWebAssemblyBuiltinExpr(unsigned BuiltinID,
const CallExpr *E) {
switch (BuiltinID) {
Index: clang/lib/AST/ExprConstant.cpp
===================================================================
--- clang/lib/AST/ExprConstant.cpp
+++ clang/lib/AST/ExprConstant.cpp
@@ -10539,6 +10539,32 @@
return ExprEvaluatorBaseTy::VisitCallExpr(E);
}
+static bool getBuiltinAlignArguments(const CallExpr *E, EvalInfo &Info,
+ APSInt &Val, APSInt &Alignment,
+ unsigned *ValWidth) {
+ Expr::EvalResult ExprResult;
+ if (!E->getArg(0)->EvaluateAsInt(ExprResult, Info.Ctx))
+ return false;
+ Val = ExprResult.Val.getInt();
+ if (!E->getArg(1)->EvaluateAsInt(ExprResult, Info.Ctx))
+ return false;
+ Alignment = ExprResult.Val.getInt();
+ if (Alignment < 0)
+ return false;
+ // XXX: can this ever happen? Will we end up here even if Sema gives an error?
+ // I guess this additional check doesn't do any harm.
+ if (!Alignment.isPowerOf2())
+ return false;
+ // Ensure both values have the same bit width so that we don't assert later.
+ *ValWidth = Val.getBitWidth();
+ if (Val.isUnsigned())
+ Val = Val.zextOrSelf(Alignment.getBitWidth());
+ else
+ Val = Val.sextOrSelf(Alignment.getBitWidth());
+ Alignment = Alignment.zextOrSelf(Val.getBitWidth());
+ return true;
+}
+
bool IntExprEvaluator::VisitBuiltinCallExpr(const CallExpr *E,
unsigned BuiltinOp) {
switch (unsigned BuiltinOp = E->getBuiltinCallee()) {
@@ -10581,6 +10607,34 @@
return Success(Layout.size().getQuantity(), E);
}
+ case Builtin::BI__builtin_is_aligned: {
+ APSInt Val;
+ APSInt Alignment;
+ unsigned ValWidth = -1;
+ if (!getBuiltinAlignArguments(E, Info, Val, Alignment, &ValWidth))
+ return false;
+ return Success((Val & (Alignment - 1)) == 0 ? 1 : 0, E);
+ }
+ case Builtin::BI__builtin_align_up: {
+ APSInt Val;
+ APSInt Alignment;
+ unsigned ValWidth = -1;
+ if (!getBuiltinAlignArguments(E, Info, Val, Alignment, &ValWidth))
+ return false;
+ APSInt Result =
+ APSInt((Val + (Alignment - 1)) & ~(Alignment - 1), Val.isUnsigned());
+ return Success(Result.extOrTrunc(ValWidth), E);
+ }
+ case Builtin::BI__builtin_align_down: {
+ APSInt Val;
+ APSInt Alignment;
+ unsigned ValWidth = -1;
+ if (!getBuiltinAlignArguments(E, Info, Val, Alignment, &ValWidth))
+ return false;
+ APSInt Result = APSInt(Val & ~(Alignment - 1), Val.isUnsigned());
+ return Success(Result.extOrTrunc(ValWidth), E);
+ }
+
case Builtin::BI__builtin_bswap16:
case Builtin::BI__builtin_bswap32:
case Builtin::BI__builtin_bswap64: {
Index: clang/include/clang/Basic/DiagnosticSemaKinds.td
===================================================================
--- clang/include/clang/Basic/DiagnosticSemaKinds.td
+++ clang/include/clang/Basic/DiagnosticSemaKinds.td
@@ -2909,6 +2909,9 @@
def err_alignment_dependent_typedef_name : Error<
"requested alignment is dependent but declaration is not dependent">;
+def warn_alignment_builtin_useless : Warning<
+ "%select{aligning a value|checking whether a value is aligned}0 to 1 byte is"
+ " %select{a no-op|always true}0">, InGroup<TautologicalCompare>;
def err_attribute_aligned_too_great : Error<
"requested alignment must be %0 bytes or smaller">;
def warn_assume_aligned_too_great
Index: clang/include/clang/Basic/Builtins.def
===================================================================
--- clang/include/clang/Basic/Builtins.def
+++ clang/include/clang/Basic/Builtins.def
@@ -1475,6 +1475,11 @@
BUILTIN(__builtin_dump_struct, "ivC*v*", "tn")
BUILTIN(__builtin_preserve_access_index, "v.", "t")
+// Alignment builtins (uses custom parsing to support pointers and integers)
+BUILTIN(__builtin_is_aligned, "bvC*z", "nct")
+BUILTIN(__builtin_align_up, "v*vC*z", "nct")
+BUILTIN(__builtin_align_down, "v*vC*z", "nct")
+
// Safestack builtins
BUILTIN(__builtin___get_unsafe_stack_start, "v*", "Fn")
BUILTIN(__builtin___get_unsafe_stack_bottom, "v*", "Fn")
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
