Hi! Comparisons of NULLPTR_TYPE operands cause all kinds of problems in the middle-end and in fold-const.c, various optimizations assume that if they see e.g. a non-equality comparison with one of the operands being INTEGER_CST and it is not INTEGRAL_TYPE_P (which has TYPE_{MIN,MAX}_VALUE), they can build_int_cst (type, 1) to find a successor.
The following patch fixes it by making sure they don't appear in the IL, optimize them away at cp_fold time as all can be folded. Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk? Though, I've just noticed that clang++ rejects the non-equality comparisons instead, foo () > 0 with invalid operands to binary expression ('decltype(nullptr)' (aka 'nullptr_t') and 'int') and foo () > nullptr with invalid operands to binary expression ('decltype(nullptr)' (aka 'nullptr_t') and 'nullptr_t') Shall we reject those too, in addition or instead of parts of this patch? If so, wouldn't this patch be still useful for backports, I bet we don't want to start reject it on the release branches when we used to accept it. 2021-07-15 Jakub Jelinek <ja...@redhat.com> PR c++/101443 * cp-gimplify.c (cp_fold): For comparisons with NULLPTR_TYPE operands, fold them right away to true or false. * g++.dg/cpp0x/nullptr46.C: New test. --- gcc/cp/cp-gimplify.c.jj 2021-06-25 10:36:22.141020337 +0200 +++ gcc/cp/cp-gimplify.c 2021-07-14 12:04:24.221860756 +0200 @@ -2424,6 +2424,32 @@ cp_fold (tree x) op0 = cp_fold_maybe_rvalue (TREE_OPERAND (x, 0), rval_ops); op1 = cp_fold_rvalue (TREE_OPERAND (x, 1)); + /* decltype(nullptr) has only one value, so optimize away all comparisons + with that type right away, keeping them in the IL causes troubles for + various optimizations. */ + if (COMPARISON_CLASS_P (org_x) + && TREE_CODE (TREE_TYPE (op0)) == NULLPTR_TYPE + && TREE_CODE (TREE_TYPE (op1)) == NULLPTR_TYPE) + { + switch (code) + { + case EQ_EXPR: + case LE_EXPR: + case GE_EXPR: + x = constant_boolean_node (true, TREE_TYPE (x)); + break; + case NE_EXPR: + case LT_EXPR: + case GT_EXPR: + x = constant_boolean_node (false, TREE_TYPE (x)); + break; + default: + gcc_unreachable (); + } + return omit_two_operands_loc (loc, TREE_TYPE (x), x, + op0, op1); + } + if (op0 != TREE_OPERAND (x, 0) || op1 != TREE_OPERAND (x, 1)) { if (op0 == error_mark_node || op1 == error_mark_node) --- gcc/testsuite/g++.dg/cpp0x/nullptr46.C.jj 2021-07-14 11:48:03.917122727 +0200 +++ gcc/testsuite/g++.dg/cpp0x/nullptr46.C 2021-07-14 11:46:52.261092097 +0200 @@ -0,0 +1,11 @@ +// PR c++/101443 +// { dg-do compile { target c++11 } } +// { dg-options "-O2" } + +decltype(nullptr) foo (); + +bool +bar () +{ + return foo () > nullptr || foo () < nullptr; +} Jakub