[PATCH] Fix PR8081, VLA returns of nested functions

[Richard Guenther]

This fixes the PR by making sure to use RSO for calls to nested
functions that return variable-sized types.  This is necessary
because GIMPLE cannot handle calls with an embedded assignment
of variable size (we can't put the WITH_SIZE_EXPR anywhere) and
thus we would ICE later when not using RSO.

Bootstrap and regtest pending on x86_64-unknown-linux-gnu.

Richard.

2012-01-13  Richard Guenther  <rguent...@suse.de>

        PR middle-end/8081
        * gimplify.c (gimplify_modify_expr_rhs): For calls with a
        variable-sized result always use RSO.

        * gcc.dg/torture/pr8081.c: New testcase.

Index: gcc/gimplify.c
===================================================================
*** gcc/gimplify.c      (revision 183121)
--- gcc/gimplify.c      (working copy)
*************** gimplify_modify_expr_rhs (tree *expr_p,
*** 4417,4422 ****
--- 4417,4427 ----
                /* It's OK to use the target directly if it's being
                   initialized. */
                use_target = true;
+             else if (variably_modified_type_p (TREE_TYPE (*to_p), NULL_TREE))
+               /* Always use the target and thus RSO for variable-sized types.
+                  GIMPLE cannot deal with a variable-sized assignment
+                  embedded in a call statement.  */
+               use_target = true;
              else if (TREE_CODE (*to_p) != SSA_NAME
                      && (!is_gimple_variable (*to_p)
                          || needs_to_live_in_memory (*to_p)))
Index: gcc/testsuite/gcc.dg/torture/pr8081.c
===================================================================
*** gcc/testsuite/gcc.dg/torture/pr8081.c       (revision 0)
--- gcc/testsuite/gcc.dg/torture/pr8081.c       (revision 0)
***************
*** 0 ****
--- 1,26 ----
+ /* { dg-do run } */
+ 
+ extern void abort (void);
+ int
+ main (int argc, char **argv)
+ {
+   int size = 10;
+   typedef struct
+     {
+       char val[size];
+     }
+   block;
+   block a, b;
+   block __attribute__((noinline))
+   retframe_block ()
+     {
+       return *(block *) &b;
+     }
+   b.val[0] = -1;
+   b.val[9] = -2;
+   a=retframe_block ();
+   if (a.val[0] != -1
+       || a.val[9] != -2)
+     abort ();
+   return 0;
+ }