[Bug tree-optimization/89060] New: Improve tail call optimization

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89060

            Bug ID: 89060
           Summary: Improve tail call optimization
           Product: gcc
           Version: 9.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jakub at gcc dot gnu.org
  Target Milestone: ---

void qux (char *);
int baz (void);

int
foo (void)
{
  char buf[64];
  qux (buf);
  return baz ();
}

int
bar (void)
{
  {
    char buf[64];
    qux (buf);
  }
  return baz ();
}

While we must not tail call optimize the baz call in foo, because an address of
a local variable escapes and it could be live during the tail call, in bar we
don't tail call optimize it either, but we should.
The IL difference in *.tailc between foo and bar is:
   char buf[64];
   int _5;

   <bb 2> [local count: 1073741824]:
   qux (&buf);
+  buf ={v} {CLOBBER};
   _5 = baz ();
-  buf ={v} {CLOBBER};
   return _5;

Right now, we just do:
  /* Make sure the tail invocation of this function does not indirectly
     refer to local variables.  (Passing variables directly by value
     is OK.)  */
  FOR_EACH_LOCAL_DECL (cfun, idx, var)
    {
      if (TREE_CODE (var) != PARM_DECL
          && auto_var_in_fn_p (var, cfun->decl)
          && may_be_aliased (var)
          && (ref_maybe_used_by_stmt_p (call, var)
              || call_may_clobber_ref_p (call, var)))
        return;
    }

Perhaps we could record all these variables in a bitmap and if there are any,
perform a variable life analysis like cfgexpand.c does to determine variable
conflicts or what has been discussed for PR86214, and if we can prove that
those variables escaped, but are not live at the point of the tail call, we
could still tail call optimize this.