Re: LTO partitioning reorg 4/n

[Jan Hubicka]

  The patch also fixed undefined reference symbol seen with max partitioning 
  on a testcase.
  Here we fold through a reference to constant variable into a reference to 
  constant pool.
  Since initializer of the constant variable is not seen by the partitioner, 
  we do not
  duplicate the constant pool entry into the unit and we end up with 
  undefined reference.
  Fixed thus.  I will try to prepare fix for 4.7, too.  I think it is the 
  problem
  Andi is seeing with kernel build.
 
 I did quite some testing with this one with building kernels with
 different configurations (including over night randomconfig runs) 
 and it seems to fix the previous problems. I reverted the workarounds 
 (making random symbols visible) reverted.  So looks good.
 
 Getting it into 4.7 would be appreciated too, otherwise I would need to 
 keep the workarounds.

Good news, the 4.7 version of the patch will be bit different, I will try to do 
it
early next week.  Thanks for testing!

Honza
 
 -Andi

LTO partitioning reorg 4/n

[Jan Hubicka]

Hi,
this patch unifies the code as promised in the previous patch. It also cleans
up the APIs.  We now have enum symbol_class specifying how the symbol should be
handled and add_symbol_to_partition handling all the magic of what needs to be
dragged into partition and what not.

I also commonized some code that was duplicated across different partitioners
and added max partitioner that puts every symbol into new partition if
possible.  

The patch also fixed undefined reference symbol seen with max partitioning on a 
testcase.
Here we fold through a reference to constant variable into a reference to 
constant pool.
Since initializer of the constant variable is not seen by the partitioner, we 
do not
duplicate the constant pool entry into the unit and we end up with undefined 
reference.
Fixed thus.  I will try to prepare fix for 4.7, too.  I think it is the problem
Andi is seeing with kernel build.

Bootstrapped/regtested x86_64-linux. Will commit it after bit of further 
testing.

* common.opt (flto-partition): Add max.
* invoke.texi (flto-partition): Document max

* lto.c (do_whole_program_analysis): Care timevars, statistics and
AUX pointer cleaning. Add max partitioning.
* lto-partition.c (enum symbol_class): New.
(get_symbol_class): New function.
(symbol_partitioned_p): New function.
(add_references_to_partition): Remove.
(add_aliases_to_partition): Remove.
(add_cgraph_node_to_partition_1): Remove.
(add_cgraph_node_to_partition): Remove.
(add_symbol_to_partition): New function.
(add_symbol_to_partition_1): New function.
(contained_in_symbol): New function.
(partition_cgraph_node_p): Remove.
(partition_varpool_node_p): Remove.
(partition_symbol_p): Remove.
(lto_1_to_1_map): Cleanup.
(lto_max_map): New.
(lto_balanced_map): Update.
(lto_promote_cross_file_statics): Update.
* lto-partition.h (lto_max_map): Declare.
* timevar.def (TV_WHOPR_PARTITIONING): New timevar.
Index: common.opt
===
--- common.opt  (revision 191215)
+++ common.opt  (working copy)
@@ -1439,12 +1439,16 @@ Link-time optimization with number of pa
 
 flto-partition=1to1
 Common Var(flag_lto_partition_1to1)
-Partition functions and vars at linktime based on object files they originate 
from
+Partition symbols and vars at linktime based on object files they originate 
from
 
 flto-partition=balanced
 Common Var(flag_lto_partition_balanced)
 Partition functions and vars at linktime into approximately same sized buckets
 
+flto-partition=max
+Common Var(flag_lto_partition_max)
+Put every symbol into separate partition
+
 flto-partition=none
 Common Var(flag_lto_partition_none)
 Disable partioning and streaming
Index: lto/lto.c
===
--- lto/lto.c   (revision 191215)
+++ lto/lto.c   (working copy)
@@ -2604,11 +2604,28 @@ lto_wpa_write_files (void)
  
  fprintf (cgraph_dump_file, Writing partition %s to file %s, %i 
insns\n,
   part-name, temp_filename, part-insns);
+ fprintf (cgraph_dump_file,   Symbols in partition: );
  for (lsei = lsei_start_in_partition (part-encoder); !lsei_end_p 
(lsei);
   lsei_next_in_partition (lsei))
{
  symtab_node node = lsei_node (lsei);
- fprintf (cgraph_dump_file, %s , symtab_node_name (node));
+ fprintf (cgraph_dump_file, %s , symtab_node_asm_name (node));
+   }
+ fprintf (cgraph_dump_file, \n  Symbols in boundary: );
+ for (lsei = lsei_start (part-encoder); !lsei_end_p (lsei);
+  lsei_next (lsei))
+   {
+ symtab_node node = lsei_node (lsei);
+ if (!lto_symtab_encoder_in_partition_p (part-encoder, node))
+   {
+ fprintf (cgraph_dump_file, %s , symtab_node_asm_name 
(node));
+ if (symtab_function_p (node)
+  lto_symtab_encoder_encode_body_p (part-encoder, 
cgraph (node)))
+   fprintf (cgraph_dump_file, (body included));
+ else if (symtab_variable_p (node)
+   lto_symtab_encoder_encode_initializer_p 
(part-encoder, varpool (node)))
+   fprintf (cgraph_dump_file, (initializer included));
+   }
}
  fprintf (cgraph_dump_file, \n);
}
@@ -3093,6 +3107,8 @@ print_lto_report_1 (void)
 static void
 do_whole_program_analysis (void)
 {
+  symtab_node node;
+
   timevar_start (TV_PHASE_OPT_GEN);
 
   /* Note that since we are in WPA mode, materialize_cgraph will not
@@ -3127,17 +3143,31 @@ do_whole_program_analysis (void)
   dump_cgraph (cgraph_dump_file);
   dump_varpool (cgraph_dump_file);
 }
+#ifdef ENABLE_CHECKING
   verify_cgraph ();
+#endif