[PATCH] D31101: [ThinLTO] Use clang's existing code gen handling for ThinLTO backends

2017-03-31 Thread Teresa Johnson via Phabricator via cfe-commits 
tejohnson abandoned this revision.
tejohnson added a comment.

Subsumed by https://reviews.llvm.org/D31534


https://reviews.llvm.org/D31101



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D31101: [ThinLTO] Use clang's existing code gen handling for ThinLTO backends

2017-03-30 Thread Teresa Johnson via Phabricator via cfe-commits 
tejohnson updated this revision to Diff 93535.
tejohnson added a comment.

Use LTO to emit LLVM IR


https://reviews.llvm.org/D31101

Files:
  lib/CodeGen/BackendUtil.cpp
  test/CodeGen/thinlto-emit-llvm.c


Index: test/CodeGen/thinlto-emit-llvm.c
===
--- /dev/null
+++ test/CodeGen/thinlto-emit-llvm.c
@@ -0,0 +1,10 @@
+// Test to ensure -emit-llvm and -emit-llvm-bc work when invoking the
+// ThinLTO backend path.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm -o 
- | FileCheck %s
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm-bc 
-o - | llvm-dis -o - | FileCheck %s
+
+// CHECK: define void @foo()
+void foo() {
+}
Index: lib/CodeGen/BackendUtil.cpp
===
--- lib/CodeGen/BackendUtil.cpp
+++ lib/CodeGen/BackendUtil.cpp
@@ -902,7 +902,7 @@
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
   std::unique_ptr OS,
-  std::string SampleProfile) {
+  std::string SampleProfile, BackendAction Action) 
{
   StringMap>
   ModuleToDefinedGVSummaries;
   
CombinedIndex->collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
@@ -954,6 +954,20 @@
   };
   lto::Config Conf;
   Conf.SampleProfile = std::move(SampleProfile);
+  switch (Action) {
+  case Backend_EmitNothing:
+Conf.IROutputType = lto::Config::EIRT_Nothing;
+break;
+  case Backend_EmitLL:
+Conf.IROutputType = lto::Config::EIRT_LL;
+break;
+  case Backend_EmitBC:
+Conf.IROutputType = lto::Config::EIRT_BC;
+break;
+  default:
+Conf.IROutputType = lto::Config::EIRT_CodeGen;
+break;
+  }
   if (Error E = thinBackend(
   Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
   ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
@@ -990,7 +1004,7 @@
 bool DoThinLTOBackend = CombinedIndex != nullptr;
 if (DoThinLTOBackend) {
   runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
-CGOpts.SampleProfileFile);
+CGOpts.SampleProfileFile, Action);
   return;
 }
   }


Index: test/CodeGen/thinlto-emit-llvm.c
===
--- /dev/null
+++ test/CodeGen/thinlto-emit-llvm.c
@@ -0,0 +1,10 @@
+// Test to ensure -emit-llvm and -emit-llvm-bc work when invoking the
+// ThinLTO backend path.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm-bc -o - | llvm-dis -o - | FileCheck %s
+
+// CHECK: define void @foo()
+void foo() {
+}
Index: lib/CodeGen/BackendUtil.cpp
===
--- lib/CodeGen/BackendUtil.cpp
+++ lib/CodeGen/BackendUtil.cpp
@@ -902,7 +902,7 @@
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
   std::unique_ptr OS,
-  std::string SampleProfile) {
+  std::string SampleProfile, BackendAction Action) {
   StringMap>
   ModuleToDefinedGVSummaries;
   CombinedIndex->collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
@@ -954,6 +954,20 @@
   };
   lto::Config Conf;
   Conf.SampleProfile = std::move(SampleProfile);
+  switch (Action) {
+  case Backend_EmitNothing:
+Conf.IROutputType = lto::Config::EIRT_Nothing;
+break;
+  case Backend_EmitLL:
+Conf.IROutputType = lto::Config::EIRT_LL;
+break;
+  case Backend_EmitBC:
+Conf.IROutputType = lto::Config::EIRT_BC;
+break;
+  default:
+Conf.IROutputType = lto::Config::EIRT_CodeGen;
+break;
+  }
   if (Error E = thinBackend(
   Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
   ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
@@ -990,7 +1004,7 @@
 bool DoThinLTOBackend = CombinedIndex != nullptr;
 if (DoThinLTOBackend) {
   runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
-CGOpts.SampleProfileFile);
+CGOpts.SampleProfileFile, Action);
   return;
 }
   }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D31101: [ThinLTO] Use clang's existing code gen handling for ThinLTO backends

2017-03-30 Thread Teresa Johnson via Phabricator via cfe-commits 
tejohnson added inline comments.



Comment at: lib/CodeGen/BackendUtil.cpp:1007
   else
 AsmHelper.EmitAssembly(Action, std::move(OS));
 

tejohnson wrote:
> I just noticed that EmitAssembly does a lot more than just emission - it is 
> also setting up an optimization pipeline in CreatePasses, which we don't want 
> to do in the ThinLTO backend case as we already do the opt in LTO. Which 
> makes me think that the simplest and most consistent solution, especially if 
> as in discussed in D31114 we'll be using the LTO API for emitting object and 
> assembly code, is to use LTO for all file emission in the ThinLTO backend 
> case. It would mean extending lto::Config to be able to flag when we want to 
> emit-llvm or emit-llvm-bc, and invoke the appropriate module writer/printer 
> instead of normal codegen. That way -emit-llvm* will always get output 
> corresponding to the native object/assembly in the ThinLTO backend case. WDYT?
I have a new patch set that will do just this that I am uploading momentarily


https://reviews.llvm.org/D31101



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D31101: [ThinLTO] Use clang's existing code gen handling for ThinLTO backends

2017-03-20 Thread Teresa Johnson via Phabricator via cfe-commits 
tejohnson added inline comments.



Comment at: lib/CodeGen/BackendUtil.cpp:1007
   else
 AsmHelper.EmitAssembly(Action, std::move(OS));
 

I just noticed that EmitAssembly does a lot more than just emission - it is 
also setting up an optimization pipeline in CreatePasses, which we don't want 
to do in the ThinLTO backend case as we already do the opt in LTO. Which makes 
me think that the simplest and most consistent solution, especially if as in 
discussed in D31114 we'll be using the LTO API for emitting object and assembly 
code, is to use LTO for all file emission in the ThinLTO backend case. It would 
mean extending lto::Config to be able to flag when we want to emit-llvm or 
emit-llvm-bc, and invoke the appropriate module writer/printer instead of 
normal codegen. That way -emit-llvm* will always get output corresponding to 
the native object/assembly in the ThinLTO backend case. WDYT?


https://reviews.llvm.org/D31101



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D31101: [ThinLTO] Use clang's existing code gen handling for ThinLTO backends

2017-03-20 Thread Teresa Johnson via Phabricator via cfe-commits 
tejohnson updated this revision to Diff 92324.
tejohnson added a comment.

As discussed in the review threads for https://reviews.llvm.org/D31114 and 
https://reviews.llvm.org/D31100, only fall
back to clang's output file emission for -emit-llvm and -emit-llvm-bc.


https://reviews.llvm.org/D31101

Files:
  lib/CodeGen/BackendUtil.cpp
  test/CodeGen/thinlto-emit-llvm.c


Index: test/CodeGen/thinlto-emit-llvm.c
===
--- /dev/null
+++ test/CodeGen/thinlto-emit-llvm.c
@@ -0,0 +1,10 @@
+// Test to ensure -emit-llvm and -emit-llvm-bc work when invoking the
+// ThinLTO backend path.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm -o 
- | FileCheck %s
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm-bc 
-o - | llvm-dis -o - | FileCheck %s
+
+// CHECK: define void @foo()
+void foo() {
+}
Index: lib/CodeGen/BackendUtil.cpp
===
--- lib/CodeGen/BackendUtil.cpp
+++ lib/CodeGen/BackendUtil.cpp
@@ -897,7 +897,8 @@
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
   std::unique_ptr OS,
-  std::string SampleProfile) {
+  std::string SampleProfile,
+  bool SkipCodeGen) {
   StringMap>
   ModuleToDefinedGVSummaries;
   
CombinedIndex->collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
@@ -949,6 +950,7 @@
   };
   lto::Config Conf;
   Conf.SampleProfile = SampleProfile;
+  Conf.SkipCodeGen = SkipCodeGen;
   if (Error E = thinBackend(
   Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
   ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
@@ -984,9 +986,16 @@
 // of an error).
 bool DoThinLTOBackend = CombinedIndex != nullptr;
 if (DoThinLTOBackend) {
-  runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
-CGOpts.SampleProfileFile);
-  return;
+  bool UsesCodeGen = (Action != Backend_EmitNothing &&
+  Action != Backend_EmitBC &&
+  Action != Backend_EmitLL);
+  if (UsesCodeGen) {
+runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
+  CGOpts.SampleProfileFile, /*SkipCodeGen*/false);
+return;
+  } else
+runThinLTOBackend(CombinedIndex.get(), M, nullptr,
+  CGOpts.SampleProfileFile, /*SkipCodeGen*/true);
 }
   }
 


Index: test/CodeGen/thinlto-emit-llvm.c
===
--- /dev/null
+++ test/CodeGen/thinlto-emit-llvm.c
@@ -0,0 +1,10 @@
+// Test to ensure -emit-llvm and -emit-llvm-bc work when invoking the
+// ThinLTO backend path.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm-bc -o - | llvm-dis -o - | FileCheck %s
+
+// CHECK: define void @foo()
+void foo() {
+}
Index: lib/CodeGen/BackendUtil.cpp
===
--- lib/CodeGen/BackendUtil.cpp
+++ lib/CodeGen/BackendUtil.cpp
@@ -897,7 +897,8 @@
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
   std::unique_ptr OS,
-  std::string SampleProfile) {
+  std::string SampleProfile,
+  bool SkipCodeGen) {
   StringMap>
   ModuleToDefinedGVSummaries;
   CombinedIndex->collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
@@ -949,6 +950,7 @@
   };
   lto::Config Conf;
   Conf.SampleProfile = SampleProfile;
+  Conf.SkipCodeGen = SkipCodeGen;
   if (Error E = thinBackend(
   Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
   ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
@@ -984,9 +986,16 @@
 // of an error).
 bool DoThinLTOBackend = CombinedIndex != nullptr;
 if (DoThinLTOBackend) {
-  runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
-CGOpts.SampleProfileFile);
-  return;
+  bool UsesCodeGen = (Action != Backend_EmitNothing &&
+  Action != Backend_EmitBC &&
+  Action != Backend_EmitLL);
+  if (UsesCodeGen) {
+runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
+  CGOpts.SampleProfileFile, /*SkipCodeGen*/false);
+return;
+  } else
+runThinLTOBackend(CombinedIndex.get(), M, nullptr,
+

[PATCH] D31101: [ThinLTO] Use clang's existing code gen handling for ThinLTO backends

2017-03-17 Thread Teresa Johnson via Phabricator via cfe-commits 
tejohnson created this revision.
Herald added subscribers: Prazek, mehdi_amini.

We noticed that when invoking the thinBackend via clang (for the
distributed build case) that flags like -ffunction-sections and
-emit-llvm were not having the intended effect. This could have been
fixed by setting up the TargetOptions and the CodeGenFileType in the LTO
Config, but since clang already has handling for all of this, it is
straightforward to just let it do the handling (just below the
early return after runThinLTOBackend being removed here).

Depends on https://reviews.llvm.org/D31100.


https://reviews.llvm.org/D31101

Files:
  lib/CodeGen/BackendUtil.cpp
  test/CodeGen/function-sections.c


Index: test/CodeGen/function-sections.c
===
--- test/CodeGen/function-sections.c
+++ test/CodeGen/function-sections.c
@@ -9,6 +9,12 @@
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu -S -fdata-sections -o - < %s | 
FileCheck %s --check-prefix=DATA_SECT
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu -S -fno-data-sections 
-fdata-sections -o - < %s | FileCheck %s --check-prefix=DATA_SECT
 
+// Try again through a clang invocation of the ThinLTO backend.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -S 
-ffunction-sections -o - | FileCheck %s --check-prefix=FUNC_SECT
+// RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -S -fdata-sections 
-o - | FileCheck %s --check-prefix=DATA_SECT
+
 const int hello = 123;
 void world() {}
 
@@ -22,7 +28,7 @@
 // FUNC_SECT: section .rodata,
 // FUNC_SECT: hello:
 
-// DATA_SECT-NOT: section
+// DATA_SECT-NOT: .section
 // DATA_SECT: world:
 // DATA_SECT: .section .rodata.hello,
 // DATA_SECT: hello:
Index: lib/CodeGen/BackendUtil.cpp
===
--- lib/CodeGen/BackendUtil.cpp
+++ lib/CodeGen/BackendUtil.cpp
@@ -896,7 +896,6 @@
 }
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
-  std::unique_ptr OS,
   std::string SampleProfile) {
   StringMap>
   ModuleToDefinedGVSummaries;
@@ -944,13 +943,11 @@
 
 OwnedImports.push_back(std::move(*MBOrErr));
   }
-  auto AddStream = [&](size_t Task) {
-return llvm::make_unique(std::move(OS));
-  };
   lto::Config Conf;
   Conf.SampleProfile = SampleProfile;
+  Conf.SkipCodeGen = true;
   if (Error E = thinBackend(
-  Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
+  Conf, 0, nullptr, *M, *CombinedIndex, ImportList,
   ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
 handleAllErrors(std::move(E), [&](ErrorInfoBase ) {
   errs() << "Error running ThinLTO backend: " << EIB.message() << '\n';
@@ -983,11 +980,8 @@
 // (LLVM will optionally ignore empty index files, returning null instead
 // of an error).
 bool DoThinLTOBackend = CombinedIndex != nullptr;
-if (DoThinLTOBackend) {
-  runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
-CGOpts.SampleProfileFile);
-  return;
-}
+if (DoThinLTOBackend)
+  runThinLTOBackend(CombinedIndex.get(), M, CGOpts.SampleProfileFile);
   }
 
   EmitAssemblyHelper AsmHelper(Diags, HeaderOpts, CGOpts, TOpts, LOpts, M);


Index: test/CodeGen/function-sections.c
===
--- test/CodeGen/function-sections.c
+++ test/CodeGen/function-sections.c
@@ -9,6 +9,12 @@
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu -S -fdata-sections -o - < %s | FileCheck %s --check-prefix=DATA_SECT
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu -S -fno-data-sections -fdata-sections -o - < %s | FileCheck %s --check-prefix=DATA_SECT
 
+// Try again through a clang invocation of the ThinLTO backend.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -S -ffunction-sections -o - | FileCheck %s --check-prefix=FUNC_SECT
+// RUN: %clang -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -S -fdata-sections -o - | FileCheck %s --check-prefix=DATA_SECT
+
 const int hello = 123;
 void world() {}
 
@@ -22,7 +28,7 @@
 // FUNC_SECT: section .rodata,
 // FUNC_SECT: hello:
 
-// DATA_SECT-NOT: section
+// DATA_SECT-NOT: .section
 // DATA_SECT: world:
 // DATA_SECT: .section .rodata.hello,
 // DATA_SECT: hello:
Index: lib/CodeGen/BackendUtil.cpp
===
--- lib/CodeGen/BackendUtil.cpp
+++ lib/CodeGen/BackendUtil.cpp
@@ -896,7 +896,6 @@
 }
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
-  std::unique_ptr OS,
   std::string SampleProfile) {
   StringMap