[clang] [compiler-rt] [llvm] [TypeProf][InstrFDO]Implement more efficient comparison sequence for indirect-call-promotion with vtable profiles. (PR #81442)

2024-05-28 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/81442
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO] [Draft] Import vtable definitions in ThinTLO and use more efficient vtable comparison sequence with cost-benefit analysis (PR #69141)

2024-05-24 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

Close this stale patch and won't merge conflicts in this one.

Working on https://github.com/llvm/llvm-project/pull/81442 now.

https://github.com/llvm/llvm-project/pull/69141
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO] [Draft] Import vtable definitions in ThinTLO and use more efficient vtable comparison sequence with cost-benefit analysis (PR #69141)

2024-05-24 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/69141
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-11 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> I think this might be breaking: FAIL: LLVM :: tools/gold/X86/thinlto.ll

fixed in 
https://github.com/llvm/llvm-project/commit/20ed5b1f45871612570d3bd447121ac43e083c6a

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-11 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> I think this might be breaking:
> FAIL: LLVM :: tools/gold/X86/thinlto.ll

Thanks for reporting and sorry for this. I'll send out a fix soon.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lld] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits


@@ -564,6 +581,12 @@ class GlobalValueSummary {
 
   bool canAutoHide() const { return Flags.CanAutoHide; }
 
+  bool shouldImportAsDec() const {

minglotus-6 wrote:

this makes sense, done.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits


@@ -16,13 +16,13 @@
 ^3 = gv: (guid: 2, summaries: (function: (module: ^1, flags: (linkage: 
external, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 0), 
insts: 10, calls: ((callee: ^15, relbf: 256, tail: 1)
 
 ; Summaries with different linkage types.
-^4 = gv: (guid: 3, summaries: (function: (module: ^0, flags: (linkage: 
internal, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1), 
insts: 1)))
+^4 = gv: (guid: 3, summaries: (function: (module: ^0, flags: (linkage: 
internal, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1, 
importType: definition), insts: 1)))
 ; Make this one an alias with a forward reference to aliasee.
-^5 = gv: (guid: 4, summaries: (alias: (module: ^0, flags: (linkage: private, 
visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1), aliasee: 
^14)))
+^5 = gv: (guid: 4, summaries: (alias: (module: ^0, flags: (linkage: private, 
visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1,  importType: 
definition), aliasee: ^14)))
 ^6 = gv: (guid: 5, summaries: (function: (module: ^0, flags: (linkage: 
available_externally, visibility: default, notEligibleToImport: 0, live: 0, 
dsoLocal: 0), insts: 1)))
 ^7 = gv: (guid: 6, summaries: (function: (module: ^0, flags: (linkage: 
linkonce, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 0), 
insts: 1)))
 ^8 = gv: (guid: 7, summaries: (function: (module: ^0, flags: (linkage: 
linkonce_odr, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 
0), insts: 1)))
-^9 = gv: (guid: 8, summaries: (function: (module: ^0, flags: (linkage: 
weak_odr, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 0, 
canAutoHide: 1), insts: 1)))
+^9 = gv: (guid: 8, summaries: (function: (module: ^0, flags: (linkage: 
weak_odr, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 0, 
canAutoHide: 0, importType: declaration), insts: 1)))

minglotus-6 wrote:

ah `canAutoHide:0` -> `canAutoHide:1` is not intentional. But I agree it's 
better to add a new record.

Done by taking `^24` and incrementing `^ID` for the rest.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits


@@ -2072,6 +2072,23 @@ void LLParser::parseOptionalVisibility(unsigned ) {
   Lex.Lex();
 }
 
+static GlobalValueSummary::ImportKind
+parseOptionalImportType(lltok::Kind Kind) {
+  GlobalValueSummary::ImportKind Res;
+  switch (Kind) {
+  default:
+Res = GlobalValueSummary::Definition;

minglotus-6 wrote:

done, by changing `parseOptionalImportType` to a member function of `LLParser` 
class (from a static free function) so it could return `tokError`.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits


@@ -432,6 +432,18 @@ class GlobalValueSummary {
   /// Sububclass discriminator (for dyn_cast<> et al.)
   enum SummaryKind : unsigned { AliasKind, FunctionKind, GlobalVarKind };
 
+  enum ImportKind : unsigned {
+// The global value definition corresponding to the summary should be
+// imported from source module
+Definition = 0,
+
+// When its definition doesn't exist in the destination module and not
+// imported (e.g., function is large to be inlined), the global value

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits


@@ -635,7 +635,8 @@ static void computeFunctionSummary(
   HasIndirBranchToBlockAddress || HasIFuncCall;
   GlobalValueSummary::GVFlags Flags(
   F.getLinkage(), F.getVisibility(), NotEligibleForImport,
-  /* Live = */ false, F.isDSOLocal(), F.canBeOmittedFromSymbolTable());
+  /* Live = */ false, F.isDSOLocal(), F.canBeOmittedFromSymbolTable(),
+  /*ImportType=*/GlobalValueSummary::ImportKind::Definition);

minglotus-6 wrote:

done. Fixed all callsites of `GVFlags` constructor.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [ThinLTO]Record import type in GlobalValueSummary::GVFlags (PR #87597)

2024-04-10 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

Resolve review feedback, and 'backfilled' three affected tests in 
`clang/test/CodeGen`. Now `ninja check-llvm check-clang check-compiler-rt` 
passed locally.

https://github.com/llvm/llvm-project/pull/87597
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [thinlto] Implement the import of vtable definitions. (PR #72551)

2024-04-01 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

Superseded by https://github.com/llvm/llvm-project/pull/79381

https://github.com/llvm/llvm-project/pull/72551
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [thinlto] Implement the import of vtable definitions. (PR #72551)

2024-04-01 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/72551
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-04-01 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -1362,8 +1372,8 @@ remapSamples(const sampleprof::FunctionSamples ,
   BodySample.second.getSamples());
 for (const auto  : BodySample.second.getCallTargets()) {
   Result.addCalledTargetSamples(BodySample.first.LineOffset,
-MaskedDiscriminator,
-Remapper(Target.first), Target.second);
+MaskedDiscriminator, 
Remapper(Target.first),

minglotus-6 wrote:

undo it.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,145 @@
+// REQUIRES: lld-available
+
+// RUN: rm -rf %t && split-file %s %t && cd %t

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,145 @@
+// REQUIRES: lld-available
+
+// RUN: rm -rf %t && split-file %s %t && cd %t
+//
+// RUN: %clangxx_pgogen -fuse-ld=lld -O2 -g -fprofile-generate=. -mllvm 
-enable-vtable-value-profiling test.cpp -o test
+// RUN: env LLVM_PROFILE_FILE=test.profraw ./test
+
+// Show vtable profiles from raw profile.
+// RUN: llvm-profdata show --function=main --ic-targets -show-vtables 
test.profraw | FileCheck %s --check-prefixes=COMMON,RAW
+
+// Generate indexed profile from raw profile and show the data.
+// RUN: llvm-profdata merge test.profraw -o test.profdata
+// RUN: llvm-profdata show --function=main --ic-targets --show-vtables 
test.profdata | FileCheck %s --check-prefixes=COMMON,INDEXED
+
+// Generate text profile from raw and indexed profiles respectively and show 
the data.
+// RUN: llvm-profdata merge --text test.profraw -o raw.proftext
+// RUN: llvm-profdata show --function=main --ic-targets --show-vtables --text 
raw.proftext | FileCheck %s --check-prefix=ICTEXT
+// RUN: llvm-profdata merge --text test.profdata -o indexed.proftext
+// RUN: llvm-profdata show --function=main --ic-targets --show-vtables --text 
indexed.proftext | FileCheck %s --check-prefix=ICTEXT
+
+// Generate indexed profile from text profiles and show the data
+// RUN: llvm-profdata merge --binary raw.proftext -o text.profraw
+// RUN: llvm-profdata show --function=main --ic-targets --show-vtables 
text.profraw | FileCheck %s --check-prefixes=COMMON,INDEXED
+// RUN: llvm-profdata merge --binary indexed.proftext -o text.profdata
+// RUN: llvm-profdata show --function=main --ic-targets --show-vtables 
text.profdata | FileCheck %s --check-prefixes=COMMON,INDEXED
+
+// COMMON: Counters:
+// COMMON-NEXT:  main:
+// COMMON-NEXT:  Hash: 0x0f9a16fe6d398548
+// COMMON-NEXT:  Counters: 2
+// COMMON-NEXT:  Indirect Call Site Count: 2
+// COMMON-NEXT:  Number of instrumented vtables: 2
+// RAW:  Indirect Target Results:
+// RAW-NEXT:   [  0, _ZN8Derived15func1Eii,250 ] (25.00%)
+// RAW-NEXT:   [  0, test.cpp;_ZN12_GLOBAL__N_18Derived25func1Eii,
750 ] (75.00%)
+// RAW-NEXT:   [  1, _ZN8Derived15func2Eii,250 ] (25.00%)
+// RAW-NEXT:   [  1, test.cpp;_ZN12_GLOBAL__N_18Derived25func2Eii,
750 ] (75.00%)
+// RAW-NEXT:  VTable Results:
+// RAW-NEXT:   [  0, _ZTV8Derived1,250 ] (25.00%)
+// RAW-NEXT:   [  0, test.cpp;_ZTVN12_GLOBAL__N_18Derived2E,750 ] 
(75.00%)
+// RAW-NEXT:   [  1, _ZTV8Derived1,250 ] (25.00%)
+// RAW-NEXT:   [  1, test.cpp;_ZTVN12_GLOBAL__N_18Derived2E,750 ] 
(75.00%)
+// INDEXED: Indirect Target Results:
+// INDEXED-NEXT: [  0, test.cpp;_ZN12_GLOBAL__N_18Derived25func1Eii,   
 750 ] (75.00%)
+// INDEXED-NEXT: [  0, _ZN8Derived15func1Eii,250 ] (25.00%)
+// INDEXED-NEXT: [  1, test.cpp;_ZN12_GLOBAL__N_18Derived25func2Eii,   
 750 ] (75.00%)
+// INDEXED-NEXT: [  1, _ZN8Derived15func2Eii,250 ] (25.00%)
+// INDEXED-NEXT: VTable Results:
+// INDEXED-NEXT: [  0, test.cpp;_ZTVN12_GLOBAL__N_18Derived2E,
750 ] (75.00%)
+// INDEXED-NEXT: [  0, _ZTV8Derived1,250 ] (25.00%)
+// INDEXED-NEXT: [  1, test.cpp;_ZTVN12_GLOBAL__N_18Derived2E,
750 ] (75.00%)
+// INDEXED-NEXT: [  1, _ZTV8Derived1,250 ] (25.00%)
+// COMMON: Instrumentation level: IR  entry_first = 0
+// COMMON-NEXT: Functions shown: 1
+// COMMON-NEXT: Total functions: 6
+// COMMON-NEXT: Maximum function count: 1000
+// COMMON-NEXT: Maximum internal block count: 250
+// COMMON-NEXT: Statistics for indirect call sites profile:
+// COMMON-NEXT:   Total number of sites: 2
+// COMMON-NEXT:   Total number of sites with values: 2
+// COMMON-NEXT:   Total number of profiled values: 4
+// COMMON-NEXT:   Value sites histogram:
+// COMMON-NEXT: NumTargets, SiteCount
+// COMMON-NEXT: 2, 2
+// COMMON-NEXT: Statistics for vtable profile:
+// COMMON-NEXT:   Total number of sites: 2
+// COMMON-NEXT:   Total number of sites with values: 2
+// COMMON-NEXT:   Total number of profiled values: 4
+// COMMON-NEXT:   Value sites histogram:
+// COMMON-NEXT: NumTargets, SiteCount
+// COMMON-NEXT: 2, 2
+
+// ICTEXT: :ir
+// ICTEXT: main
+// ICTEXT: # Func Hash:
+// ICTEXT: 1124236338992350536
+// ICTEXT: # Num Counters:
+// ICTEXT: 2
+// ICTEXT: # Counter Values:
+// ICTEXT: 1000
+// ICTEXT: 1
+// ICTEXT: # Num Value Kinds:
+// ICTEXT: 2
+// ICTEXT: # ValueKind = IPVK_IndirectCallTarget:
+// ICTEXT: 0
+// ICTEXT: # NumValueSites:
+// ICTEXT: 2
+// ICTEXT: 2
+// ICTEXT: test.cpp;_ZN12_GLOBAL__N_18Derived25func1Eii:750
+// ICTEXT: _ZN8Derived15func1Eii:250
+// ICTEXT: 2
+// ICTEXT: test.cpp;_ZN12_GLOBAL__N_18Derived25func2Eii:750
+// ICTEXT: _ZN8Derived15func2Eii:250
+// ICTEXT: # ValueKind = IPVK_VTableTarget:
+// ICTEXT: 2
+// ICTEXT: # NumValueSites:
+// ICTEXT: 2
+// ICTEXT: 2
+// ICTEXT: test.cpp;_ZTVN12_GLOBAL__N_18Derived2E:750

[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -676,24 +722,66 @@ TEST_P(InstrProfReaderWriterTest, icall_data_read_write) {
 
   Expected R = Reader->getInstrProfRecord("caller", 0x1234);
   ASSERT_THAT_ERROR(R.takeError(), Succeeded());
+
+  // Test the number of instrumented indirect call sites and the number of
+  // profiled values at each site.
   ASSERT_EQ(4U, R->getNumValueSites(IPVK_IndirectCallTarget));
   EXPECT_EQ(3U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 0));
   EXPECT_EQ(0U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 1));
   EXPECT_EQ(2U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 2));
-  EXPECT_EQ(1U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 3));
+  EXPECT_EQ(2U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 3));
+
+  // Test the number of instrumented vtable sites and the number of profiled
+  // values at each site.
+  ASSERT_EQ(2U, R->getNumValueSites(IPVK_VTableTarget));
+  EXPECT_EQ(3U, R->getNumValueDataForSite(IPVK_VTableTarget, 0));
+  EXPECT_EQ(2U, R->getNumValueDataForSite(IPVK_VTableTarget, 1));
+
+  // First indirect site.
+  {
+uint64_t TotalC;
+std::unique_ptr VD =
+R->getValueForSite(IPVK_IndirectCallTarget, 0, );
+
+EXPECT_EQ(3U * getProfWeight(), VD[0].Count);
+EXPECT_EQ(2U * getProfWeight(), VD[1].Count);
+EXPECT_EQ(1U * getProfWeight(), VD[2].Count);
+EXPECT_EQ(6U * getProfWeight(), TotalC);
+
+EXPECT_EQ(StringRef((const char *)VD[0].Value, 7), StringRef("callee3"));

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,145 @@
+// REQUIRES: lld-available
+
+// RUN: rm -rf %t && split-file %s %t && cd %t
+//
+// RUN: %clangxx_pgogen -fuse-ld=lld -O2 -g -fprofile-generate=. -mllvm 
-enable-vtable-value-profiling test.cpp -o test
+// RUN: env LLVM_PROFILE_FILE=test.profraw ./test
+
+// Show vtable profiles from raw profile.
+// RUN: llvm-profdata show --function=main --ic-targets -show-vtables 
test.profraw | FileCheck %s --check-prefixes=COMMON,RAW

minglotus-6 wrote:

fixed it.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -676,24 +722,66 @@ TEST_P(InstrProfReaderWriterTest, icall_data_read_write) {
 
   Expected R = Reader->getInstrProfRecord("caller", 0x1234);
   ASSERT_THAT_ERROR(R.takeError(), Succeeded());
+
+  // Test the number of instrumented indirect call sites and the number of
+  // profiled values at each site.
   ASSERT_EQ(4U, R->getNumValueSites(IPVK_IndirectCallTarget));
   EXPECT_EQ(3U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 0));
   EXPECT_EQ(0U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 1));
   EXPECT_EQ(2U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 2));
-  EXPECT_EQ(1U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 3));
+  EXPECT_EQ(2U, R->getNumValueDataForSite(IPVK_IndirectCallTarget, 3));
+
+  // Test the number of instrumented vtable sites and the number of profiled
+  // values at each site.
+  ASSERT_EQ(2U, R->getNumValueSites(IPVK_VTableTarget));
+  EXPECT_EQ(3U, R->getNumValueDataForSite(IPVK_VTableTarget, 0));
+  EXPECT_EQ(2U, R->getNumValueDataForSite(IPVK_VTableTarget, 1));
+
+  // First indirect site.
+  {
+uint64_t TotalC;
+std::unique_ptr VD =
+R->getValueForSite(IPVK_IndirectCallTarget, 0, );
+
+EXPECT_EQ(3U * getProfWeight(), VD[0].Count);

minglotus-6 wrote:

 > Gunit expectations should be written as EXPECT_EQ(actual, expected).

I see. Used `EXPECT_EQ(actual, expected)` in new lines, and updated some 
surrounding existing lines in this style in this PR. Will send out a patch 
later to update the rest of `EXPECT_EQ` to make the style consistent.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -431,23 +441,34 @@ class InstrProfSymtab {
   using AddrHashMap = std::vector>;
 
 private:
+  using AddrIntervalMap =
+  IntervalMap>;
   StringRef Data;
   uint64_t Address = 0;
-  // Unique name strings.
+  // Unique name strings. Used to ensure entries in MD5NameMap (a vector that's
+  // going to be sorted) has unique MD5 keys in the first place.
   StringSet<> NameTab;
+  // Records the unique virtual table names. This is used by InstrProfWriter to
+  // write out an on-disk chained hash table of virtual table names.
+  // InstrProfWriter stores per function profile data (keyed by function names)
+  // so it doesn't use a StringSet for function names.
+  StringSet<> VTableNames;
   // A map from MD5 keys to function name strings.
   std::vector> MD5NameMap;

minglotus-6 wrote:

Sure. i'll send out another patch later.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-29 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,145 @@
+// REQUIRES: lld-available
+
+// RUN: rm -rf %t && split-file %s %t && cd %t
+//
+// RUN: %clangxx_pgogen -fuse-ld=lld -O2 -g -fprofile-generate=. -mllvm 
-enable-vtable-value-profiling test.cpp -o test
+// RUN: env LLVM_PROFILE_FILE=test.profraw ./test

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-28 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> I think we should resort to scripts and profraw in LLVM if we don't have 
> support for textual format. This is the case for memprof for example. 

Got it. I updated the test cases. Now compiler-rt test provides raw-related 
test coverage, and IR test provides test coverage for the {show, merge} related 
with {indexed, text} formats.

> It seems cleanest if the allocator and the map are owned by the 
> InstrProfSymtab class.

Agree. With https://github.com/llvm/llvm-project/pull/86882, InstrProfSymtab 
could own both. Thanks for the discussion!
- Added a few 
[lines](https://github.com/llvm/llvm-project/pull/66825/commits/01585236e7c350830544482c9917320a75e9ac57#diff-f959bf0ff4ce781c36e6eddb19d46fadb78e0c79fa13f7df384bb72f38fcf405)
 in unit test to have some test coverage for the IntervalMap usage.


https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-26 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> My concern with this approach is that compiler-rt is treated as a different 
> project and updating the code within LLVM makes it easy to miss running the 
> test locally for the other project. I think such issues will be caught by the 
> buildbot but having it flagged earlier is better for the developer. What do 
> you think?

Yeah presubmit test coverage makes sense.

If it's desirable to get rid of 'update_vtable_value_prof_inputs.sh' and 
'vtable-value-prof-basic.profraw' in the repo, here is another way:
* For LLVM IR tests, store textual profiles in the repo, and run `llvm-profdata 
merge` to convert it to indexed profiles.
* To have test coverage on raw profiles (generate raw profiles or convert it 
into other formats), have a compiler-rt test.

I wonder if that is preferred over the current status (with script and 
`.profraw` in the repo).

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-26 Thread Mingming Liu via cfe-commits


@@ -560,6 +602,28 @@ Error InstrProfSymtab::addFuncWithName(Function , 
StringRef PGOFuncName) {
   return Error::success();
 }
 
+uint64_t InstrProfSymtab::getVTableHashFromAddress(uint64_t Address) {
+  finalizeSymtab();

minglotus-6 wrote:

Added a comment.

Thinking about it, there might be a way to call 'finalize' once to make the 
code simpler (illustrated below). If the overall idea looks good, I could work 
on a separate change to refactor existing code.

'Symtab' implementation is something like this

```
Status Symtab::addElement() {
   if (finalized)
 return error("symtab cannot have new element after being finalized");
  // add element implementation
}

void Symtab::finalize() {
  llvm::sort(vectors);
  finalized=true;
}
```

And user code is like

```
auto symtab = createSymtab();

for (auto element : elements) {
  symtab.addElement(element);
}

symtab.finalize();

auto result = symtab.getFuncHashFromAddress(addr);
...
```




https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-06 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-06 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-06 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -1237,6 +1259,101 @@ void InstrLowerer::maybeSetComdat(GlobalVariable *GV, 
Function *Fn,
 GV->setLinkage(GlobalValue::InternalLinkage);
 }
 
+static inline bool shouldRecordVTableAddr(GlobalVariable *GV) {
+  if (!profDataReferencedByCode(*GV->getParent()))
+return false;
+
+  if (!GV->hasLinkOnceLinkage() && !GV->hasLocalLinkage() &&
+  !GV->hasAvailableExternallyLinkage())
+return true;
+
+  // This avoids the profile data from referencing internal symbols in
+  // COMDAT.
+  if (GV->hasLocalLinkage() && GV->hasComdat())
+return false;
+
+  return true;
+}
+
+// FIXME: Introduce an internal alias like what's done for functions to reduce
+// the number of relocation entries.
+static inline Constant *getVTableAddrForProfData(GlobalVariable *GV) {
+  auto *Int8PtrTy = PointerType::getUnqual(GV->getContext());
+
+  // Store a nullptr in __profvt_ if a real address shouldn't be used.
+  if (!shouldRecordVTableAddr(GV))
+return ConstantPointerNull::get(Int8PtrTy);
+
+  return ConstantExpr::getBitCast(GV, Int8PtrTy);
+}
+
+void InstrLowerer::getOrCreateVTableProfData(GlobalVariable *GV) {
+  assert(!DebugInfoCorrelate &&
+ "Value profiling is not supported with lightweight instrumentation");
+  if (GV->isDeclaration() || GV->hasAvailableExternallyLinkage())
+return;
+
+  if (GV->getName().starts_with("llvm.") ||
+  GV->getName().starts_with("__llvm") ||
+  GV->getName().starts_with("__prof"))
+return;
+
+  // VTableProfData already created
+  auto It = VTableDataMap.find(GV);
+  if (It != VTableDataMap.end() && It->second)
+return;
+
+  GlobalValue::LinkageTypes Linkage = GV->getLinkage();
+  GlobalValue::VisibilityTypes Visibility = GV->getVisibility();
+
+  // This is to keep consistent with per-function profile data
+  // for correctness.
+  if (TT.isOSBinFormatXCOFF()) {
+Linkage = GlobalValue::InternalLinkage;
+Visibility = GlobalValue::DefaultVisibility;
+  }
+
+  LLVMContext  = M.getContext();
+  Type *DataTypes[] = {
+#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Init) LLVMType,
+#include "llvm/ProfileData/InstrProfData.inc"
+  };
+
+  auto *DataTy = StructType::get(Ctx, ArrayRef(DataTypes));
+
+  // Used by INSTR_PROF_VTABLE_DATA MACRO
+  Constant *VTableAddr = getVTableAddrForProfData(GV);
+  const std::string PGOVTableName = getPGOName(*GV);
+  // Record the length of the vtable. This is needed since vtable pointers
+  // loaded from C++ objects might be from the middle of a vtable definition.
+  uint32_t VTableSizeVal =
+  M.getDataLayout().getTypeAllocSize(GV->getValueType());
+
+  Constant *DataVals[] = {
+#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Init) Init,
+#include "llvm/ProfileData/InstrProfData.inc"
+  };
+
+  std::string VarName = getInstrProfVTableVarPrefix().str() + PGOVTableName;

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -605,6 +703,19 @@ Function* InstrProfSymtab::getFunction(uint64_t 
FuncMD5Hash) {
   return nullptr;
 }
 
+GlobalVariable *
+InstrProfSymtab::getGlobalVariable(uint64_t GlobalVariableMD5Hash) {
+  finalizeSymtab();

minglotus-6 wrote:

Indeed. With a DenseMap there is no need to call 'finalizeSymtab' (which sorts 
the vectors for look-up). Done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -538,14 +541,30 @@ Error RawInstrProfReader::readNextHeader(const 
char *CurrentPos) {
 
 template 
 Error RawInstrProfReader::createSymtab(InstrProfSymtab ) {
-  if (Error E = Symtab.create(StringRef(NamesStart, NamesEnd - NamesStart)))
+  if (Error E = Symtab.create(StringRef(NamesStart, NamesEnd - NamesStart),
+  StringRef(VNamesStart, VNamesEnd - VNamesStart)))
 return error(std::move(E));
   for (const RawInstrProf::ProfileData *I = Data; I != DataEnd; ++I) {
 const IntPtrT FPtr = swap(I->FunctionPointer);
 if (!FPtr)
   continue;
 Symtab.mapAddress(FPtr, swap(I->NameRef));
   }
+
+  if (VTableBegin != nullptr && VTableEnd != nullptr) {
+for (const RawInstrProf::VTableProfileData *I = VTableBegin;
+ I != VTableEnd; ++I) {
+  const IntPtrT VPtr = swap(I->VTablePointer);

minglotus-6 wrote:

`swap` handles byte order properly for fields in raw profiles. Interestingly I 
thought it was 'std::swap' until spotting the bug related with endianness 
(fixed by https://github.com/llvm/llvm-project/pull/76312)

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -1237,6 +1259,101 @@ void InstrLowerer::maybeSetComdat(GlobalVariable *GV, 
Function *Fn,
 GV->setLinkage(GlobalValue::InternalLinkage);
 }
 
+static inline bool shouldRecordVTableAddr(GlobalVariable *GV) {
+  if (!profDataReferencedByCode(*GV->getParent()))
+return false;
+
+  if (!GV->hasLinkOnceLinkage() && !GV->hasLocalLinkage() &&
+  !GV->hasAvailableExternallyLinkage())
+return true;
+
+  // This avoids the profile data from referencing internal symbols in
+  // COMDAT.
+  if (GV->hasLocalLinkage() && GV->hasComdat())
+return false;
+
+  return true;
+}
+
+// FIXME: Introduce an internal alias like what's done for functions to reduce
+// the number of relocation entries.
+static inline Constant *getVTableAddrForProfData(GlobalVariable *GV) {
+  auto *Int8PtrTy = PointerType::getUnqual(GV->getContext());
+
+  // Store a nullptr in __profvt_ if a real address shouldn't be used.
+  if (!shouldRecordVTableAddr(GV))
+return ConstantPointerNull::get(Int8PtrTy);
+
+  return ConstantExpr::getBitCast(GV, Int8PtrTy);
+}
+
+void InstrLowerer::getOrCreateVTableProfData(GlobalVariable *GV) {
+  assert(!DebugInfoCorrelate &&
+ "Value profiling is not supported with lightweight instrumentation");
+  if (GV->isDeclaration() || GV->hasAvailableExternallyLinkage())
+return;
+
+  if (GV->getName().starts_with("llvm.") ||

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -1237,6 +1259,101 @@ void InstrLowerer::maybeSetComdat(GlobalVariable *GV, 
Function *Fn,
 GV->setLinkage(GlobalValue::InternalLinkage);
 }
 
+static inline bool shouldRecordVTableAddr(GlobalVariable *GV) {
+  if (!profDataReferencedByCode(*GV->getParent()))
+return false;
+
+  if (!GV->hasLinkOnceLinkage() && !GV->hasLocalLinkage() &&
+  !GV->hasAvailableExternallyLinkage())
+return true;
+
+  // This avoids the profile data from referencing internal symbols in
+  // COMDAT.
+  if (GV->hasLocalLinkage() && GV->hasComdat())
+return false;
+
+  return true;
+}
+
+// FIXME: Introduce an internal alias like what's done for functions to reduce
+// the number of relocation entries.
+static inline Constant *getVTableAddrForProfData(GlobalVariable *GV) {
+  auto *Int8PtrTy = PointerType::getUnqual(GV->getContext());
+
+  // Store a nullptr in __profvt_ if a real address shouldn't be used.
+  if (!shouldRecordVTableAddr(GV))
+return ConstantPointerNull::get(Int8PtrTy);
+
+  return ConstantExpr::getBitCast(GV, Int8PtrTy);
+}
+
+void InstrLowerer::getOrCreateVTableProfData(GlobalVariable *GV) {
+  assert(!DebugInfoCorrelate &&
+ "Value profiling is not supported with lightweight instrumentation");
+  if (GV->isDeclaration() || GV->hasAvailableExternallyLinkage())
+return;
+
+  if (GV->getName().starts_with("llvm.") ||
+  GV->getName().starts_with("__llvm") ||
+  GV->getName().starts_with("__prof"))
+return;
+
+  // VTableProfData already created
+  auto It = VTableDataMap.find(GV);
+  if (It != VTableDataMap.end() && It->second)
+return;
+
+  GlobalValue::LinkageTypes Linkage = GV->getLinkage();
+  GlobalValue::VisibilityTypes Visibility = GV->getVisibility();
+
+  // This is to keep consistent with per-function profile data
+  // for correctness.
+  if (TT.isOSBinFormatXCOFF()) {
+Linkage = GlobalValue::InternalLinkage;
+Visibility = GlobalValue::DefaultVisibility;
+  }
+
+  LLVMContext  = M.getContext();
+  Type *DataTypes[] = {
+#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Init) LLVMType,
+#include "llvm/ProfileData/InstrProfData.inc"
+  };
+
+  auto *DataTy = StructType::get(Ctx, ArrayRef(DataTypes));
+
+  // Used by INSTR_PROF_VTABLE_DATA MACRO
+  Constant *VTableAddr = getVTableAddrForProfData(GV);
+  const std::string PGOVTableName = getPGOName(*GV);
+  // Record the length of the vtable. This is needed since vtable pointers
+  // loaded from C++ objects might be from the middle of a vtable definition.
+  uint32_t VTableSizeVal =
+  M.getDataLayout().getTypeAllocSize(GV->getValueType());
+
+  Constant *DataVals[] = {
+#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Init) Init,
+#include "llvm/ProfileData/InstrProfData.inc"
+  };
+
+  std::string VarName = getInstrProfVTableVarPrefix().str() + PGOVTableName;
+  auto *Data =
+  new GlobalVariable(M, DataTy, false /* constant */, Linkage,

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -567,6 +643,21 @@ Error InstrProfSymtab::create(const NameIterRange 
) {
   return Error::success();
 }
 
+template 
+Error InstrProfSymtab::create(const FuncNameIterRange ,
+  const VTableNameIterRange ) {
+  for (auto Name : FuncIterRange)

minglotus-6 wrote:

Spell out 'auto' by using 'StringRef'. `for (StringRef str : strs)` should be 
efficient whether `strs` is `std::vector` or 
`std::vector`.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -560,6 +602,28 @@ Error InstrProfSymtab::addFuncWithName(Function , 
StringRef PGOFuncName) {
   return Error::success();
 }
 
+uint64_t InstrProfSymtab::getVTableHashFromAddress(uint64_t Address) {
+  finalizeSymtab();

minglotus-6 wrote:

This is to make sure the vectors are sorted.
* `finalizeSymtab` runs `llvm::sort` the first time it's called and marks 
`Sorted` as true ; subsequent calls returns early if `Sorted` is true.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -459,6 +472,16 @@ Error InstrProfSymtab::create(Module , bool InLTO) {
 if (Error E = addFuncWithName(F, getPGOFuncName(F, InLTO)))
   return E;
   }
+
+  SmallVector Types;
+  for (GlobalVariable  : M.globals()) {
+if (!G.hasName())
+  continue;
+Types.clear();
+G.getMetadata(LLVMContext::MD_type, Types);
+if (!Types.empty())

minglotus-6 wrote:

yes, and this is more consistent with vtable detection in InstrProfiling.cpp. 
Done. 

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 commented:

> Here are some things I noticed, haven't looked at the tests yet.

@snehasish Thanks for the review! 

Talking about tests, I wonder if it looks better if I change the current LLVM 
IR test under `llvm/test/tools/llvm-profdata/` into a compiler-rt test under 
`compiler-rt/test/profile/Linux`, demonstrated in 
https://github.com/llvm/llvm-project/commit/167d5ce49e41ca098085cda315687ade194981b1

A compiler-rt test requires less files (e.g., no need to submit `.profraw` or 
its re-generation shell script in the llvm repo) and more self-contained, but 
it's put under `profile/Linux` so people using non-Linux environments won't run 
it (less coverage compared with LLVM IR test in this sense). However the 
support is limited to ELF. 
* The compiler-rt test should work for Darwins (apple). But since this test 
involves matching mangled names, the matchers could be exact (no regex 
matching) if test could assume it runs only on ELF.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -490,6 +591,23 @@ Error InstrProfSymtab::addFuncWithName(Function , 
StringRef PGOFuncName) {
   return Error::success();
 }
 
+uint64_t InstrProfSymtab::getVTableHashFromAddress(uint64_t Address) {
+  finalizeSymtab();
+  auto It = lower_bound(
+  VTableAddrRangeToMD5Map, Address,

minglotus-6 wrote:

IntervalMap could save the {sort, unique, lower_bound} over vectors. I tried to 
use it, but I thinkits implementation overhead doesn't make it an obvious win 
over STL usages 
1) IntervalMap 
[coalesces](https://github.com/llvm/llvm-project/blob/7bad74e66756ca2fd1fe4f5864e7123fb4553d78/llvm/include/llvm/ADT/IntervalMap.h#L127)
 adjacent ranges. `[a, b)` and `[c, d)` are considered adjacent `b + 1 == c` 
with the 
[default](https://github.com/llvm/llvm-project/blob/7bad74e66756ca2fd1fe4f5864e7123fb4553d78/llvm/include/llvm/ADT/IntervalMap.h#L155)
 trait. While vtables in `.data.rel.ro` may not be adjacent in real binaries,  
a custom trait which doesn't coalesce adjacent ranges is more foolproof and 
less likely to go wrong.
2) The code might look a little quirky w.r.t class method interfaces.
 a) This 
[line](https://github.com/llvm/llvm-project/blob/7bad74e66756ca2fd1fe4f5864e7123fb4553d78/llvm/lib/ProfileData/Coverage/CoverageMappingReader.cpp#L901)
 uses the implicit move assignment operator of `InstrProfSymtab`.
 b) `IntervalMap` constructor 
[requires](https://github.com/llvm/llvm-project/blob/7bad74e66756ca2fd1fe4f5864e7123fb4553d78/llvm/include/llvm/ADT/IntervalMap.h#L1041)
 an allocator, and the allocator is owned outside of interval map.
 c) Allocator doesn't have implicit move assignment operator because it 
[defines](https://github.com/llvm/llvm-project/blob/7bad74e66756ca2fd1fe4f5864e7123fb4553d78/llvm/include/llvm/Support/RecyclingAllocator.h#L37)
 destructor. 
   Considering a) and c), if 'InstrProfSymtab' has 'allocator' as a member, it 
needs to implement move assignment operator. Alternatively, since only raw 
profile reader sees profiled address and uses this map, raw profile reader 
could have an allocator member and pass it to `InstrSymtab::create` for 
interval map creation. But this way the code doesn't look very clean either.


https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -16,23 +16,72 @@
 #include 
 
 namespace llvm {
-// Visitor class that finds all indirect call.
+// Visitor class that finds indirect calls or instructions that gives vtable
+// value, depending on Type.
 struct PGOIndirectCallVisitor : public InstVisitor {
+  enum class InstructionType {
+kIndirectCall = 0,
+kVTableVal = 1,
+  };
   std::vector IndirectCalls;
-  PGOIndirectCallVisitor() = default;
+  std::vector ProfiledAddresses;
+  PGOIndirectCallVisitor(InstructionType Type) : Type(Type) {}
 
   void visitCallBase(CallBase ) {
-if (Call.isIndirectCall())
+if (Call.isIndirectCall()) {

minglotus-6 wrote:

done.

I also updated the code such that 'IndirectCalls' is maintained iff 'Type' is 
'kIndirectCall' and 'ProfiledAddresses' is maintained iff 'Type' is 
'kVTableVal'.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -378,6 +384,13 @@ std::string getPGOFuncName(const Function , bool InLTO, 
uint64_t Version) {
   return getPGOFuncName(F.getName(), GlobalValue::ExternalLinkage, "");
 }
 
+std::string getPGOName(const GlobalVariable , bool InLTO) {
+  // PGONameMetadata should be set by compiler at profile use time
+  // and read by symtab creation to look up symbols corresponding to
+  // a MD5 hash.
+  return getIRPGOObjectName(V, InLTO, nullptr /* PGONameMetadata */);

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -1237,6 +1259,101 @@ void InstrLowerer::maybeSetComdat(GlobalVariable *GV, 
Function *Fn,
 GV->setLinkage(GlobalValue::InternalLinkage);
 }
 
+static inline bool shouldRecordVTableAddr(GlobalVariable *GV) {
+  if (!profDataReferencedByCode(*GV->getParent()))
+return false;
+
+  if (!GV->hasLinkOnceLinkage() && !GV->hasLocalLinkage() &&
+  !GV->hasAvailableExternallyLinkage())
+return true;
+
+  // This avoids the profile data from referencing internal symbols in
+  // COMDAT.
+  if (GV->hasLocalLinkage() && GV->hasComdat())
+return false;
+
+  return true;
+}
+
+// FIXME: Introduce an internal alias like what's done for functions to reduce
+// the number of relocation entries.
+static inline Constant *getVTableAddrForProfData(GlobalVariable *GV) {
+  auto *Int8PtrTy = PointerType::getUnqual(GV->getContext());
+
+  // Store a nullptr in __profvt_ if a real address shouldn't be used.
+  if (!shouldRecordVTableAddr(GV))
+return ConstantPointerNull::get(Int8PtrTy);
+
+  return ConstantExpr::getBitCast(GV, Int8PtrTy);
+}
+
+void InstrLowerer::getOrCreateVTableProfData(GlobalVariable *GV) {
+  assert(!DebugInfoCorrelate &&
+ "Value profiling is not supported with lightweight instrumentation");
+  if (GV->isDeclaration() || GV->hasAvailableExternallyLinkage())
+return;
+
+  if (GV->getName().starts_with("llvm.") ||
+  GV->getName().starts_with("__llvm") ||
+  GV->getName().starts_with("__prof"))
+return;
+
+  // VTableProfData already created
+  auto It = VTableDataMap.find(GV);
+  if (It != VTableDataMap.end() && It->second)
+return;
+
+  GlobalValue::LinkageTypes Linkage = GV->getLinkage();
+  GlobalValue::VisibilityTypes Visibility = GV->getVisibility();
+
+  // This is to keep consistent with per-function profile data
+  // for correctness.
+  if (TT.isOSBinFormatXCOFF()) {
+Linkage = GlobalValue::InternalLinkage;
+Visibility = GlobalValue::DefaultVisibility;
+  }
+
+  LLVMContext  = M.getContext();
+  Type *DataTypes[] = {
+#define INSTR_PROF_VTABLE_DATA(Type, LLVMType, Name, Init) LLVMType,
+#include "llvm/ProfileData/InstrProfData.inc"
+  };

minglotus-6 wrote:

done.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-05 Thread Mingming Liu via cfe-commits


@@ -429,20 +439,36 @@ uint64_t ComputeHash(StringRef K);
 class InstrProfSymtab {
 public:
   using AddrHashMap = std::vector>;
+  using RangeHashMap =
+  std::vector, uint64_t>>;

minglotus-6 wrote:

done, and making the struct definition private to class 'InstrProfSymtab'.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-03-04 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

Besides the [non-elf 
warning](https://github.com/llvm/llvm-project/pull/66825#issuecomment-1968344800),
 I did some clean-ups (e.g., [delete braces around one-line 
if/loop](https://github.com/llvm/llvm-project/pull/66825#issuecomment-1968344800)
 and merges.

Planning to merge this one in by end of the week if no objections or concerns. 
It has been a long time, and thanks for the reviews on this one and the smaller 
nfc patches.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-02-27 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

New changes since last review besides clang-formats
1. Merged profile format change from main branch.
2. In instrumentation pass, emit unsupported warnings for non-ELF object file 
formats. 
- Updated vtable_profile.ll and add vtable_prof_unsupported.ll as test 
cases.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-02-27 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 ready_for_review 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [InstrFDO][TypeProf] Implement binary instrumentation and profile read/write (PR #66825)

2024-02-27 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table address comparision for indirect-call-promotion. (PR #66825)

2024-02-27 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 converted_to_draft 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [nfc]Generalize PGOFuncName helper methods for general global objects (PR #73570)

2024-02-23 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

This is superseded by https://github.com/llvm/llvm-project/pull/75954 

https://github.com/llvm/llvm-project/pull/73570
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [llvm] [nfc]Generalize PGOFuncName helper methods for general global objects (PR #73570)

2024-02-23 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/73570
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table address comparision for indirect-call-promotion. (PR #66825)

2024-02-14 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table address comparision for indirect-call-promotion. (PR #66825)

2024-02-14 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table address comparision for indirect-call-promotion. (PR #66825)

2024-02-14 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

I have been putting this on hold to coordinate with some tooling change (which 
likely become behind schedule).

At this point, both code reviews and testing the (pending) changes in would be 
much more time efficient if this initial change could get in sooner. 

Now with stacked reviews possible in llvm main branch, I'm planning to split 
the profile format change into https://github.com/llvm/llvm-project/pull/81691 
first to mitigate the risk of repeated roll backs that comes with a PR with 2k 
lines. After the profile format change is in llvm main branch, will run 'git 
merge main' to update this PR.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table address comparision for indirect-call-promotion. (PR #66825)

2024-02-14 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 ready_for_review 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table address comparision for indirect-call-promotion. (PR #66825)

2024-02-14 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 converted_to_draft 
https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [Driver] Allow -fbasic-block-sections for AArch64 ELF (PR #80916)

2024-02-07 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 approved this pull request.


https://github.com/llvm/llvm-project/pull/80916
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [clang] Add fbasic-block-sections support for AArch64 (PR #80916)

2024-02-06 Thread Mingming Liu via cfe-commits


@@ -5972,6 +5972,15 @@ void Clang::ConstructJob(Compilation , const JobAction 
,
 << A->getAsString(Args) << A->getValue();
   else
 A->render(Args, CmdArgs);
+} else if (Triple.isAArch64()) {
+  // "all" is not supported on AArch64 since branch relaxation creates new

minglotus-6 wrote:

I think `Triple.isOSBinFormatELF()` is needed (just like for x86).

https://github.com/llvm/llvm-project/pull/80916
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [clang] Add fbasic-block-sections support for AArch64 (PR #80916)

2024-02-06 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/80916
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [clang] Add fbasic-block-sections support for AArch64 (PR #80916)

2024-02-06 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 commented:

Optional suggestion to revise the title as something like 'Clang options for 
basic block sections on aarch64"

https://github.com/llvm/llvm-project/pull/80916
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [clang-tools-extra] [clang] [CGProfile] Use callee's PGO name when caller->callee is an indirect call. (PR #78610)

2024-01-22 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/78610
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [clang-tools-extra] [llvm] [CGProfile] Use callee's PGO name when caller->callee is an indirect call. (PR #78610)

2024-01-22 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

>  I'm missing the full context behind this patch, but code seems reasonable

Thanks for the review! This patch is supposed to be self-contained as a fix, 
although it's spotted when I worked on relevant larger efforts ([type 
profiling](https://discourse.llvm.org/t/rfc-dynamic-type-profiling-and-optimizations-in-llvm/74600)
 and [better call 
graph](https://discourse.llvm.org/t/rfc-for-better-call-graph-sort-build-a-more-complete-call-graph-by-adding-more-indirect-call-edges/74029))
 when trying to figure out how PGO names of functions work in the pipeline.

https://github.com/llvm/llvm-project/pull/78610
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [libunwind] [libcxx] [llvm] [libc] [mlir] [compiler-rt] [lldb] [clang] [flang] [lld] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table a

2024-01-21 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

I created a 
[branch](https://github.com/minglotus-6/llvm-project/tree/type_profile) and 
submitted the end-to-end implementation there. Besides (a [squashed 
version](https://github.com/llvm/llvm-project/commit/de71eacf04a236031be6f1b53168f8f566244297)
 of) this pull request, the rest of the branch contains [thinlto import of 
vtables](https://github.com/llvm/llvm-project/commit/f6c1fc894717c6e7c2b591868b20e85aaa5b7b6a)
 and actual icp transformations ([commit 
1](https://github.com/llvm/llvm-project/commit/094c8a9aa5b5f2bc30d3db5e1332f0a81a3f0411)
 and 
[2](https://github.com/llvm/llvm-project/commit/84582fef567b1839ca73ab14735b6cbdc8adc073))

> we could first teach profile reader to understand new sections in a forward 
> compatible way without bumping the version and let it soak everywhere (e.g, 
> for a few weeks), and increase the version number as a second change. I'm 
> planning to make some progress on this aspect and might also discuss f2f.

(Overdue public update) Turns out it's not feasible to pursue backward 
compatibility with raw profiles, or forward compatibility with indexed profiles 
without breaking them at least once. For instance, currently, the indexed 
profile reader relies on the C++ struct 
[definition](https://github.com/llvm/llvm-project/blob/a31a60074717fc40887cfe132b77eec93bedd307/llvm/include/llvm/ProfileData/InstrProf.h#L1040-L1063)
 of Header (e.g., the layout of fields) [to know the byte size of the profile 
header](https://github.com/llvm/llvm-project/blob/a31a60074717fc40887cfe132b77eec93bedd307/llvm/lib/ProfileData/InstrProf.cpp#L1549-L1570).
 As a result, older compilers won't be able to tell the byte size of header C++ 
struct if any new field is added in the header C++ struct (i.e., recording byte 
size in Header breaks forward compat once). So the code changes in the branch 
above doesn't make any effort to have additional version compat other than 
what's guaranteed currently (i.e. none for raw profiles, and backward compat 
for indexed profiles).

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [llvm] [clang] [CGProfile] Use callee's PGO name when caller->callee is an indirect call. (PR #78610)

2024-01-18 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/78610
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [clang] [clang-tools-extra] [CGProfile] Use callee's PGO name when caller->callee is an indirect call. (PR #78610)

2024-01-18 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 updated 
https://github.com/llvm/llvm-project/pull/78610

>From 90074a8722090a9144d987b10ee59ff37891c109 Mon Sep 17 00:00:00 2001
From: mingmingl 
Date: Thu, 11 Jan 2024 12:46:09 -0800
Subject: [PATCH] [CGProfile] Use callee's PGO name when caller->callee is an
 indirect call.

- With PGO, indirect call edges are constructed using value profiles,
  and the profile address is mapped to a function's PGO name.
- With ThinLTO, local functions could be promoted to have external
  linkage; and with full or thin LTO, global functions could be internalized.
  Edge construction should use a function's PGO name before its linkage
  is updated.
---
 .../llvm/Transforms/Instrumentation/CGProfile.h |  4 
 llvm/lib/Passes/PassBuilder.cpp |  4 
 llvm/lib/Passes/PassBuilderPipelines.cpp|  5 +++--
 llvm/lib/Passes/PassRegistry.def|  5 -
 .../Transforms/Instrumentation/CGProfile.cpp|  8 
 llvm/test/Instrumentation/cgprofile.ll  | 17 -
 6 files changed, 31 insertions(+), 12 deletions(-)

diff --git a/llvm/include/llvm/Transforms/Instrumentation/CGProfile.h 
b/llvm/include/llvm/Transforms/Instrumentation/CGProfile.h
index 9f9ce42277a0c12..0011b7b8b36c770 100644
--- a/llvm/include/llvm/Transforms/Instrumentation/CGProfile.h
+++ b/llvm/include/llvm/Transforms/Instrumentation/CGProfile.h
@@ -18,7 +18,11 @@ namespace llvm {
 class Module;
 class CGProfilePass : public PassInfoMixin {
 public:
+  CGProfilePass(bool InLTO) : InLTO(InLTO) {}
   PreservedAnalyses run(Module , ModuleAnalysisManager );
+
+private:
+  bool InLTO = false;
 };
 } // end namespace llvm
 
diff --git a/llvm/lib/Passes/PassBuilder.cpp b/llvm/lib/Passes/PassBuilder.cpp
index 439f749bda8bb78..f5cd6c507a3d4ea 100644
--- a/llvm/lib/Passes/PassBuilder.cpp
+++ b/llvm/lib/Passes/PassBuilder.cpp
@@ -732,6 +732,10 @@ Expected parseGlobalDCEPassOptions(StringRef Params) 
{
   return parseSinglePassOption(Params, "vfe-linkage-unit-visibility", 
"GlobalDCE");
 }
 
+Expected parseCGProfilePassOptions(StringRef Params) {
+  return parseSinglePassOption(Params, "in-lto-post-link", "CGProfile");
+}
+
 Expected parseInlinerPassOptions(StringRef Params) {
   return parseSinglePassOption(Params, "only-mandatory", "InlinerPass");
 }
diff --git a/llvm/lib/Passes/PassBuilderPipelines.cpp 
b/llvm/lib/Passes/PassBuilderPipelines.cpp
index 5c6c391049a7b25..525b83dee79b0dd 100644
--- a/llvm/lib/Passes/PassBuilderPipelines.cpp
+++ b/llvm/lib/Passes/PassBuilderPipelines.cpp
@@ -1477,7 +1477,8 @@ 
PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
   MPM.addPass(ConstantMergePass());
 
   if (PTO.CallGraphProfile && !LTOPreLink)
-MPM.addPass(CGProfilePass());
+MPM.addPass(CGProfilePass(LTOPhase == ThinOrFullLTOPhase::FullLTOPostLink 
||
+  LTOPhase == 
ThinOrFullLTOPhase::ThinLTOPostLink));
 
   // TODO: Relative look table converter pass caused an issue when full lto is
   // enabled. See https://reviews.llvm.org/D94355 for more details.
@@ -1972,7 +1973,7 @@ PassBuilder::buildLTODefaultPipeline(OptimizationLevel 
Level,
 MPM.addPass(MergeFunctionsPass());
 
   if (PTO.CallGraphProfile)
-MPM.addPass(CGProfilePass());
+MPM.addPass(CGProfilePass(/*InLTOPostLink=*/true));
 
   invokeFullLinkTimeOptimizationLastEPCallbacks(MPM, Level);
 
diff --git a/llvm/lib/Passes/PassRegistry.def b/llvm/lib/Passes/PassRegistry.def
index 82ce040c6496267..04e61a60addf796 100644
--- a/llvm/lib/Passes/PassRegistry.def
+++ b/llvm/lib/Passes/PassRegistry.def
@@ -47,7 +47,6 @@ MODULE_PASS("attributor", AttributorPass())
 MODULE_PASS("attributor-light", AttributorLightPass())
 MODULE_PASS("called-value-propagation", CalledValuePropagationPass())
 MODULE_PASS("canonicalize-aliases", CanonicalizeAliasesPass())
-MODULE_PASS("cg-profile", CGProfilePass())
 MODULE_PASS("check-debugify", NewPMCheckDebugifyPass())
 MODULE_PASS("constmerge", ConstantMergePass())
 MODULE_PASS("coro-cleanup", CoroCleanupPass())
@@ -150,6 +149,10 @@ MODULE_PASS_WITH_PARAMS(
 "asan", "AddressSanitizerPass",
 [](AddressSanitizerOptions Opts) { return AddressSanitizerPass(Opts); },
 parseASanPassOptions, "kernel")
+MODULE_PASS_WITH_PARAMS(
+"cg-profile", "CGProfilePass",
+[](bool InLTOPostLink) { return CGProfilePass(InLTOPostLink);},
+parseCGProfilePassOptions, "in-lto-post-link")
 MODULE_PASS_WITH_PARAMS(
 "globaldce", "GlobalDCEPass",
 [](bool InLTOPostLink) { return GlobalDCEPass(InLTOPostLink); },
diff --git a/llvm/lib/Transforms/Instrumentation/CGProfile.cpp 
b/llvm/lib/Transforms/Instrumentation/CGProfile.cpp
index e2e5f21b376b035..c322d0abd6bc1d1 100644
--- a/llvm/lib/Transforms/Instrumentation/CGProfile.cpp
+++ b/llvm/lib/Transforms/Instrumentation/CGProfile.cpp
@@ -44,8 +44,8 @@ addModuleFlags(Module ,
   return true;
 }
 
-static bool runCGProfilePass(
-Module , FunctionAnalysisManager ) {
+static bool 

[clang] [compiler-rt] [clang-tools-extra] [PGO] Reland PGO's Counter Reset and File Dumping APIs #76471 (PR #78285)

2024-01-16 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

thanks for the fix!

One test `Clang :: Preprocessor/init.c` failed 
(https://lab.llvm.org/buildbot/#/builders/74/builds/24843) and the error is due 
to mismatched macros (if I'm reading correctly wants `__LONG_LONG_MAX__` right 
after `__LLONG_WIDTH__` but sees `__LLVM_INSTR_PROFILE_GENERATE` in the 
middle). I'm wondering if the test (and other tests like 
`Preprocessor/init-aarch64.c`) needs update (or if they should be fixed by 
https://github.com/llvm/llvm-project/issues/77546#issuecomment-1887953628)

https://github.com/llvm/llvm-project/pull/78285
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-09 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> > Breaks https://lab.llvm.org/buildbot/#/builders/127/builds/60635 Please fix 
> > or revert.
> 
> Thanks for reporting the problem @vitalybuka !
> 
> I don't have a Windows machine to reproduce the issue. It is not clear to me 
> how the IR can contain names like 
> `"??_C@_0BA@MIKMMAII@rawprof?4profraw?$AA@"`. Do we have some special name 
> mangling treatment on Windows?
> 
> I will modify the test to check for a more general pattern, but I will not 
> know the result till the fix lands.

Sometimes godbolt output (like https://godbolt.org/z/MxbjcnanW) could be faster 
and more convenient (e.g., there is no need to get windows executable to for 
debugging, only LLVM IR is needed).

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [clang-tools-extra] [clang] [compiler-rt] [llvm-profdata] Use semicolon as the delimiter for supplementary profiles. (PR #75080)

2024-01-04 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/75080
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [clang-tools-extra] [clang] [compiler-rt] [llvm-profdata] Use semicolon as the delimiter for supplementary profiles. (PR #75080)

2024-01-04 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/75080
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [compiler-rt] [clang-tools-extra] [clang] [llvm-profdata] Use semicolon as the delimiter for supplementary profiles. (PR #75080)

2024-01-04 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/75080
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [clang] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [clang-tools-extra] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> > thanks! Mostly lg with a pending discussion on whether we want to have test 
> > coverage for `clang_pgogen` in `compiler-rt/test/profile/instrprof-api.c`, 
> > and the open-ended discussion about the observed build failure in another 
> > compiler-rt test.
> 
> Thanks for the feedback! The latest commit aims at resolving all comments but 
> the one related to the BE failure case. I will look into the BE related 
> failure next.

Actually most compiler-rt test doesn't encounter the build failure as the 
specific test (`instrprof-thinlto-indirect-call-promotion.cpp`) does. I now 
begin to wonder if it has something to do with the fact that that affected test 
is C++ (intentionally not pure C to have mangled names)..

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,16 @@
+// Test the linker feature that treats undefined weak symbols as null values.
+
+// RUN: %clang_pgogen -o %t %s

minglotus-6 wrote:

Please discard this comment. 

@snehasish pointed out build-bots doesn't really have coverage of AIX linker, 
and the executable returns 1 if weak symbol is not null.

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [clang-tools-extra] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,38 @@
+// RUN: %clang_profgen %s --target=ppc64le-unknown-linux-gnu -S \
+// RUN:-emit-llvm -o - | FileCheck %s --check-prefix=PROFGEN
+// RUN: %clang_profgen -o %t %s
+// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t

minglotus-6 wrote:

> I don't have access to a BE machine at the moment to reproduce but I will go 
> find one and take a look.

Or maybe just add `REQUIRES: host-byteorder-little-endian` if this test passes 
on little-endian systems. 

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [clang-tools-extra] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 commented:

thanks for making the changes!

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,16 @@
+// Test the linker feature that treats undefined weak symbols as null values.
+
+// RUN: %clang_pgogen -o %t %s

minglotus-6 wrote:

This just occurred to me, if the tested feature relies on linker 
implementation, does it need `REQUIRES: ` so that `not %t` passes? 

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [compiler-rt] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [compiler-rt] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [clang-tools-extra] [compiler-rt] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits




minglotus-6 wrote:

nit: maybe add a brief description about what this file tests.

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [clang] [compiler-rt] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,38 @@
+// RUN: %clang_profgen %s --target=ppc64le-unknown-linux-gnu -S \
+// RUN:-emit-llvm -o - | FileCheck %s --check-prefix=PROFGEN
+// RUN: %clang_profgen -o %t %s
+// RUN: env LLVM_PROFILE_FILE=%t.profraw %run %t

minglotus-6 wrote:

After reading your comment, I was about to say the binary is compiled with 
`ppc64le` and requires `host-byteorder-little-endian` but realized the 
`--target=ppc64le-unknown-linux-gnu` is only used with `-emit-llvm`, and the 
`RUN` lines that build executables doesn't specify a target.

Nevertheless, I wonder test coverage for `%clang_pgogen` should be added here 
as well.

Relatedly, I do get compilation errors on ppc big-endian systems for 
`clang_pgogen` in another compiler-rt test 
([commit](https://github.com/llvm/llvm-project/commit/5136c167a2573fbd05179849cb41b198c4862b12)
 and [buildbot link](https://lab.llvm.org/buildbot/#/builders/18/builds/13228)

>From the error message, I *think* lack of ABI implementation should manifest 
>whether it's `clang_pgogen` or `clang_profgen`.

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [compiler-rt] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits


@@ -0,0 +1,92 @@
+/*=== instr_prof_interface.h - Instrumentation PGO User Program API ===
+ *
+ * Part of the LLVM Project, under the Apache License v2.0 with LLVM 
Exceptions.
+ * See https://llvm.org/LICENSE.txt for license information.
+ * SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+ *
+ *===---===
+ *
+ * This header provides a public interface for fine-grained control of counter
+ * reset and profile dumping. These interface functions can be directly called
+ * in user programs.
+ *
+\*===-===*/
+
+#ifndef COMPILER_RT_INSTR_PROFILING
+#define COMPILER_RT_INSTR_PROFILING
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifdef __LLVM_INSTR_PROFILE_GENERATE
+// Profile file reset and dump interfaces.
+// When `-fprofile[-instr]-generate`/`-fcs-profile-generate` is in effect,
+// clang defines __LLVM_INSTR_PROFILE_GENERATE to pick up the API calls.
+
+/*!
+ * \brief Set the filename for writing instrumentation data.
+ *
+ * Sets the filename to be used for subsequent calls to
+ * \a __llvm_profile_write_file().
+ *
+ * \c Name is not copied, so it must remain valid.  Passing NULL resets the
+ * filename logic to the default behaviour.
+ *
+ * Note: There may be multiple copies of the profile runtime (one for each
+ * instrumented image/DSO). This API only modifies the filename within the
+ * copy of the runtime available to the calling image.
+ *
+ * Warning: This is a no-op if continuous mode (\ref
+ * __llvm_profile_is_continuous_mode_enabled) is on. The reason for this is
+ * that in continuous mode, profile counters are mmap()'d to the profile at
+ * program initialization time. Support for transferring the mmap'd profile
+ * counts to a new file has not been implemented.
+ */
+void __llvm_profile_set_filename(const char *Name);
+
+/*!
+ * \brief Interface to set all PGO counters to zero for the current process.
+ *
+ */
+void __llvm_profile_reset_counters(void);
+
+/*!
+ * \brief this is a wrapper interface to \c __llvm_profile_write_file.
+ * After this interface is invoked, an already dumped flag will be set
+ * so that profile won't be dumped again during program exit.
+ * Invocation of interface __llvm_profile_reset_counters will clear
+ * the flag. This interface is designed to be used to collect profile
+ * data from user selected hot regions. The use model is
+ *  __llvm_profile_reset_counters();
+ *  ... hot region 1
+ *  __llvm_profile_dump();
+ *  .. some other code
+ *  __llvm_profile_reset_counters();
+ *  ... hot region 2
+ *  __llvm_profile_dump();
+ *
+ *  It is expected that on-line profile merging is on with \c %m specifier
+ *  used in profile filename . If merging is  not turned on, user is expected
+ *  to invoke __llvm_profile_set_filename  to specify different profile names

minglotus-6 wrote:

nittest nit: remove the additional whitespace

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [compiler-rt] [clang] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [clang-tools-extra] [PGO] Exposing PGO's Counter Reset and File Dumping APIs (PR #76471)

2024-01-02 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 approved this pull request.

thanks! Mostly lg with a pending discussion on whether we want to have test 
coverage for `clang_pgogen` in `compiler-rt/test/profile/instrprof-api.c`, and 
the open-ended discussion about the observed build failure in another 
compiler-rt test.

https://github.com/llvm/llvm-project/pull/76471
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[libunwind] [lldb] [llvm] [libc] [clang-tools-extra] [compiler-rt] [flang] [clang] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

And pushed https://github.com/llvm/llvm-project/pull/76005 to require 64-bit 
systems for IR test. Raw profile reader swaps byte orders properly so 
`REQUIRES: host-byteorder-little-endian` looks actually unnecessary but will 
keep it fwiw for now.

I feel sorry for dance around but figured fix forwards of test requirements is 
much faster than getting various machines and setting up environments there.. 
Will monitor the build-bots .

https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [lldb] [libunwind] [libc] [clang-tools-extra] [compiler-rt] [llvm] [flang] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> I'm seeing spurious failures from big-endian systems. I'll revert and 
> investigate a fix.

I ended up sending a fix forward in 
https://github.com/llvm/llvm-project/pull/76001 . Will monitor the build-bots.

https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [libc] [flang] [lldb] [clang-tools-extra] [libunwind] [llvm] [compiler-rt] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

I'm seeing spurious failures from big-endian systems. I'll revert and 
investigate a fix.

https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [lldb] [llvm] [libunwind] [clang] [flang] [clang-tools-extra] [libc] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> It fails here https://lab.llvm.org/buildbot/#/builders/18/builds/13216

thanks for the report. I marked the two relevant tests as `UNSUPPORTED` in 
[commit 
85525f8](https://github.com/llvm/llvm-project/commit/85525f8fb6740a4cc117b00a238c85bda4ea01bf).

Will need to investigate and re-enable them after getting access to ppc machine 
from [gcc farm](https://gcc.gnu.org/wiki/CompileFarm)


https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang-tools-extra] [libunwind] [lldb] [libc] [llvm] [clang] [compiler-rt] [flang] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[libc] [lldb] [clang-tools-extra] [compiler-rt] [flang] [llvm] [clang] [libunwind] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

I'll go ahead and push it since the diff is test only.

https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[flang] [libc] [lldb] [libunwind] [llvm] [clang-tools-extra] [compiler-rt] [clang] [mlir] [lld] [libcxx] [IRPGO][ValueProfile] Instrument virtual table address that could be used to do virtual table a

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

> After this patch, follow up with a patch documenting raw and index format. 
> This has long being requested by many in the community.

Sure, I'm working on an `rst` doc and would like to focus on the PGO profile 
format (while compiler-rt infrastructure supports other use cases, like 
coverage instrumentation, etc), for example, how profile-data control structure 
references counters and value profiles.

I just updated this pull request (fix clang-format after using the same 
clang-format as bot, etc). @amharc and I discussed how to work around possible 
profile version skews without waiting for version compatibility full-fledged 
solution, and he suggested we could first teach profile reader to understand 
new sections without bumping the version and let it soak everywhere (e.g, for a 
few weeks), and increase the version number as a second change. I'm planning to 
make some progress on this aspect.

https://github.com/llvm/llvm-project/pull/66825
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [lldb] [flang] [clang-tools-extra] [compiler-rt] [libunwind] [libc] [clang] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

The diff of this reland compared with the 1st reland is clearer in this commit 
(https://github.com/llvm/llvm-project/pull/75954/commits/cca510e569e9325ec089974e006eb2a8e2d0a2a7)

https://github.com/llvm/llvm-project/pull/75954
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[llvm] [lldb] [flang] [clang-tools-extra] [compiler-rt] [libunwind] [libc] [clang] Reland the reland "[PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-

2023-12-19 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 updated 
https://github.com/llvm/llvm-project/pull/75954

>From 4936920fbbe5e70a47be35b057200de3b07a087f Mon Sep 17 00:00:00 2001
From: mingmingl 
Date: Mon, 18 Dec 2023 20:21:40 -0800
Subject: [PATCH 1/2] Reapply "Reland "[PGO][GlobalValue][LTO]In
 GlobalValues::getGlobalIdentifier, use semicolon as delimiter for
 local-linkage varibles. "" (#75888)

This reverts commit 6ce23ea0ab6370c944f5e426a20217f93f41aa15.
---
 compiler-rt/test/profile/CMakeLists.txt   |   2 +-
 ...trprof-thinlto-indirect-call-promotion.cpp | 115 ++
 llvm/include/llvm/IR/GlobalValue.h|   4 +
 llvm/include/llvm/ProfileData/InstrProf.h |  26 ++--
 llvm/lib/IR/Globals.cpp   |  12 +-
 llvm/lib/ProfileData/InstrProf.cpp|  36 --
 llvm/lib/ProfileData/InstrProfReader.cpp  |   9 +-
 .../thinlto-function-summary-originalnames.ll |  10 +-
 llvm/test/ThinLTO/X86/memprof-basic.ll|  26 ++--
 .../X86/memprof-duplicate-context-ids.ll  |  10 +-
 .../ThinLTO/X86/memprof-funcassigncloning.ll  |   6 +-
 llvm/test/ThinLTO/X86/memprof-indirectcall.ll |  32 ++---
 llvm/test/ThinLTO/X86/memprof-inlined.ll  |  14 +--
 .../Inputs/thinlto_indirect_call_promotion.ll |  16 ---
 .../thinlto_indirect_call_promotion.profraw   | Bin 0 -> 528 bytes
 ..._thinlto_indirect_call_promotion_inputs.sh |  62 ++
 .../thinlto_indirect_call_promotion.ll| 105 +++-
 llvm/unittests/ProfileData/InstrProfTest.cpp  |   4 +-
 18 files changed, 362 insertions(+), 127 deletions(-)
 create mode 100644 
compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
 delete mode 100644 
llvm/test/Transforms/PGOProfile/Inputs/thinlto_indirect_call_promotion.ll
 create mode 100644 
llvm/test/Transforms/PGOProfile/Inputs/thinlto_indirect_call_promotion.profraw
 create mode 100755 
llvm/test/Transforms/PGOProfile/Inputs/update_thinlto_indirect_call_promotion_inputs.sh

diff --git a/compiler-rt/test/profile/CMakeLists.txt 
b/compiler-rt/test/profile/CMakeLists.txt
index 975e4c42f4b640..eebe0469efebe0 100644
--- a/compiler-rt/test/profile/CMakeLists.txt
+++ b/compiler-rt/test/profile/CMakeLists.txt
@@ -6,7 +6,7 @@ set(PROFILE_TESTSUITES)
 set(PROFILE_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS} compiler-rt-headers)
 list(APPEND PROFILE_TEST_DEPS profile)
 if(NOT COMPILER_RT_STANDALONE_BUILD)
-  list(APPEND PROFILE_TEST_DEPS llvm-profdata llvm-cov)
+  list(APPEND PROFILE_TEST_DEPS llvm-cov llvm-dis llvm-lto llvm-profdata opt)
   if(NOT APPLE AND COMPILER_RT_HAS_LLD AND "lld" IN_LIST LLVM_ENABLE_PROJECTS)
 list(APPEND PROFILE_TEST_DEPS lld)
   endif()
diff --git 
a/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp 
b/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
new file mode 100644
index 00..82ca1cd7d0a564
--- /dev/null
+++ b/compiler-rt/test/profile/instrprof-thinlto-indirect-call-promotion.cpp
@@ -0,0 +1,115 @@
+// This is a regression test for ThinLTO indirect-call-promotion when candidate
+// callees need to be imported from another IR module.  In the C++ test case,
+// `main` calls `global_func` which is defined in another module. `global_func`
+// has two indirect callees, one has external linkage and one has local 
linkage.
+// All three functions should be imported into the IR module of main.
+
+// What the test does:
+// - Generate raw profiles from executables and convert it to indexed profiles.
+//   During the conversion, a profiled callee address in raw profiles will be
+//   converted to function hash in indexed profiles.
+// - Run IRPGO profile use and ThinTLO prelink pipeline and get LLVM bitcodes
+//   for both cpp files in the C++ test case.
+// - Generate ThinLTO summary file with LLVM bitcodes, and run 
`function-import` pass.
+// - Run `pgo-icall-prom` pass for the IR module which needs to import callees.
+
+// Use lld as linker for more robust test. We need to REQUIRE LLVMgold.so for
+// LTO if default linker is GNU ld or gold anyway.
+// REQUIRES: lld-available
+
+// Test should fail where linkage-name and mangled-name diverges, see issue 
https://github.com/llvm/llvm-project/issues/74565).
+// Currently, this name divergence happens on Mach-O object file format, or on
+// many (but not all) 32-bit Windows systems.
+//
+// XFAIL: system-darwin
+//
+// Mark 32-bit Windows as UNSUPPORTED for now as opposed to XFAIL. This test
+// should fail on many (but not all) 32-bit Windows systems and succeed on the
+// rest. The flexibility in triple string parsing makes it tricky to capture
+// both sets accurately. i[3-9]86 specifies arch as Triple::ArchType::x86, 
(win32|windows)
+// specifies OS as Triple::OS::Win32
+//
+// UNSUPPORTED: target={{i.86.*windows.*}}
+
+// RUN: rm -rf %t && split-file %s %t && cd %t
+
+// Do setup work for all below tests.
+// Generate raw profiles from real programs and convert it into indexed 
profiles.
+// Use clangxx_pgogen for IR 

[llvm] [libcxx] [libc] [compiler-rt] [clang] [compiler-rt]Add lld into dependency for apple builds now that lld Mach-O backend is available (PR #75884)

2023-12-19 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/75884
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[libc] [libcxx] [compiler-rt] [llvm] [clang] [compiler-rt]Add lld into dependency for apple builds now that lld Mach-O backend is available (PR #75884)

2023-12-18 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 updated 
https://github.com/llvm/llvm-project/pull/75884

>From 2af4eed3a742173b8f385a5b8222e74d1c6e71ed Mon Sep 17 00:00:00 2001
From: Mingming Liu 
Date: Mon, 18 Dec 2023 17:22:10 -0800
Subject: [PATCH] [compiler-rt]Add lld into dependency for apple builds now
 that lld Mach-O backend is available

---
 compiler-rt/test/profile/CMakeLists.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/compiler-rt/test/profile/CMakeLists.txt 
b/compiler-rt/test/profile/CMakeLists.txt
index 975e4c42f4b640..3057abebbe52cf 100644
--- a/compiler-rt/test/profile/CMakeLists.txt
+++ b/compiler-rt/test/profile/CMakeLists.txt
@@ -7,7 +7,7 @@ set(PROFILE_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS} 
compiler-rt-headers)
 list(APPEND PROFILE_TEST_DEPS profile)
 if(NOT COMPILER_RT_STANDALONE_BUILD)
   list(APPEND PROFILE_TEST_DEPS llvm-profdata llvm-cov)
-  if(NOT APPLE AND COMPILER_RT_HAS_LLD AND "lld" IN_LIST LLVM_ENABLE_PROJECTS)
+  if(COMPILER_RT_HAS_LLD AND "lld" IN_LIST LLVM_ENABLE_PROJECTS)
 list(APPEND PROFILE_TEST_DEPS lld)
   endif()
 endif()

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[libcxx] [clang] [llvm] [compiler-rt] [libc] [compiler-rt]Add lld into dependency for apple builds now that lld Mach-O backend is available (PR #75884)

2023-12-18 Thread Mingming Liu via cfe-commits

minglotus-6 wrote:

thanks for the reviews!

https://github.com/llvm/llvm-project/pull/75884
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [llvm] [libc] [libcxx] [clang] [compiler-rt]Add lld into dependency for apple builds now that lld Mach-O backend is available (PR #75884)

2023-12-18 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 updated 
https://github.com/llvm/llvm-project/pull/75884

>From 2af4eed3a742173b8f385a5b8222e74d1c6e71ed Mon Sep 17 00:00:00 2001
From: Mingming Liu 
Date: Mon, 18 Dec 2023 17:22:10 -0800
Subject: [PATCH] [compiler-rt]Add lld into dependency for apple builds now
 that lld Mach-O backend is available

---
 compiler-rt/test/profile/CMakeLists.txt | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/compiler-rt/test/profile/CMakeLists.txt 
b/compiler-rt/test/profile/CMakeLists.txt
index 975e4c42f4b640..3057abebbe52cf 100644
--- a/compiler-rt/test/profile/CMakeLists.txt
+++ b/compiler-rt/test/profile/CMakeLists.txt
@@ -7,7 +7,7 @@ set(PROFILE_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS} 
compiler-rt-headers)
 list(APPEND PROFILE_TEST_DEPS profile)
 if(NOT COMPILER_RT_STANDALONE_BUILD)
   list(APPEND PROFILE_TEST_DEPS llvm-profdata llvm-cov)
-  if(NOT APPLE AND COMPILER_RT_HAS_LLD AND "lld" IN_LIST LLVM_ENABLE_PROJECTS)
+  if(COMPILER_RT_HAS_LLD AND "lld" IN_LIST LLVM_ENABLE_PROJECTS)
 list(APPEND PROFILE_TEST_DEPS lld)
   endif()
 endif()

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[compiler-rt] [clang] [llvm] [clang-tools-extra] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

2023-12-18 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 closed 
https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[clang] [compiler-rt] [llvm] [clang-tools-extra] [PGO][GlobalValue][LTO]In GlobalValues::getGlobalIdentifier, use semicolon as delimiter for local-linkage varibles. (PR #74008)

2023-12-13 Thread Mingming Liu via cfe-commits

https://github.com/minglotus-6 edited 
https://github.com/llvm/llvm-project/pull/74008
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


  1   2   3   >