Re: [PATCH] D20715: [docs] Document the source-based code coverage feature

[Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rL271454: [docs] Document the source-based code coverage 
feature (authored by vedantk).

Changed prior to commit:
  http://reviews.llvm.org/D20715?vs=59078=59322#toc

Repository:
  rL LLVM

http://reviews.llvm.org/D20715

Files:
  cfe/trunk/docs/SourceBasedCodeCoverage.rst
  cfe/trunk/docs/index.rst

Index: cfe/trunk/docs/index.rst
===
--- cfe/trunk/docs/index.rst
+++ cfe/trunk/docs/index.rst
@@ -33,6 +33,7 @@
ControlFlowIntegrity
LTOVisibility
SafeStack
+   SourceBasedCodeCoverage
Modules
MSVCCompatibility
CommandGuide/index
Index: cfe/trunk/docs/SourceBasedCodeCoverage.rst
===
--- cfe/trunk/docs/SourceBasedCodeCoverage.rst
+++ cfe/trunk/docs/SourceBasedCodeCoverage.rst
@@ -0,0 +1,186 @@
+==
+Source-based Code Coverage
+==
+
+.. contents::
+   :local:
+
+Introduction
+
+
+This document explains how to use clang's source-based code coverage feature.
+It's called "source-based" because it operates on AST and preprocessor
+information directly. This allows it to generate very precise coverage data.
+
+Clang ships two other code coverage implementations:
+
+* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
+  various sanitizers. It can provide up to edge-level coverage.
+
+* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
+
+From this point onwards "code coverage" will refer to the source-based kind.
+
+The code coverage workflow
+==
+
+The code coverage workflow consists of three main steps:
+
+1. Compiling with coverage enabled.
+
+2. Running the instrumented program.
+
+3. Creating coverage reports.
+
+The next few sections work through a complete, copy-'n-paste friendly example
+based on this program:
+
+.. code-block:: console
+
+% cat < foo.cc
+#define BAR(x) ((x) || (x))
+template  void foo(T x) {
+  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+}
+int main() {
+  foo(0);
+  foo(0);
+  return 0;
+}
+EOF
+
+Compiling with coverage enabled
+===
+
+To compile code with coverage enabled pass ``-fprofile-instr-generate
+-fcoverage-mapping`` to the compiler:
+
+.. code-block:: console
+
+# Step 1: Compile with coverage enabled.
+% clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
+
+Note that linking together code with and without coverage instrumentation is
+supported: any uninstrumented code simply won't be accounted for.
+
+Running the instrumented program
+
+
+The next step is to run the instrumented program. When the program exits it
+will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
+environment variable. If that variable does not exist the profile is written to
+``default.profraw`` in the current directory of the program.
+
+If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
+missing directory structure will be created.  Additionally, the following
+special **pattern strings** are replaced:
+
+* "%p" expands out to the process ID.
+
+* "%h" expands out to the hostname of the machine running the program.
+
+.. code-block:: console
+
+# Step 2: Run the program.
+% LLVM_PROFILE_FILE="foo.profraw" ./foo
+
+Creating coverage reports
+=
+
+Raw profiles have to be **indexed** before they can be used to generated
+coverage reports. This is done using the "merge" tool in ``llvm-profdata``, so
+named because it can combine and index profiles at the same time:
+
+.. code-block:: console
+
+# Step 3(a): Index the raw profile.
+% llvm-profdata merge -sparse foo.profraw -o foo.profdata
+
+There are multiple different ways to render coverage reports. One option is to
+generate a line-oriented report:
+
+.. code-block:: console
+
+# Step 3(b): Create a line-oriented coverage report.
+% llvm-cov show ./foo -instr-profile=foo.profdata
+
+To demangle any C++ identifiers in the ouput, use:
+
+.. code-block:: console
+% llvm-cov show ./foo -instr-profile=foo.profdata | c++filt -n
+
+This report includes a summary view as well as dedicated sub-views for
+templated functions and their instantiations. For our example program, we get
+distinct views for ``foo(...)`` and ``foo(...)``.  If
+``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays sub-line
+region counts (even in macro expansions):
+
+.. code-block:: console
+
+   20|1|#define BAR(x) ((x) || (x))
+   ^20 ^2
+2|2|template  void foo(T x) {
+   22|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+   ^22 ^20  ^20^20
+2|4|}
+

Re: [PATCH] D20715: [docs] Document the source-based code coverage feature

[Vedant Kumar via cfe-commits]

> On Jun 1, 2016, at 11:30 AM, Justin Bogner  wrote:
> 
> Vedant Kumar  writes:
>> vsk created this revision.
>> vsk added a reviewer: bogner.
>> vsk added subscribers: kcc, cfe-commits, silvas.
>> 
>> It would be helpful to have a user-friendly guide for code
>> coverage. There is some overlap with [1], but this document visits
>> issues which may affect users in more depth.
>> 
>> Prompted by: https://llvm.org/bugs/show_bug.cgi?id=27781
>> 
>> [1] http://llvm.org/docs/CoverageMappingFormat.html
> ...
>> vsk updated this revision to Diff 59078.
>> vsk marked an inline comment as done.
>> vsk added a comment.
>> 
>> - Actually link in the new document into Index.rst.
> 
> A couple of comments below, but since this is prose it's basically all
> just opinion. Feel free to commit any time - we can always make
> improvements in tree later.

Thanks for all the feedback! I've incorporated all of it with minor
changes.

Let's use r271454 as a starting point, then.

vedant

> 
>> 
>> http://reviews.llvm.org/D20715
>> 
>> Files:
>>  docs/SourceBasedCodeCoverage.rst
>>  docs/index.rst
>> 
>> Index: docs/index.rst
>> ===
>> --- docs/index.rst
>> +++ docs/index.rst
>> @@ -33,6 +33,7 @@
>>ControlFlowIntegrity
>>LTOVisibility
>>SafeStack
>> +   SourceBasedCodeCoverage
>>Modules
>>MSVCCompatibility
>>CommandGuide/index
>> Index: docs/SourceBasedCodeCoverage.rst
>> ===
>> --- /dev/null
>> +++ docs/SourceBasedCodeCoverage.rst
>> @@ -0,0 +1,187 @@
>> +==
>> +Source-based Code Coverage
>> +==
>> +
>> +.. contents::
>> +   :local:
>> +
>> +Introduction
>> +
>> +
>> +This document explains how to use clang's source-based code coverage 
>> feature.
>> +It's called "source-based" because it operates on AST and preprocessor
>> +information directly. This allows it to generate very precise coverage data.
>> +
>> +Clang ships two other code coverage implementations:
>> +
>> +* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
>> +  various sanitizers. It can provide up to edge-level coverage.
>> +
>> +* gcov - A GCC-compatible coverage implementation which operates on 
>> DebugInfo.
>> +
>> +From this point onwards "code coverage" will refer to the source-based kind.
>> +
>> +The code coverage workflow
>> +==
>> +
>> +The code coverage workflow consists of three main steps:
>> +
>> +1. Compiling with coverage enabled.
>> +
>> +2. Running the instrumented program.
>> +
>> +3. Creating coverage reports.
>> +
>> +The next few sections work through a complete, copy-'n-paste friendly 
>> example
>> +based on this program:
>> +
>> +.. code-block:: console
>> +
>> +% cat < foo.cc
>> +#define BAR(x) ((x) || (x))
>> +template  void foo(T x) {
>> +  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
>> +}
>> +int main() {
>> +  foo(0);
>> +  foo(0);
>> +  return 0;
>> +}
>> +EOF
>> +
>> +Compiling with coverage enabled
>> +===
>> +
>> +To compile code with coverage enabled pass ``-fprofile-instr-generate
>> +-fcoverage-mapping`` to the compiler:
>> +
>> +.. code-block:: console
>> +
>> +# Step 1: Compile with coverage enabled.
>> +% clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
>> +
>> +Note that linking together code with and without coverage instrumentation is
>> +supported: any uninstrumented code simply won't be accounted for.
>> +
>> +Running the instrumented program
>> +
>> +
>> +The next step is to run the instrumented program. When the program exits it
>> +will write a **raw profile** to the path specified by the 
>> ``LLVM_PROFILE_FILE``
>> +environment variable. If that variable does not exist the profile is 
>> written to
>> +``./default.profraw``.
> 
> Something like "``default.profraw`` in the current directory of the
> program" might be clearer.
> 
>> +
>> +If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
>> +missing directory structure will be created.  Additionally, the following
>> +special **pattern strings** are replaced:
>> +
>> +* "%p" expands out to the PID.
> 
> Maybe spell out process ID?
> 
>> +
>> +* "%h" expands out to the hostname of the machine running the program.
>> +
>> +.. code-block:: console
>> +
>> +# Step 2: Run the program.
>> +% LLVM_PROFILE_FILE="foo.profraw" ./foo
>> +
>> +Creating coverage reports
>> +=
>> +
>> +Raw profiles have to be **indexed** before they can be used to generated
>> +coverage reports:
>> +
>> +.. code-block:: console
>> +
>> +# Step 3(a): Index the raw profile.
>> +% llvm-profdata merge -sparse foo.profraw -o foo.profdata
>> +
>> +You may be wondering why raw profiles aren't indexed 

Re: [PATCH] D20715: [docs] Document the source-based code coverage feature

[Justin Bogner via cfe-commits]

Vedant Kumar  writes:
> vsk created this revision.
> vsk added a reviewer: bogner.
> vsk added subscribers: kcc, cfe-commits, silvas.
>
> It would be helpful to have a user-friendly guide for code
> coverage. There is some overlap with [1], but this document visits
> issues which may affect users in more depth.
>
> Prompted by: https://llvm.org/bugs/show_bug.cgi?id=27781
>
> [1] http://llvm.org/docs/CoverageMappingFormat.html
 ...
> vsk updated this revision to Diff 59078.
> vsk marked an inline comment as done.
> vsk added a comment.
>
> - Actually link in the new document into Index.rst.

A couple of comments below, but since this is prose it's basically all
just opinion. Feel free to commit any time - we can always make
improvements in tree later.

>
> http://reviews.llvm.org/D20715
>
> Files:
>   docs/SourceBasedCodeCoverage.rst
>   docs/index.rst
>
> Index: docs/index.rst
> ===
> --- docs/index.rst
> +++ docs/index.rst
> @@ -33,6 +33,7 @@
> ControlFlowIntegrity
> LTOVisibility
> SafeStack
> +   SourceBasedCodeCoverage
> Modules
> MSVCCompatibility
> CommandGuide/index
> Index: docs/SourceBasedCodeCoverage.rst
> ===
> --- /dev/null
> +++ docs/SourceBasedCodeCoverage.rst
> @@ -0,0 +1,187 @@
> +==
> +Source-based Code Coverage
> +==
> +
> +.. contents::
> +   :local:
> +
> +Introduction
> +
> +
> +This document explains how to use clang's source-based code coverage feature.
> +It's called "source-based" because it operates on AST and preprocessor
> +information directly. This allows it to generate very precise coverage data.
> +
> +Clang ships two other code coverage implementations:
> +
> +* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
> +  various sanitizers. It can provide up to edge-level coverage.
> +
> +* gcov - A GCC-compatible coverage implementation which operates on 
> DebugInfo.
> +
> +From this point onwards "code coverage" will refer to the source-based kind.
> +
> +The code coverage workflow
> +==
> +
> +The code coverage workflow consists of three main steps:
> +
> +1. Compiling with coverage enabled.
> +
> +2. Running the instrumented program.
> +
> +3. Creating coverage reports.
> +
> +The next few sections work through a complete, copy-'n-paste friendly example
> +based on this program:
> +
> +.. code-block:: console
> +
> +% cat < foo.cc
> +#define BAR(x) ((x) || (x))
> +template  void foo(T x) {
> +  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
> +}
> +int main() {
> +  foo(0);
> +  foo(0);
> +  return 0;
> +}
> +EOF
> +
> +Compiling with coverage enabled
> +===
> +
> +To compile code with coverage enabled pass ``-fprofile-instr-generate
> +-fcoverage-mapping`` to the compiler:
> +
> +.. code-block:: console
> +
> +# Step 1: Compile with coverage enabled.
> +% clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
> +
> +Note that linking together code with and without coverage instrumentation is
> +supported: any uninstrumented code simply won't be accounted for.
> +
> +Running the instrumented program
> +
> +
> +The next step is to run the instrumented program. When the program exits it
> +will write a **raw profile** to the path specified by the 
> ``LLVM_PROFILE_FILE``
> +environment variable. If that variable does not exist the profile is written 
> to
> +``./default.profraw``.

Something like "``default.profraw`` in the current directory of the
program" might be clearer.

> +
> +If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
> +missing directory structure will be created.  Additionally, the following
> +special **pattern strings** are replaced:
> +
> +* "%p" expands out to the PID.

Maybe spell out process ID?

> +
> +* "%h" expands out to the hostname of the machine running the program.
> +
> +.. code-block:: console
> +
> +# Step 2: Run the program.
> +% LLVM_PROFILE_FILE="foo.profraw" ./foo
> +
> +Creating coverage reports
> +=
> +
> +Raw profiles have to be **indexed** before they can be used to generated
> +coverage reports:
> +
> +.. code-block:: console
> +
> +# Step 3(a): Index the raw profile.
> +% llvm-profdata merge -sparse foo.profraw -o foo.profdata
> +
> +You may be wondering why raw profiles aren't indexed automatically. In
> +real-world projects multiple profiles have to be merged together before a
> +report can be generated. This merge step is inevitable, so it makes sense to
> +handle the compute-intensive indexing process at that point. A separate
> +indexing step has the added benefit of keeping the compiler runtime small and
> +simple.

This comment feels like it comes out of nowhere, and I'm 

Re: [PATCH] D20715: [docs] Document the source-based code coverage feature

[Vedant Kumar via cfe-commits]

vsk updated this revision to Diff 59078.
vsk marked an inline comment as done.
vsk added a comment.

- Actually link in the new document into Index.rst.


http://reviews.llvm.org/D20715

Files:
  docs/SourceBasedCodeCoverage.rst
  docs/index.rst

Index: docs/index.rst
===
--- docs/index.rst
+++ docs/index.rst
@@ -33,6 +33,7 @@
ControlFlowIntegrity
LTOVisibility
SafeStack
+   SourceBasedCodeCoverage
Modules
MSVCCompatibility
CommandGuide/index
Index: docs/SourceBasedCodeCoverage.rst
===
--- /dev/null
+++ docs/SourceBasedCodeCoverage.rst
@@ -0,0 +1,187 @@
+==
+Source-based Code Coverage
+==
+
+.. contents::
+   :local:
+
+Introduction
+
+
+This document explains how to use clang's source-based code coverage feature.
+It's called "source-based" because it operates on AST and preprocessor
+information directly. This allows it to generate very precise coverage data.
+
+Clang ships two other code coverage implementations:
+
+* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
+  various sanitizers. It can provide up to edge-level coverage.
+
+* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
+
+From this point onwards "code coverage" will refer to the source-based kind.
+
+The code coverage workflow
+==
+
+The code coverage workflow consists of three main steps:
+
+1. Compiling with coverage enabled.
+
+2. Running the instrumented program.
+
+3. Creating coverage reports.
+
+The next few sections work through a complete, copy-'n-paste friendly example
+based on this program:
+
+.. code-block:: console
+
+% cat < foo.cc
+#define BAR(x) ((x) || (x))
+template  void foo(T x) {
+  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+}
+int main() {
+  foo(0);
+  foo(0);
+  return 0;
+}
+EOF
+
+Compiling with coverage enabled
+===
+
+To compile code with coverage enabled pass ``-fprofile-instr-generate
+-fcoverage-mapping`` to the compiler:
+
+.. code-block:: console
+
+# Step 1: Compile with coverage enabled.
+% clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
+
+Note that linking together code with and without coverage instrumentation is
+supported: any uninstrumented code simply won't be accounted for.
+
+Running the instrumented program
+
+
+The next step is to run the instrumented program. When the program exits it
+will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
+environment variable. If that variable does not exist the profile is written to
+``./default.profraw``.
+
+If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
+missing directory structure will be created.  Additionally, the following
+special **pattern strings** are replaced:
+
+* "%p" expands out to the PID.
+
+* "%h" expands out to the hostname of the machine running the program.
+
+.. code-block:: console
+
+# Step 2: Run the program.
+% LLVM_PROFILE_FILE="foo.profraw" ./foo
+
+Creating coverage reports
+=
+
+Raw profiles have to be **indexed** before they can be used to generated
+coverage reports:
+
+.. code-block:: console
+
+# Step 3(a): Index the raw profile.
+% llvm-profdata merge -sparse foo.profraw -o foo.profdata
+
+You may be wondering why raw profiles aren't indexed automatically. In
+real-world projects multiple profiles have to be merged together before a
+report can be generated. This merge step is inevitable, so it makes sense to
+handle the compute-intensive indexing process at that point. A separate
+indexing step has the added benefit of keeping the compiler runtime small and
+simple.
+
+There are multiple different ways to render coverage reports. One option is to
+generate a line-oriented report:
+
+.. code-block:: console
+
+# Step 3(b): Create a line-oriented coverage report.
+% llvm-cov show ./foo -instr-profile=foo.profdata
+
+This report includes a summary view as well as dedicated sub-views for
+templated functions and their instantiations. For our example program, we get
+distinct views for ``foo(...)`` and ``foo(...)``.  If
+``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays sub-line
+region counts (even in macro expansions):
+
+.. code-block:: console
+
+   20|1|#define BAR(x) ((x) || (x))
+   ^20 ^2
+2|2|template  void foo(T x) {
+   22|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+   ^22 ^20  ^20^20
+2|4|}
+--
+| _Z3fooIiEvT_:
+|  1|2|template  void foo(T x) {
+| 11|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+|

Re: [PATCH] D20715: [docs] Document the source-based code coverage feature

[Vedant Kumar via cfe-commits]

vsk updated this revision to Diff 58810.
vsk marked 4 inline comments as done.
vsk added a comment.

Thanks for the feedback!

- Addressed Sean's review comments.
- Fixed the line count displayed in the summary view of `foo`.


http://reviews.llvm.org/D20715

Files:
  docs/SourceBasedCodeCoverage.rst

Index: docs/SourceBasedCodeCoverage.rst
===
--- /dev/null
+++ docs/SourceBasedCodeCoverage.rst
@@ -0,0 +1,187 @@
+==
+Source-based Code Coverage
+==
+
+.. contents::
+   :local:
+
+Introduction
+
+
+This document explains how to use clang's source-based code coverage feature.
+It's called "source-based" because it operates on AST and preprocessor
+information directly. This allows it to generate very precise coverage data.
+
+Clang ships two other code coverage implementations:
+
+* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
+  various sanitizers. It can provide up to edge-level coverage.
+
+* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
+
+From this point onwards "code coverage" will refer to the source-based kind.
+
+The code coverage workflow
+==
+
+The code coverage workflow consists of three main steps:
+
+1. Compiling with coverage enabled.
+
+2. Running the instrumented program.
+
+3. Creating coverage reports.
+
+The next few sections work through a complete, copy-'n-paste friendly example
+based on this program:
+
+.. code-block:: console
+
+% cat < foo.cc
+#define BAR(x) ((x) || (x))
+template  void foo(T x) {
+  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+}
+int main() {
+  foo(0);
+  foo(0);
+  return 0;
+}
+EOF
+
+Compiling with coverage enabled
+===
+
+To compile code with coverage enabled pass ``-fprofile-instr-generate
+-fcoverage-mapping`` to the compiler:
+
+.. code-block:: console
+
+# Step 1: Compile with coverage enabled.
+% clang++ -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
+
+Note that linking together code with and without coverage instrumentation is
+supported: any uninstrumented code simply won't be accounted for.
+
+Running the instrumented program
+
+
+The next step is to run the instrumented program. When the program exits it
+will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
+environment variable. If that variable does not exist the profile is written to
+``./default.profraw``.
+
+If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
+missing directory structure will be created.  Additionally, the following
+special **pattern strings** are replaced:
+
+* "%p" expands out to the PID.
+
+* "%h" expands out to the hostname of the machine running the program.
+
+.. code-block:: console
+
+# Step 2: Run the program.
+% LLVM_PROFILE_FILE="foo.profraw" ./foo
+
+Creating coverage reports
+=
+
+Raw profiles have to be **indexed** before they can be used to generated
+coverage reports:
+
+.. code-block:: console
+
+# Step 3(a): Index the raw profile.
+% llvm-profdata merge -sparse foo.profraw -o foo.profdata
+
+You may be wondering why raw profiles aren't indexed automatically. In
+real-world projects multiple profiles have to be merged together before a
+report can be generated. This merge step is inevitable, so it makes sense to
+handle the compute-intensive indexing process at that point. A separate
+indexing step has the added benefit of keeping the compiler runtime small and
+simple.
+
+There are multiple different ways to render coverage reports. One option is to
+generate a line-oriented report:
+
+.. code-block:: console
+
+# Step 3(b): Create a line-oriented coverage report.
+% llvm-cov show ./foo -instr-profile=foo.profdata
+
+This report includes a summary view as well as dedicated sub-views for
+templated functions and their instantiations. For our example program, we get
+distinct views for ``foo(...)`` and ``foo(...)``.  If
+``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays nested
+region counts (even in macro expansions):
+
+.. code-block:: console
+
+   20|1|#define BAR(x) ((x) || (x))
+   ^20 ^2
+2|2|template  void foo(T x) {
+   22|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+   ^22 ^20  ^20^20
+2|4|}
+--
+| _Z3fooIiEvT_:
+|  1|2|template  void foo(T x) {
+| 11|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+| ^11 ^10  ^10^10
+|  1|4|}
+--
+| _Z3fooIfEvT_:
+|  1|2|template  void foo(T x) {
+| 11|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+|  

Re: [PATCH] D20715: [docs] Document the source-based code coverage feature

[Sean Silva via cfe-commits]

silvas added subscribers: MaggieYi, phillip.power.
silvas added a comment.

This is looking really good. So based on reading this, a rough sketch of how we 
can expand this to PGO would be something like:

- there is a separate PGO page that has similar structure, but is more tuned 
for the PGO use case.
- there is a separate page (maybe in the LLVM docs and not clang since other 
frontends might be using libprofile?) which describes "advanced usage of the 
profile runtime". That would cover the file and buffer API, 
`__llvm_profile_runtime`, runtime counter resetting and merging, etc.

Adding Maggie and Phillip, which are working on coverage at PlayStation.



Comment at: docs/SourceBasedCodeCoverage.rst:33
@@ +32,3 @@
+
+3. Create a report out of the generated profile.
+

Would it be possible to make these consistent with the later section headers?


Comment at: docs/SourceBasedCodeCoverage.rst:61
@@ +60,3 @@
+# Step 1: Compile with coverage enabled.
+% cc -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
+

Since these are clang's docs, we should just say `clang` (or `clang++`)


Comment at: docs/SourceBasedCodeCoverage.rst:91
@@ +90,3 @@
+Raw profiles have to be **indexed** before they can be used to generated
+coverage reports:
+

As a user, this would make me ask "why can't llvm-profdata do the indexing if I 
pass it a raw profile"?

I think the answer is largely that in a real program there are multiple 
profraws, so you need to merge them first. You probably want to mention that.


Comment at: docs/SourceBasedCodeCoverage.rst:140
@@ +139,3 @@
+# Step 3(c): Create a coverage summary.
+% llvm-cov report ./foo -instr-profile=foo.profdata
+

Can you show example output? That is one of the most useful things in a how-to 
document like this. (for example, a person reading this might be a prospective 
user of the feature, and seeing the output is valuable to them to see if it 
provides what they want)


Comment at: docs/SourceBasedCodeCoverage.rst:149
@@ +148,3 @@
+* Raw profiles can be discarded after they are indexed. It **is** possible for
+  instrumented programs to merge profiling information directly into an 
existing
+  raw profile on disk. The details are out of this document's scope.

Small wording nit: Instead of saying "It is possible for an instrumented 
program to..." I would say something like "Advanced use of the profile runtime 
library allows an instrumented program to..." or something like that.


http://reviews.llvm.org/D20715



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D20715: [docs] Document the source-based code coverage feature

[Vedant Kumar via cfe-commits]

vsk created this revision.
vsk added a reviewer: bogner.
vsk added subscribers: kcc, cfe-commits, silvas.

It would be helpful to have a user-friendly guide for code coverage. There is 
some overlap with [1], but this document visits issues which may affect users 
in more depth.

Prompted by: https://llvm.org/bugs/show_bug.cgi?id=27781

[1] http://llvm.org/docs/CoverageMappingFormat.html

http://reviews.llvm.org/D20715

Files:
  docs/SourceBasedCodeCoverage.rst

Index: docs/SourceBasedCodeCoverage.rst
===
--- /dev/null
+++ docs/SourceBasedCodeCoverage.rst
@@ -0,0 +1,174 @@
+==
+Source-based Code Coverage
+==
+
+.. contents::
+   :local:
+
+Introduction
+
+
+This document explains how to use clang's source-based code coverage feature.
+It's called "source-based" because it operates on AST and preprocessor
+information directly. This allows it to generate very precise coverage data.
+
+Clang ships two other code coverage implementations:
+
+* :doc:`SanitizerCoverage` - A low-overhead tool meant for use alongside the
+  various sanitizers. It can provide up to edge-level coverage.
+
+* gcov - A GCC-compatible coverage implementation which operates on DebugInfo.
+
+From this point onwards "code coverage" will refer to the source-based kind.
+
+The code coverage workflow
+==
+
+The code coverage workflow consists of three main steps:
+
+1. Compile with coverage enabled.
+
+2. Run the program.
+
+3. Create a report out of the generated profile.
+
+The next few sections work through a complete, copy-'n-paste friendly example
+based on this program:
+
+.. code-block:: console
+
+% cat < foo.cc
+#define BAR(x) ((x) || (x))
+template  void foo(T x) {
+  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+}
+int main() {
+  foo(0);
+  foo(0);
+  return 0;
+}
+EOF
+
+Compiling with coverage enabled
+===
+
+To compile code with coverage enabled pass ``-fprofile-instr-generate
+-fcoverage-mapping`` to the compiler:
+
+.. code-block:: console
+
+# Step 1: Compile with coverage enabled.
+% cc -fprofile-instr-generate -fcoverage-mapping foo.cc -o foo
+
+Note that linking together code with and without coverage instrumentation is
+supported: any uninstrumented code simply won't be accounted for.
+
+Running the instrumented program
+
+
+The next step is to run the instrumented program. When the program exits it
+will write a **raw profile** to the path specified by the ``LLVM_PROFILE_FILE``
+environment variable. If that variable does not exist the profile is written to
+``./default.profraw``.
+
+If ``LLVM_PROFILE_FILE`` contains a path to a non-existent directory the
+missing directory structure will be created.  Additionally, the following
+special **pattern strings** are replaced:
+
+* "%p" expands out to the PID.
+
+* "%h" expands out to the hostname of the machine running the program.
+
+.. code-block:: console
+
+# Step 2: Run the program.
+% LLVM_PROFILE_FILE="foo.profraw" ./foo
+
+Creating coverage reports
+=
+
+Raw profiles have to be **indexed** before they can be used to generated
+coverage reports:
+
+.. code-block:: console
+
+# Step 3(a): Index the raw profile.
+% llvm-profdata merge -sparse foo.profraw -o foo.profdata
+
+There are multiple different ways to render coverage reports. One option is to
+generate a line-oriented report:
+
+.. code-block:: console
+
+# Step 3(b): Create a line-oriented coverage report.
+% llvm-cov show ./foo -instr-profile=foo.profdata
+
+This report includes a summary view as well as dedicated sub-views for
+templated functions and their instantiations. For our example program, we get
+distinct views for ``foo(...)`` and ``foo(...)``.  If
+``-show-line-counts-or-regions`` is enabled, ``llvm-cov`` displays nested
+region counts (even in macro expansions):
+
+.. code-block:: console
+
+   20|1|#define BAR(x) ((x) || (x))
+   ^20 ^2
+2|2|template  void foo(T x) {
+   22|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+   ^22 ^20  ^20^20
+1|4|}
+--
+| _Z3fooIiEvT_:
+|  1|2|template  void foo(T x) {
+| 11|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+| ^11 ^10  ^10^10
+|  1|4|}
+--
+| _Z3fooIfEvT_:
+|  1|2|template  void foo(T x) {
+| 11|3|  for (unsigned I = 0; I < 10; ++I) { BAR(I); }
+| ^11 ^10  ^10^10
+|  1|4|}
+--
+
+It's possible to generate a file-level summary of coverage statistics (instead
+of a line-oriented report) with:
+
+..