[PATCH] D155445: [analyzer][docs] Add CSA release notes

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie accepted this revision.
OikawaKirie added a comment.

LGTM for my part. Thx.

Since I am not very familiar with other changes, I have no detailed suggestions 
for the order.




Comment at: clang/docs/ReleaseNotes.rst:922-923
+- The ``CStringChecker`` will invalidate less if the copy operation is
+  inferable to be bounded. For example, if the argument of ``strcpy`` is known
+  to be of certain length and that is in-bounds.
+

The lengths of both src and dst buffers need to be known.



Comment at: clang/docs/ReleaseNotes.rst:937
+
+  Similarly, functions like ``strsep`` now won't invalidate the source buffer,
+  because it can never overflow.

I think this may be a typo here, as we do not invalidate the source buffer 
originally.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D155445/new/

https://reviews.llvm.org/D155445

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D155445: [analyzer][docs] Add CSA release notes

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

The key idea of my commit 1bd2d335b649 
:

- For string APIs that will not provide the copy length (`strcpy`), we will use 
the buffer decl and literal length to infer whether it overflows. If the copy 
operation does not overflow, we will now only invalidate the buffer string 
being copied to.
- For string APIs that never overflow (`strsep`), we will always invalidate the 
target buffer only.
- For those that we cannot correctly handle now (`std::copy`), we will also 
invalidate the base region and make all pointers in the base region escape.

Hence,
For `strcpy`s, we infer through buffer size and string literals.
For `strsep`, we believe it never overflows through its functionality 
specification. It is also an inference.

Whereas for `memcpy` where the copy length is given in arguments, the 
non-inferring circumstances, it was implemented previously in patch D12571 
, not a part of my changes.




Comment at: clang/docs/ReleaseNotes.rst:920-922
+- The ``CStringChecker`` will invalidate less if the copy operation is bounded.
+  (`1bd2d335b649 `_)
+  (`#55019 `_)

One tiny change to the abstraction.
The ``CStringChecker`` will invalidate less if the copy operation is 
**inferable to be** bounded.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D155445/new/

https://reviews.llvm.org/D155445

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG1bd2d335b649: [analyzer][CStringChecker] Adjust the 
invalidation operation on the super… (authored by OikawaKirie).

Changed prior to commit:
  https://reviews.llvm.org/D152435?vs=536662=536683#toc

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -556,12 +556,13 @@
   x263.s2 = strdup("hello");
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
-  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}\
+  expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void 

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I will submit it again later to update the code as you suggested.

Besides,

> Build Status
> Buildable 242737  
> Build 376801: pre-merge checkslibcxx CI FAILED

What does this mean? Was this patch correctly compiled and checked in the 
previous build? How can I avoid this failure?

Thx


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 536662.
OikawaKirie added a comment.

Sorry for the formatting error... :(


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -556,12 +556,13 @@
   x263.s2 = strdup("hello");
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
-  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}\
+  expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void *memcpy(void *dst, const void *src, size_t n);
+void *memset(void *s, int c, size_t n);
 
 typedef unsigned long __darwin_pthread_key_t;
 typedef __darwin_pthread_key_t pthread_key_t;
Index: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:1054
+SVal SizeV, QualType SizeTy) {
+  return InvalidateBufferAux(
+  C, S, BufE, BufV,

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > Shouldn't we assert that `SizeV` is some known value or at least smaller 
> > > than (or equal to) the extent of the buffer?
> > The calls in `CStringChecker::evalCopyCommon` and 
> > `CStringChecker::evalStrcpyCommon` seem to have chances able to pass an 
> > unknown value. But I am not very sure about this.
> > 
> > If the `SizeV` is sure to be smaller or equal to the extent of the buffer, 
> > the `IsFirstBufInBound` check seems useless.
> > Maybe I need to dig deeper into the callers.
> Indeed.
Function `CStringChecker::CheckBufferAccess` is responsible to check whether 
the size overflows.
However, when `Filter.CheckCStringOutOfBounds` is set to false, the function 
will skip the overflow checking.
This makes that we cannot assert SizeV is always inbound, and the check in 
`IsFirstBufInBound` seems to be necessary.
(`evalMemset` -> `CheckBufferAccess`(skip overflow checking) -> `memsetAux` -> 
`InvalidateDestinationBufferBySize` with function call of `memset(x.arr, 'a', 
42)`)

Besides, if the size argument is originally an unknown value expression (e.g. 
`(unsigned char) 1.0`), the unknown value can be passed to this function.
(`CStringChecker::evalCopyCommon` -> `invalidateDestinationBufferBySize`, with 
function call of `memcpy(x.arr, "hi", (unsigned char) 1.0)`)

Hence, we cannot make such an assertion here. : (


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 536578.
OikawaKirie marked 12 inline comments as done and an inline comment as not done.
OikawaKirie added a comment.

1. Function and variable names: functions: lower camel, variables: upper camel, 
lambdas: upper camel
2. std::function -> llvm::function_ref
3. update test case verification directions by following others in this file.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -556,12 +556,13 @@
   x263.s2 = strdup("hello");
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
-  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}\
+  expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void 

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

BTW, what does the `Done` checkbox mean in the code comments?




Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:286-287
+  SVal V,
+  std::function);
 

steakhal wrote:
> I'd highly suggest making this a template taking the functor that way.
> Given that we modify these functions, we should make them comply with the 
> lower camel case naming convention.
> And their variables with upper camel case.
> I'd highly suggest making this a template taking the functor that way.

Any examples?
Do you mean
```
template 
static ProgramStateRef
InvalidateBufferAux(CheckerContext , ProgramStateRef state, const Expr *Ex,
SVal V, std::function CallBack);
```



Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:1054
+SVal SizeV, QualType SizeTy) {
+  return InvalidateBufferAux(
+  C, S, BufE, BufV,

steakhal wrote:
> Shouldn't we assert that `SizeV` is some known value or at least smaller than 
> (or equal to) the extent of the buffer?
The calls in `CStringChecker::evalCopyCommon` and 
`CStringChecker::evalStrcpyCommon` seem to have chances able to pass an unknown 
value. But I am not very sure about this.

If the `SizeV` is sure to be smaller or equal to the extent of the buffer, the 
`IsFirstBufInBound` check seems useless.
Maybe I need to dig deeper into the callers.



Comment at: clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp:1075-1077
+  [](RegionAndSymbolInvalidationTraits &, const MemRegion *R) {
+return MemRegion::FieldRegionKind == R->getKind();
+  });

steakhal wrote:
> The lambdas inline in the call look a bit messy. I would prefer declaring it 
> and passing it as an argument separately.
> This applies to the rest of the lambdas as well.
> And their variables with upper camel case.

What about the variable names for the lambdas? Lower camel or upper camel?
I cannot find the rules for named lambdas from the guidance 
.



Comment at: clang/test/Analysis/issue-55019.cpp:80-87
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.

> So, you say that It should be `HeapSymRegion` instead because `x.arr` 
> shouldn't be invalidated because the `std::copy` won't modify it. Right?

Yes, but just in this test case.

The size of `x.arr` is 4 and the copy length is 1, which means the call to 
`std::copy` here never overflows.
Therefore, we should not invalidate the super region `x` and keep `x.ptr` still 
pointing to the original `HeapSymRegion` unchanged.
However, the copy length computed through iterators is not modeled currently in 
`CStringChecker::evalStdCopyCommon`, where 
`InvalidateDestinationBufferAlwaysEscapeSuperRegion` is called.
When it is modeled in the future, the function should call 
`InvalidateDestinationBufferBySize` instead to make the invalidation of the 
super region determined by the copy length to report potential leaks.



Comment at: clang/test/Analysis/pr22954.c:560
   clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  // expected-warning@-1{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}

steakhal wrote:
> Ah, the leak should have been reported at the end of the full-expression of 
> `memcpy`.
> If we have this expect line here it could mislead the reader that it has 
> anything to do with the `eval` call before.
> 
> In fact, any statement after `memcpy` would have this diagnostic.
> I have seen cases where we have a `next_line()` opaque call after such places 
> to anchor the leak diagnostic.
> I think we can do the same here.
> Its not advised to have different, but similar looking hunks out of which 
> some are the consequence of the semantic change we made and also have others 
> where they are just refactors.

Do you mean I only need to update the verification direction here as you 
suggested above and leave others unchanged?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/test/Analysis/issue-55019.cpp:13-14
+
+void *malloc(size_t);
+void free(void *);
+

> Ah, I see that it's for c function declarations. If that's the case, have you 
> considered adding the malloc and free declarations to that header?

What about doing this in another patch and updating all test cases that use 
malloc and free? Maybe other libc APIs as well?
The test case malloc-three-arg.c declares malloc and in a different signature 
and also includes this header. Simply doing so in this patch will lead to other 
conflicts.



Comment at: clang/test/Analysis/pr22954.c:581
   clang_analyzer_eval(m27.s3[i] == 1); // expected-warning{{UNKNOWN}}\
   expected-warning{{Potential leak of memory pointed to by 'm27.s4'}}
   return 0;

Do I need to update all other leak expectation directions in this file as well? 
Such as this one here.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 532372.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.

Update the implementation of InvalidateBuffer in a multi-entrance-with-callback 
manner.
Update test cases as suggested.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D152435/new/

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -557,11 +557,12 @@
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
   clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  // expected-warning@-1{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,89 @@
+// Refer issue 55019 for more details.
+// A supplemental test case of pr22954.c for other functions modeled in
+// the CStringChecker.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+#include "Inputs/system-header-simulator-cxx.h"
+
+void *malloc(size_t);
+void free(void *);
+
+struct mystruct {
+  void *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strcpy(x.arr, "hi");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+void fstrncpy() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  strncpy(x.arr, "hi", sizeof(x.arr));
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStrsep
+void fstrsep() {
+  mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  char *p = x.arr;
+  (void)strsep(, "x");
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-leak-warning
+}
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we currently cannot know whether the copy overflows, the checker
+  // invalidates the entire `x` object. When the copy size through iterators
+  // can be correctly modeled, we can then update the verify direction from
+  // SymRegion to HeapSymRegion as this std::copy call never overflows and
+  // hence the pointer `x.ptr` shall not be invalidated.
+  clang_analyzer_dump(x.ptr);   // expected-warning {{SymRegion}}
+  delete static_cast(x.ptr); // no-leak-warning
+}
Index: clang/test/Analysis/Inputs/system-header-simulator.h
===
--- clang/test/Analysis/Inputs/system-header-simulator.h
+++ clang/test/Analysis/Inputs/system-header-simulator.h
@@ -63,7 +63,9 @@
 
 char *strcpy(char *restrict, const char *restrict);
 char *strncpy(char *dst, const char *src, size_t n);
+char *strsep(char **stringp, const char *delim);
 void *memcpy(void *dst, const void *src, size_t n);
+void *memset(void *s, int c, size_t n);
 
 typedef unsigned long __darwin_pthread_key_t;
 typedef __darwin_pthread_key_t pthread_key_t;
Index: 

[PATCH] D152435: [analyzer][CStringChecker] Adjust the invalidation operation on the super region of the destination buffer during string copy

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: pgousseau, NoQ, steakhal, balazske, xazax.hun.
OikawaKirie added a project: clang.
Herald added subscribers: manas, ASDenysPetrov, martong, dkrupp, donat.nagy, 
Szelethus, mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware.
Herald added a project: All.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Fixing GitHub issue: https://github.com/llvm/llvm-project/issues/55019
Following the previous fix https://reviews.llvm.org/D12571 on issue 
https://github.com/llvm/llvm-project/issues/23328

The two issues report false memory leaks after calling string-copy APIs with a 
buffer field in an object as the destination.
The buffer invalidation incorrectly drops the assignment to a heap memory block 
when no overflow problems happen.
And the pointer of the dropped assignment is declared in the same object of the 
destination buffer.

The previous fix only considers the `memcpy` functions whose copy length is 
available from arguments.
In this issue, the copy length is inferable from the buffer declaration and 
string literals being copied.
Therefore, I have adjusted the previous fix to reuse the copy length computed 
before.

Besides, for APIs that never overflow (strsep) or we never know whether they 
can overflow (std::copy),
new flags have been introduced to inform CStringChecker::InvalidateBuffer 
whether or not to
invalidate the super region that encompasses the destination buffer.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D152435

Files:
  clang/lib/StaticAnalyzer/Checkers/CStringChecker.cpp
  clang/test/Analysis/Inputs/system-header-simulator.h
  clang/test/Analysis/issue-55019.c
  clang/test/Analysis/issue-55019.cpp
  clang/test/Analysis/pr22954.c

Index: clang/test/Analysis/pr22954.c
===
--- clang/test/Analysis/pr22954.c
+++ clang/test/Analysis/pr22954.c
@@ -557,11 +557,12 @@
   char input[] = {'a', 'b', 'c', 'd'};
   memcpy(x263.s1, input, *(len + n));
   clang_analyzer_eval(x263.s1[0] == 0); // expected-warning{{UNKNOWN}}
+  // expected-warning@-1{{Potential leak of memory pointed to by 'x263.s2'}}
   clang_analyzer_eval(x263.s1[1] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[2] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s1[3] == 0); // expected-warning{{UNKNOWN}}
   clang_analyzer_eval(x263.s2 == 0); // expected-warning{{UNKNOWN}}
-  return 0; // expected-warning{{Potential leak of memory pointed to by 'x263.s2'}}
+  return 0;
 }
 
 
Index: clang/test/Analysis/issue-55019.cpp
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.cpp
@@ -0,0 +1,29 @@
+// Refer issue 55019 for more details.
+
+// RUN: %clang_analyze_cc1 %s -verify \
+// RUN:   -analyzer-checker=core,debug.ExprInspection
+
+#include "Inputs/system-header-simulator-cxx.h"
+
+struct mystruct {
+  char *ptr;
+  char arr[4];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::evalStdCopyCommon
+void fstdcopy() {
+  mystruct x;
+  x.ptr = new char;
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+
+  const char *p = "x";
+  std::copy(p, p + 1, x.arr);
+
+  // FIXME: As we cannot know whether the copy overflows, we will invalidate the
+  // entire object. Modify the verify direction from SymRegion to HeapSymRegion
+  // when the size if modeled in CStringChecker.
+  clang_analyzer_dump(x.ptr); // expected-warning {{SymRegion}}
+  delete x.ptr;   // no-warning
+}
Index: clang/test/Analysis/issue-55019.c
===
--- /dev/null
+++ clang/test/Analysis/issue-55019.c
@@ -0,0 +1,69 @@
+// Refer issue 55019 for more details.
+
+// RUN: %clang_analyze_cc1 %s -verify -Wno-strict-prototypes \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-checker=unix \
+// RUN:   -analyzer-checker=debug.ExprInspection
+
+#include "Inputs/system-header-simulator.h"
+void *malloc(size_t);
+void free(void *);
+
+#define SIZE 4
+
+struct mystruct {
+  void *ptr;
+  char arr[SIZE];
+};
+
+void clang_analyzer_dump(const void *);
+
+// CStringChecker::memsetAux
+void fmemset() {
+  struct mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memset(x.arr, 0, SIZE);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-warning
+}
+
+// CStringChecker::evalCopyCommon
+void fmemcpy() {
+  struct mystruct x;
+  x.ptr = malloc(1);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  memcpy(x.arr, "hi", 2);
+  clang_analyzer_dump(x.ptr); // expected-warning {{HeapSymRegion}}
+  free(x.ptr);// no-warning
+}
+
+// CStringChecker::evalStrcpyCommon
+void fstrcpy() {
+  struct mystruct x;
+  x.ptr = malloc(1);
+  

[PATCH] D129737: [analyzer] Fixing SVal::getType returns Null Type for NonLoc::ConcreteInt in boolean type

[Ella Ma via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG32fe1a4be95c: [analyzer] Fixing SVal::getType returns Null 
Type for NonLoc::ConcreteInt in… (authored by OikawaKirie).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D129737/new/

https://reviews.llvm.org/D129737

Files:
  clang/lib/StaticAnalyzer/Core/SVals.cpp
  clang/unittests/StaticAnalyzer/SValTest.cpp


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt  = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), 
Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt  = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D129737: [analyzer] Fixing SVal::getType returns Null Type for NonLoc::ConcreteInt in boolean type

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: vsavchenko, NoQ, ASDenysPetrov.
OikawaKirie added a project: clang.
Herald added subscribers: steakhal, manas, martong, dkrupp, donat.nagy, 
Szelethus, mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware, xazax.hun.
Herald added a project: All.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

In method `TypeRetrievingVisitor::VisitConcreteInt`, 
`ASTContext::getIntTypeForBitwidth` is used to get the type for `ConcreteInt`s.
However, the getter in ASTContext cannot handle the boolean type with the bit 
width of 1, which will make method `SVal::getType` return a Null `Type`.
In this patch, a check for this case is added to fix this problem by returning 
the bool type directly when the bit width is 1.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D129737

Files:
  clang/lib/StaticAnalyzer/Core/SVals.cpp
  clang/unittests/StaticAnalyzer/SValTest.cpp


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt  = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), 
Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {


Index: clang/unittests/StaticAnalyzer/SValTest.cpp
===
--- clang/unittests/StaticAnalyzer/SValTest.cpp
+++ clang/unittests/StaticAnalyzer/SValTest.cpp
@@ -161,6 +161,7 @@
 void foo() {
   int x = 42;
   int *y = nullptr;
+  bool z = true;
 })") {
   SVal X = getByName("x");
   ASSERT_FALSE(X.getType(Context).isNull());
@@ -170,6 +171,10 @@
   ASSERT_FALSE(Y.getType(Context).isNull());
   expectSameSignAndBitWidth(Context.getUIntPtrType(), Y.getType(Context),
 Context);
+
+  SVal Z = getByName("z");
+  ASSERT_FALSE(Z.getType(Context).isNull());
+  EXPECT_EQ(Context.BoolTy, Z.getType(Context));
 }
 
 SVAL_TEST(GetLocAsIntType, R"(
Index: clang/lib/StaticAnalyzer/Core/SVals.cpp
===
--- clang/lib/StaticAnalyzer/Core/SVals.cpp
+++ clang/lib/StaticAnalyzer/Core/SVals.cpp
@@ -136,6 +136,8 @@
   }
   template  QualType VisitConcreteInt(ConcreteInt CI) {
 const llvm::APSInt  = CI.getValue();
+if (1 == Value.getBitWidth())
+  return Context.BoolTy;
 return Context.getIntTypeForBitwidth(Value.getBitWidth(), Value.isSigned());
   }
   QualType VisitLocConcreteInt(loc::ConcreteInt CI) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D92160#3485400 , @Kale wrote:

> In D92160#2449507 , @dexonsmith 
> wrote:
>
>> But it's possible we don't need this. If it's safe for us to update the 
>> tests and make `FileManager::getFileRef` always canonicalize to an absolute 
>> path, that would definitely be cleaner. `FileManager::makeAbsolute` can use 
>> whatever the FS's CWD is at the time of query... nice and clean.
>
> I've tried to fix this bug through this way but found that "making 
> `FileManager::getFileRef` always canonicalize to an absolute path" was pretty 
> hard, because there are many test cases using states stored in FileManager 
> and assuming that they are all in relative form. Besides, the assumption that 
> "the CWD won't change" indeed is correct by design and works fine for single 
> compiler execution. We might not change that unless for a strong necessity.
>
> So I personally believed that this bug is caused by libtooling's incorrect 
> use of `FileManger`. I plan to fix it by implementing a class like 
> `LibtoolingFileManager`, or `AbsoluteFileManager`, which extends the original 
> `FileManager` and using abs path as key to store the status of seen files, 
> but only used for libtooling.
>
> I will try to upload a patch later to verify the possibility.

Thanks a lot for taking over this bug. Please remember to close this patch if 
your new patch can solve this problem.
I will forward all the comments from the original author (Hao Zhang) if he has 
any suggestions for your new patch.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG9f90254286dc: [analyzer][ctu] Fix wrong multiple 
definitions errors caused by space… (authored by OikawaKirie).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,41 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '41:c:@S@G@F@G#@Sa@F@operator void (*)(int)#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '38:c:@S@G@F@G#@Sa@F@operator void (*)()#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '14:c:@F@importee# %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+
+// RUN: cd %t
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true \
+// RUN:   -verify %s 2>&1 | FileCheck %s
+
+// CHECK: CTU loaded AST file
+
+// FIXME: In this test case, we cannot use the on-demand-parsing approach to
+//load the external TU.
+//
+//In the Darwin system, the target triple is determined by the driver,
+//rather than using the default one like other systems. However, when
+//using bare `clang -cc1`, the adjustment is not done, which cannot
+//match the one loaded with on-demand-parsing (adjusted triple).
+//We bypass this problem by loading AST files, whose target triple is
+//also unadjusted when generated via `clang -cc1 -emit-pch`.
+//
+//Refer to: https://discourse.llvm.org/t/60762
+//
+//This is also the reason why the test case of D75665 (introducing
+//the on-demand-parsing feature) is enabled only on Linux.
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 416861.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.

- Add FIXME in test case.
- Add discourse topic link in summary.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,41 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '41:c:@S@G@F@G#@Sa@F@operator void (*)(int)#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '38:c:@S@G@F@G#@Sa@F@operator void (*)()#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '14:c:@F@importee# %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+
+// RUN: cd %t
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true \
+// RUN:   -verify %s 2>&1 | FileCheck %s
+
+// CHECK: CTU loaded AST file
+
+// FIXME: In this test case, we cannot use the on-demand-parsing approach to
+//load the external TU.
+//
+//In the Darwin system, the target triple is determined by the driver,
+//rather than using the default one like other systems. However, when
+//using bare `clang -cc1`, the adjustment is not done, which cannot
+//match the one loaded with on-demand-parsing (adjusted triple).
+//We bypass this problem by loading AST files, whose target triple is
+//also unadjusted when generated via `clang -cc1 -emit-pch`.
+//
+//Refer to: https://discourse.llvm.org/t/60762
+//
+//This is also the reason why the test case of D75665 (introducing
+//the on-demand-parsing feature) is enabled only on Linux.
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.
Herald added a project: All.

Thanks, @keith.

I agree with @keith to commit this patch without using on-demand-parsing 
through cc1.
As this patch has nothing to do with the target triple issue we found.

In the current version, I use the PCH file to load the external TU.
And it seems to work fine on my system and on the Windows CI.

IMO, maybe we can just leave a FIXME or something else in the test case and 
commit this patch to fix the original problem we want to fix.
(of course, re-submit to rerun the test case on Linux)
What do you think? @steakhal


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116924: [clang-extdef-mapping] Allow clang-extdef-mapping tool to output customized filename for each index entry

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116924/new/

https://reviews.llvm.org/D116924

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I think I have found out the reason for the problem, and it proved my guesses.

When executing the test case of the static analyzer, we usually use 
`%clang_analyze_cc1` as the entry, which is `%clang_cc1 -analyze`.  And we 
usually do not set a target triple, as it is not required by the analyzer. 
However, things are complicated in Darwin Unix.

In Darwin, the default target triple is `ARCH-apple-darwinXX.XX.XX`, where 
`ARCH` is the architecture (e.g. `x86_64`) and `XX.XX.XX` is the version of the 
Darwin system. And the default target triple will be then adjusted to 
`ARCH-apple-SYSTEMXX.XX.XX` by the 
`Driver::Darwin::ComputeEffectiveClangTriple` in driver related code, where 
`SYSTEM` can be `watchos`, `tvos`, `ios` and `macosx`.

In the clang driver, the adjusted target triple will be passed to the new cc1 
process; whereas in tooling and `ASTUnit::LoadFromCommandLine`, the adjusted 
target triple will be used to generate cc1 arguments to create 
`CompilerInvocation`. But when executing bare `clang -cc1`, if the target 
triple argument is not provided, it remains the default 
`ARCH-apple-darwinXX.XX.XX`. And this is the reason for the conflict.

The CTU on-demand-parsing mechanism uses `ASTUnit::LoadFromCommandLine` to load 
external ASTs. The tool `clang-check` uses clang tooling to parse the entry 
file. Therefore, both target triples are the adjusted ones, which can be 
matched. And so is the driver (`clang --analyze ...`). But not the bare 
`%clang_cc1`, its target triple is the default one.

---

Let's have a look at the simple example, suppose externalDefMap.txt and 
invocations.yaml are generated correctly.

input.cc:

  void bar();
  void foo() { bar(); }

importee.cc:

  void bar() { }



---

Using driver:

  clang -v --analyze input.cc -Xanalyzer -analyzer-config -Xanalyzer 
experimental-enable-naive-ctu-analysis=true,ctu-dir=.

Output: OK, adjusted to adjusted

  (in-process)
  /path/to/clang-15 -cc1 -triple x86_64-apple-macosx10.15.0 ...



---

Using clang-check:

  clang-check -analyze input.cc -- -v -Xanalyzer -analyzer-config -Xanalyzer 
experimental-enable-naive-ctu-analysis=true,ctu-dir=.

Output: OK, adjusted to adjusted

  clang Invocation:
   "/path/to/clang-tool" "-cc1" "-triple" "x86_64-apple-macosx10.15.0"



---

Using cc1:

  clang -cc1 -v -analyze input.cc  -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=.

Output: ERROR, default to adjusted

  warning: imported AST from 'importee.cc' had been generated for a different 
target, current: x86_64-apple-darwin19.6.0, imported: 
x86_64-apple-macosx10.15.0 [-Wctu]



---

What do you think is a better way to fix this problem? @gamesh411 @steakhal 
@martong 
Using `clang-check` to run the test case seems to be a good way to overcome the 
problem, but the problem still exists.
However, IMO it is not a good idea to make clang cc1 to adjust the default 
target triple manually.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

It seems that it is not committed on my behalf. Maybe you have forgotten to add 
me as the author?
Sad : (


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116329/new/

https://reviews.llvm.org/D116329

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Could you please commit it on my behalf (Ella Ma )?
Thanks a lot.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116329/new/

https://reviews.llvm.org/D116329

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 399841.
OikawaKirie added a comment.

Updated as suggested.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116329/new/

https://reviews.llvm.org/D116329

Files:
  clang/test/Tooling/clang-check-analyze-save-temps.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -208,27 +208,37 @@
   ClangTool Tool(OptionsParser.getCompilations(),
  OptionsParser.getSourcePathList());
 
-  // Clear adjusters because -fsyntax-only is inserted by the default chain.
-  Tool.clearArgumentsAdjusters();
-
-  // Reset output path if is provided by user.
-  Tool.appendArgumentsAdjuster(
-  Analyze ? [&](const CommandLineArguments , StringRef File) {
-  auto Ret = getClangStripOutputAdjuster()(Args, File);
-  if (!AnalyzerOutput.empty()) {
-Ret.emplace_back("-o");
-Ret.emplace_back(AnalyzerOutput);
-  }
-  return Ret;
-}
-  : getClangStripOutputAdjuster());
-
-  Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
-
-  // Running the analyzer requires --analyze. Other modes can work with the
-  // -fsyntax-only option.
-  Tool.appendArgumentsAdjuster(getInsertArgumentAdjuster(
-  Analyze ? "--analyze" : "-fsyntax-only", ArgumentInsertPosition::BEGIN));
+  if (Analyze) {
+// Set output path if is provided by user.
+//
+// As the original -o options have been removed by default via the
+// strip-output adjuster, we only need to add the analyzer -o options here
+// when it is provided by users.
+if (!AnalyzerOutput.empty())
+  Tool.appendArgumentsAdjuster(
+  getInsertArgumentAdjuster(CommandLineArguments{"-o", AnalyzerOutput},
+ArgumentInsertPosition::END));
+
+// Running the analyzer requires --analyze. Other modes can work with the
+// -fsyntax-only option.
+//
+// The syntax-only adjuster is installed by default.
+// Good: It also strips options that trigger extra output, like 
-save-temps.
+// Bad:  We don't want the -fsyntax-only when executing the static 
analyzer.
+//
+// To enable the static analyzer, we first strip all -fsyntax-only options
+// and then add an --analyze option to the front.
+Tool.appendArgumentsAdjuster(
+[&](const CommandLineArguments , StringRef /*unused*/) {
+  CommandLineArguments AdjustedArgs;
+  for (const std::string  : Args)
+if (Arg != "-fsyntax-only")
+  AdjustedArgs.emplace_back(Arg);
+  return AdjustedArgs;
+});
+Tool.appendArgumentsAdjuster(
+getInsertArgumentAdjuster("--analyze", ArgumentInsertPosition::BEGIN));
+  }
 
   ClangCheckActionFactory CheckFactory;
   std::unique_ptr FrontendFactory;
Index: clang/test/Tooling/clang-check-analyze-save-temps.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-analyze-save-temps.cpp
@@ -0,0 +1,19 @@
+// Check whether output generation options (like -save-temps) will not affect
+// the execution of the analyzer.
+
+// RUN: clang-check -analyze %s -- -save-temps -c -Xclang -verify
+
+// Check whether redundant -fsyntax-only options will affect the execution of
+// the analyzer.
+
+// RUN: clang-check -analyze %s -- \
+// RUN:   -fsyntax-only -c -fsyntax-only -Xclang -verify 2>&1 | \
+// RUN:   FileCheck %s --allow-empty
+
+// CHECK-NOT: argument unused during compilation: '--analyze'
+
+void a(int *x) {
+  if (x) {
+  }
+  *x = 47; // expected-warning {{Dereference of null pointer}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -208,27 +208,37 @@
   ClangTool Tool(OptionsParser.getCompilations(),
  OptionsParser.getSourcePathList());
 
-  // Clear adjusters because -fsyntax-only is inserted by the default chain.
-  Tool.clearArgumentsAdjusters();
-
-  // Reset output path if is provided by user.
-  Tool.appendArgumentsAdjuster(
-  Analyze ? [&](const CommandLineArguments , StringRef File) {
-  auto Ret = getClangStripOutputAdjuster()(Args, File);
-  if (!AnalyzerOutput.empty()) {
-Ret.emplace_back("-o");
-Ret.emplace_back(AnalyzerOutput);
-  }
-  return Ret;
-}
-  : getClangStripOutputAdjuster());
-
-  Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
-
-  // Running the analyzer requires --analyze. Other modes can work with the
-  // -fsyntax-only option.
-  

[PATCH] D116924: [clang-extdef-mapping] Allow clang-extdef-mapping tool to output customized filename for each index entry

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: martong, NoQ, steakhal.
OikawaKirie added a project: clang.
Herald added subscribers: arphaman, rnkovacs.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

The clang-extdef-mapping tool can only output the real file name of the input 
source file. When analyzing with AST file based CTU analysis, the file name 
should be adjusted to the corresponding AST file, where `.ast` will be appended 
to the file name, and sometimes the path will even be changed. It is very 
inconvenient to adjust the file path with such a separated step, especially on 
Windows, where utility tools such as `sed` are not available.

This patch uses `Regex::sub` function to adjust the output file name for each 
index entry with the pattern provided by the user.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D116924

Files:
  clang/test/Analysis/func-mapping-path.c
  clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp

Index: clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp
===
--- clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp
+++ clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp
@@ -30,11 +30,45 @@
 using namespace clang::tooling;
 
 static cl::OptionCategory ClangExtDefMapGenCategory("clang-extdefmapgen options");
+static cl::opt OutputPathAdjusterMatcher(
+"m", cl::desc("Match the part of output path to be replaced with regex"),
+cl::cat(ClangExtDefMapGenCategory));
+static cl::opt OutputPathAdjusterNewPath(
+"r", cl::desc("The string which matched pattern will be replaced to"),
+cl::cat(ClangExtDefMapGenCategory));
+
+namespace {
+class OutputPathAdjuster {
+  Regex Matcher;
+  std::string NewPath;
+
+public:
+  std::string adjust(StringRef Path) const {
+return Matcher.sub(NewPath, Path);
+  }
+
+  bool resetMatcher(StringRef MatcherStr, StringRef NewPathStr);
+};
+
+bool OutputPathAdjuster::resetMatcher(StringRef MatcherStr,
+  StringRef NewPathStr) {
+  std::string ErrMsg;
+  Regex MatcherRegex(MatcherStr);
+  if (!MatcherRegex.isValid(ErrMsg)) {
+errs() << "Invalid adjuster template: " << ErrMsg << '\n';
+return false;
+  }
+
+  Matcher = std::move(MatcherRegex);
+  NewPath = NewPathStr.str();
+  return true;
+}
+} // namespace
 
 class MapExtDefNamesConsumer : public ASTConsumer {
 public:
-  MapExtDefNamesConsumer(ASTContext )
-  : Ctx(Context), SM(Context.getSourceManager()) {}
+  MapExtDefNamesConsumer(const OutputPathAdjuster *Adjuster)
+  : Adjuster(Adjuster) {}
 
   ~MapExtDefNamesConsumer() {
 // Flush results to standard output.
@@ -49,10 +83,9 @@
   void handleDecl(const Decl *D);
   void addIfInMain(const DeclaratorDecl *DD, SourceLocation defStart);
 
-  ASTContext 
-  SourceManager 
   llvm::StringMap Index;
   std::string CurrentFileName;
+  const OutputPathAdjuster *Adjuster = nullptr;
 };
 
 void MapExtDefNamesConsumer::handleDecl(const Decl *D) {
@@ -64,7 +97,7 @@
   if (const Stmt *Body = FD->getBody())
 addIfInMain(FD, Body->getBeginLoc());
   } else if (const auto *VD = dyn_cast(D)) {
-if (cross_tu::containsConst(VD, Ctx) && VD->hasInit())
+if (cross_tu::containsConst(VD, VD->getASTContext()) && VD->hasInit())
   if (const Expr *Init = VD->getInit())
 addIfInMain(VD, Init->getBeginLoc());
   }
@@ -82,9 +115,12 @@
 return;
   assert(!LookupName->empty() && "Lookup name should be non-empty.");
 
+  SourceManager  = DD->getASTContext().getSourceManager();
   if (CurrentFileName.empty()) {
 CurrentFileName = std::string(
 SM.getFileEntryForID(SM.getMainFileID())->tryGetRealPathName());
+if (Adjuster)
+  CurrentFileName = Adjuster->adjust(CurrentFileName);
 if (CurrentFileName.empty())
   CurrentFileName = "invalid_file";
   }
@@ -101,11 +137,15 @@
   }
 }
 
-class MapExtDefNamesAction : public ASTFrontendAction {
-protected:
-  std::unique_ptr CreateASTConsumer(CompilerInstance ,
- llvm::StringRef) override {
-return std::make_unique(CI.getASTContext());
+class MapExtDefNamesConsumerFactory {
+  const OutputPathAdjuster *Adjuster = nullptr;
+
+public:
+  MapExtDefNamesConsumerFactory(const OutputPathAdjuster *Adjuster)
+  : Adjuster(Adjuster) {}
+
+  std::unique_ptr newASTConsumer() {
+return std::make_unique(Adjuster);
   }
 };
 
@@ -118,7 +158,8 @@
 
   const char *Overview = "\nThis tool collects the USR name and location "
  "of external definitions in the source files "
- "(excluding headers).\n";
+ "(excluding headers).\n\nReset output path with -m "
+ " -r .\n";
   auto ExpectedParser = CommonOptionsParser::create(
   argc, argv, ClangExtDefMapGenCategory, cl::ZeroOrMore, Overview);
   if (!ExpectedParser) {

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 397767.
OikawaKirie added a comment.

To make it work on Windows, Linux, and Mac OS, using `echo` to create the 
external function map, and using AST file for CTU analysis.
Tested on Windows, Linux, and Mac OS under x64.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,26 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '41:c:@S@G@F@G#@Sa@F@operator void (*)(int)#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '38:c:@S@G@F@G#@Sa@F@operator void (*)()#1 %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: echo '14:c:@F@importee# %/t/importee.ast' >> %t/externalDefMap.txt
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+
+// RUN: cd %t
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true \
+// RUN:   -verify %s 2>&1 | FileCheck %s
+
+// CHECK: CTU loaded AST file
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp
-// RUN: echo "c:@N@clang@S@DeclContextLookupResult@SingleElementDummyList ctu-inherited-default-ctor-other.cpp.ast" \
+// RUN: echo 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 397289.
OikawaKirie added a comment.

Replace on-demand-parsing with loading AST file for the new test case.
Tested on Linux and MacOS(x86).
If it can also pass the CI test on Windows, I think we can have another try on 
landing this patch.

Besides, as mentioned above, to trigger the problem of target triple on MacOS, 
we can simply remove the requirement of Linux system for the two test cases of 
on-demand-parsing, i.e. `clang/test/Analysis/ctu-on-demand-parsing.c` and 
`clang/test/Analysis/ctu-on-demand-parsing.cpp`.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,22 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: %clang_extdef_map %S/Inputs/ctu-lookup-name-with-space.cpp -- | \
+// RUN:   sed 's:%/S/Inputs/ctu-lookup-name-with-space.cpp:%/t/importee.ast:g' > %t/externalDefMap.txt
+
+// RUN: cd %t
+// RUN: %clang_cc1 -emit-pch %/S/Inputs/ctu-lookup-name-with-space.cpp -o %t/importee.ast
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -verify %s
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp
-// RUN: echo "c:@N@clang@S@DeclContextLookupResult@SingleElementDummyList 

[PATCH] D115716: [Analyzer][BugReporter] Replace the example bug report with the one used to generate PathDiagnostic

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 397207.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.

In D115716#3217786 , @Szelethus wrote:

> First off, your patch is great, and I'm pretty sure we want it!
>
> I remember working around here, and I either have never quite understood what 
> makes `exampleReport` an example, or have long forgotten. Can we not just 
> rename it to `chosenReport`, or simply `report`?

Thanks a lot.

New updates:

1. add a new `BugReporter::generateDiagnosticForConsumerMap` method to handle 
the job before adding notes
2. add a `BugReport` field to `DiagnosticForConsumerMapTy`
3. move `findReportInEquivalenceClass` calls to 
`generateDiagnosticForConsumerMap` methods
4. the `exampleReport` variable in method `findReportInEquivalenceClass` is 
unchanged, others are removed as they are no longer required


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D115716/new/

https://reviews.llvm.org/D115716

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporter.h
  clang/lib/StaticAnalyzer/Core/BugReporter.cpp
  clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
  clang/test/Analysis/keychainAPI.m
  clang/test/Analysis/malloc-plist.c
  clang/test/Analysis/stream.c

Index: clang/test/Analysis/stream.c
===
--- clang/test/Analysis/stream.c
+++ clang/test/Analysis/stream.c
@@ -261,9 +261,7 @@
   if (!F1)
 return;
   if (Test == 1) {
-return; // no warning
+return; // expected-warning {{Opened stream never closed. Potential resource leak}}
   }
   rewind(F1);
-} // expected-warning {{Opened stream never closed. Potential resource leak}}
-// FIXME: This warning should be placed at the `return` above.
-// See https://reviews.llvm.org/D83120 about details.
+} // no warning
Index: clang/test/Analysis/malloc-plist.c
===
--- clang/test/Analysis/malloc-plist.c
+++ clang/test/Analysis/malloc-plist.c
@@ -135,8 +135,8 @@
 static void function_with_leak3(int y) {
 char *x = (char*)malloc(12);
 if (y)
-y++;
-}//expected-warning{{Potential leak}}
+  y++; // expected-warning{{Potential leak}}
+}
 void use_function_with_leak3(int y) {
 function_with_leak3(y);
 }
Index: clang/test/Analysis/keychainAPI.m
===
--- clang/test/Analysis/keychainAPI.m
+++ clang/test/Analysis/keychainAPI.m
@@ -402,10 +402,10 @@
 OSStatus st = 0;
 void *outData = my_AllocateReturn(); 
 if (x) {
-  consumeChar(*(char*)outData); // expected-warning{{Allocated data is not released:}}
+  consumeChar(*(char *)outData);
   return;
 } else {
-  consumeChar(*(char*)outData);
+  consumeChar(*(char *)outData); // expected-warning{{Allocated data is not released:}}
 }
 return;
 }
Index: clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
===
--- clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
+++ clang/test/Analysis/Inputs/expected-plists/malloc-plist.c.plist
@@ -3764,12 +3764,12 @@
  
   
line138
-   col9
+   col7
file0
   
   
line138
-   col9
+   col7
file0
   
  
@@ -3781,7 +3781,7 @@
  location
  
   line138
-  col9
+  col7
   file0
  
  depth1
@@ -3803,7 +3803,7 @@
   location
   
line138
-   col9
+   col7
file0
   
   ExecutedLines
Index: clang/lib/StaticAnalyzer/Core/BugReporter.cpp
===
--- clang/lib/StaticAnalyzer/Core/BugReporter.cpp
+++ clang/lib/StaticAnalyzer/Core/BugReporter.cpp
@@ -244,6 +244,9 @@
   std::unique_ptr
   generate(const PathDiagnosticConsumer *PDC) const;
 
+  /// Get the bug report used to generate the PathDiagnostic.
+  const PathSensitiveBugReport *getBugReport() const { return R; }
+
 private:
   void updateStackPiecesWithMessage(PathDiagnosticPieceRef P,
 const CallWithEntryStack ) const;
@@ -271,8 +274,6 @@
   PathDiagnosticLocation
   ExecutionContinues(llvm::raw_string_ostream ,
  const PathDiagnosticConstruct ) const;
-
-  const PathSensitiveBugReport *getBugReport() const { return R; }
 };
 
 } // namespace
@@ -2889,6 +2890,7 @@
 (*Out)[PC] = std::move(PD);
   }
 }
+Out->Report = PDB->getBugReport();
   }
 
   return Out;
@@ -3060,24 +3062,28 @@
   return exampleReport;
 }
 
-void BugReporter::FlushReport(BugReportEquivClass& EQ) {
-  SmallVector bugReports;
-  BugReport *report = findReportInEquivalenceClass(EQ, bugReports);
-  if (!report)
-return;
-
+std::unique_ptr
+BugReporter::generateDiagnosticForConsumerMap(BugReportEquivClass ) 

[PATCH] D116329: [clang-check] Adjust argument adjusters for clang-check to strip options blocking the static analyzer

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: kbobyrev, sammccall, labath.
OikawaKirie added a project: clang.
Herald added subscribers: manas, ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
a.sidorin, baloghadamsoftware.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Output generation options (like `-save-temps`) will make the analyzer not 
executed even `--analyze` option is provided in the driver arguments.
Besides, the original approach of adding `--analyze` option will not work when 
(more than one) `-fsyntax-only` options are provided in the driver arguments.

This patch fixes these two problems by using the syntax-only adjuster to remove 
output generation options and manually filter out redundant `-fsyntax-only` 
options.

In the new implementation, the adjusters added by `ClangTool` will not be 
removed but used as dependencies for clang-check adjusters for analyzer options.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D116329

Files:
  clang/test/Tooling/clang-check-analyze-save-temps.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -208,27 +208,39 @@
   ClangTool Tool(OptionsParser.getCompilations(),
  OptionsParser.getSourcePathList());
 
-  // Clear adjusters because -fsyntax-only is inserted by the default chain.
-  Tool.clearArgumentsAdjusters();
-
-  // Reset output path if is provided by user.
-  Tool.appendArgumentsAdjuster(
-  Analyze ? [&](const CommandLineArguments , StringRef File) {
-  auto Ret = getClangStripOutputAdjuster()(Args, File);
-  if (!AnalyzerOutput.empty()) {
-Ret.emplace_back("-o");
-Ret.emplace_back(AnalyzerOutput);
-  }
-  return Ret;
-}
-  : getClangStripOutputAdjuster());
-
-  Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
-
-  // Running the analyzer requires --analyze. Other modes can work with the
-  // -fsyntax-only option.
-  Tool.appendArgumentsAdjuster(getInsertArgumentAdjuster(
-  Analyze ? "--analyze" : "-fsyntax-only", ArgumentInsertPosition::BEGIN));
+  if (Analyze) {
+// Set output path if is provided by user.
+//
+// As the original -o options have been removed by strip-output adjuster, 
we
+// only need to add the analyzer -o options here.
+if (!AnalyzerOutput.empty()) {
+  Tool.appendArgumentsAdjuster(getInsertArgumentAdjuster("-o"));
+  Tool.appendArgumentsAdjuster(
+  getInsertArgumentAdjuster(AnalyzerOutput.c_str()));
+}
+
+// Running the analyzer requires --analyze. Other modes can work with the
+// -fsyntax-only option.
+//
+// The syntax-only adjuster can also help us to remove other options that
+// trigger output generation, e.g. -save-temps. Besides, to enable the
+// analyzer, we also need to replace the first -fsyntax-only option with
+// option --analyze, and remove redundant ones.
+Tool.appendArgumentsAdjuster(
+[&](const CommandLineArguments , StringRef /*unused*/) {
+  CommandLineArguments AdjustedArgs;
+  bool HasAnalyze = false;
+  for (const StringRef Arg : Args) {
+if (Arg != "-fsyntax-only") {
+  AdjustedArgs.emplace_back(Arg);
+} else if (!HasAnalyze) {
+  AdjustedArgs.emplace_back("--analyze");
+  HasAnalyze = true;
+}
+  }
+  return AdjustedArgs;
+});
+  }
 
   ClangCheckActionFactory CheckFactory;
   std::unique_ptr FrontendFactory;
Index: clang/test/Tooling/clang-check-analyze-save-temps.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-analyze-save-temps.cpp
@@ -0,0 +1,19 @@
+// Check whether output generation options (like -save-temps) will not affect
+// the execution of the analyzer.
+
+// RUN: clang-check -analyze %s -- -save-temps -c -Xclang -verify
+
+
+// Check whether redundant -fsyntax-only options will affect the execution of
+// the analyzer.
+
+// RUN: clang-check -analyze %s -- \
+// RUN:   -fsyntax-only -c -fsyntax-only -Xclang -verify 2>&1 | \
+// RUN:   FileCheck %s --allow-empty
+
+// CHECK-NOT: argument unused during compilation: '--analyze'
+
+void a(int *x) {
+  if (x) { }
+  *x = 47; // expected-warning {{Dereference of null pointer}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -208,27 +208,39 @@
   ClangTool Tool(OptionsParser.getCompilations(),
  OptionsParser.getSourcePathList());
 
-  // Clear adjusters 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I have confirmed that this problem is not due to this patch.
Besides, on Mac, both m1 and intel, the on-demand-parsing as well as loading an 
AST file generated via driver argument `-emit-ast` will also trigger this 
problem.
However, when loading an AST file generated via cc1 argument `-emit-pch`, the 
problem is not triggered.

---

See the example below, which is executed on an intel Mac laptop with clang 
13.0.0.

/tmp/test/test.c:

  void f();
  void g() { f(); }

/tmp/test/importee.c:

  void f() { }

/tmp/test/odp/externalDefMap.txt:

  c:@F@f /tmp/test/importee.c

/tmp/test/odp/invocations.yaml:

  "/tmp/test/importee.c": ["gcc", "-c", "/tmp/test/importee.c"]

/tmp/test/ast/externalDefMap.txt:

  c:@F@f /tmp/test/ast/importee.c.ast

When executing the analyzer with CTU analysis via on-demand-parsing:

  /tmp/test$ clang -cc1 -analyze -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=odp,ctu-invocation-list=invocations.yaml
 test.c

Or loading AST file generated via driver argument `-emit-ast`:

  /tmp/test$ clang -emit-ast importee.c -o ast/importee.c.ast
  /tmp/test$ clang -cc1 -analyze -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=ast test.c

The same diagnostic message is generated, though the triples are different from 
the ones for m1.

  warning: imported AST from '/tmp/test/importee.c' had been generated for a 
different target, current: x86_64-apple-darwin21.2.0, imported: 
x86_64-apple-macosx12.0.0 [-Wctu]



---

However, the problem will not be triggered if triple is given:
(On demand parsing: setting the triple of the entry file to the one of imported 
ASTUnit)

  /tmp/test$ clang -cc1 -analyze -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=odp,ctu-invocation-list=invocations.yaml
 test.c -triple x86_64-apple-macosx12.0.0

(AST)

  /tmp/test$ clang -target arm-apple-macosx -emit-ast importee.c -o 
ast/importee.c.ast
  /tmp/test$ clang -cc1 -analyze -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=ast test.c -triple 
arm-apple-macosx

Or the AST file is generated via cc1 argument `-emit-pch`:

  /tmp/test$ clang -cc1 -emit-pch importee.c -o ast/importee.c.ast
  /tmp/test$ clang -cc1 -analyze -analyzer-checker=core -analyzer-config 
experimental-enable-naive-ctu-analysis=true,ctu-dir=ast test.c

I think we can bypass the problem temporarily by loading the AST file generated 
by cc1 argument `-emit-pch`, just as shown in the last code snippet above.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102669#3206089 , @steakhal wrote:

> Prior to this patch, it worked on M1 ; after 
> landing it broke something, so we clearly shouldn't land this.

I do not think it is this patch that breaks the functionality on M1 
, as it depends on the *on-demand-parsing* feature 
that is not tested on M1  currently.

> We should add a test-case demonstrating the problem with M1 
>  with a given configuration.

If I got it correct (it is on-demand-parsing that triggers the problem), this 
problem can be triggered by enabling the test case of D75665 
 on M1 .

> Then we need to track down and fix the underlying issue causing it. That 
> should be done probably in a separate patch and add it as a parent patch to 
> this one.
>
> If all of these are done, we can probably land both of them.

Maybe currently a simpler way is trying to use AST dump to load the external TU 
to be imported, rather than on-demand-parsing, which can make us fix this 
failure with the test case still enabled on M1 .

I will have a series of tests on my concerns later, and I will reply with my 
results if I can find something.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102669#3199270 , @steakhal wrote:

> We shouldn't skip mac targets. I CC ASTImporter folks, they probably have an 
> M1 .

I am not intended to ignore this problem triggered on M1 
. However, I think it is not this patch that leads 
to this problem, it just triggers it.
I mean we can just disable the test case temporarily on M1 
, and fix this problem as well as enable this 
patch and the one of on-demand-parsing in another patch.
I think they trigger the same problem for the same reason on M1 
.

Besides, it seems to be the problem of `ASTUnit::LoadFromCommandLine`, rather 
than the ASTImporter.




Comment at: clang/test/Analysis/ctu-lookup-name-with-space.cpp:13
+// RUN:   -verify %s
+
+void importee();

arichardson wrote:
> OikawaKirie wrote:
> > Adding this line here.
> Disabling the test on non- Linux is not a good idea IMO since it means we 
> lose coverage on other platforms. My guess is that you just need to specify 
> an explicit triple in the clang invocations.
AFAIK, we cannot do that. If this test case is executed on different platforms, 
we cannot determine the triple ahead of time and specify it in the invocation 
list.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

It seems that it is not this patch that triggers the problem, which is similar 
to D75665 .
IMO it is the problem of on-demand-parsing, but I do not have a Mac M1 
 device to reproduce this bug.
Maybe we can just land this patch by restricting the test case to be executed 
only on Linux, just as what D75665  does 
(rG5cc18516c483 
 vs 
rG97e07d0c352c 
), and 
leave the problem for future fixes.

Could you please do the update as provided below and land this patch again? 
@steakhal or other reviewers?




Comment at: clang/test/Analysis/ctu-lookup-name-with-space.cpp:13
+// RUN:   -verify %s
+
+void importee();

Adding this line here.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Could you please revert this on my behalf? I currently have no idea to fix this 
problem.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I do not know how this error happens. Maybe we can currently revert this patch 
an have another try in the future.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

It seems this patch has nothing to do with the failure in the Linux build. I 
think it is now ready to land.
Thanks a lot for your suggestions during the revision.

Could you please commit this patch on my behalf? Thanks.
Ella Ma 


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 394748.
OikawaKirie added a comment.

Fix `YAML:1:293: error: Unrecognized escape code` error by replacing lit 
substitution pattern `%S` to `%/S`.
Fix `cp` problems by removing the file copy operations.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,21 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: echo '"%/S/Inputs/ctu-lookup-name-with-space.cpp" : ["g++", "-c", "%/S/Inputs/ctu-lookup-name-with-space.cpp"]' > %t/invocations.yaml
+// RUN: %clang_extdef_map %S/Inputs/ctu-lookup-name-with-space.cpp -- > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify %s
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp
-// RUN: echo "c:@N@clang@S@DeclContextLookupResult@SingleElementDummyList ctu-inherited-default-ctor-other.cpp.ast" \
+// RUN: echo "59:c:@N@clang@S@DeclContextLookupResult@SingleElementDummyList ctu-inherited-default-ctor-other.cpp.ast" \
 // RUN:   > %t/ctudir/externalDefMap.txt
 //
 // RUN: %clang_analyze_cc1 -std=c++14 -triple 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

When running my test case `ctu-lookup-name-with-space.cpp` on Windows, 
`llvm-lit` reports `'cp': command not found`. And this is the reason why it 
fails on Windows.
And when I remove the `cp`s and replace them with original file names, clang 
reports `YAML:1:293: error: Unrecognized escape code`, it seems that the static 
analyzer only reads compilation database in YAML format on Windows.
Should I disable this test case on Windows? Or is there any other approaches to 
make it work on Windows?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D115716: [Analyzer][BugReporter] Replace the example bug report with the one used to generate PathDiagnostic

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: NoQ, steakhal, vsavchenko, ASDenysPetrov.
OikawaKirie added a project: clang.
Herald added subscribers: manas, martong, dkrupp, donat.nagy, Szelethus, 
mikhail.ramalho, a.sidorin, szepet, baloghadamsoftware, xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

In function `BugReporter::FlushReport`, function 
`PathSensitiveBugReporter::generatePathDiagnostics` will re-select one report 
from the equivalence class to construct the `PathDiagnostic`. It makes the 
report used to generate `PathDiagnostic` may be different from the one used in 
function `BugReporter::FlushReport`. Changes to the bug report via a 
`BugReporterVisitor` may not be reflected when generating report notes or 
fix-hints.

This patch tries to fix this problem by resetting the pointer to the example 
report selected in function `BugReporter::FlushReport` with the one used to 
generate `PathDiagnostic`.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D115716

Files:
  clang/include/clang/StaticAnalyzer/Core/BugReporter/BugReporter.h
  clang/lib/StaticAnalyzer/Core/BugReporter.cpp
  clang/test/Analysis/keychainAPI.m
  clang/test/Analysis/malloc-plist.c
  clang/test/Analysis/stream.c

Index: clang/test/Analysis/stream.c
===
--- clang/test/Analysis/stream.c
+++ clang/test/Analysis/stream.c
@@ -261,9 +261,7 @@
   if (!F1)
 return;
   if (Test == 1) {
-return; // no warning
+return; // expected-warning {{Opened stream never closed. Potential resource leak}}
   }
   rewind(F1);
-} // expected-warning {{Opened stream never closed. Potential resource leak}}
-// FIXME: This warning should be placed at the `return` above.
-// See https://reviews.llvm.org/D83120 about details.
+} // no warning
Index: clang/test/Analysis/malloc-plist.c
===
--- clang/test/Analysis/malloc-plist.c
+++ clang/test/Analysis/malloc-plist.c
@@ -135,8 +135,8 @@
 static void function_with_leak3(int y) {
 char *x = (char*)malloc(12);
 if (y)
-y++;
-}//expected-warning{{Potential leak}}
+y++;//expected-warning{{Potential leak}}
+}
 void use_function_with_leak3(int y) {
 function_with_leak3(y);
 }
Index: clang/test/Analysis/keychainAPI.m
===
--- clang/test/Analysis/keychainAPI.m
+++ clang/test/Analysis/keychainAPI.m
@@ -402,10 +402,10 @@
 OSStatus st = 0;
 void *outData = my_AllocateReturn(); 
 if (x) {
-  consumeChar(*(char*)outData); // expected-warning{{Allocated data is not released:}}
+  consumeChar(*(char*)outData);
   return;
 } else {
-  consumeChar(*(char*)outData);
+  consumeChar(*(char*)outData); // expected-warning{{Allocated data is not released:}}
 }
 return;
 }
Index: clang/lib/StaticAnalyzer/Core/BugReporter.cpp
===
--- clang/lib/StaticAnalyzer/Core/BugReporter.cpp
+++ clang/lib/StaticAnalyzer/Core/BugReporter.cpp
@@ -244,6 +244,9 @@
   std::unique_ptr
   generate(const PathDiagnosticConsumer *PDC) const;
 
+  /// Get the bug report used to generate the PathDiagnostic.
+  const PathSensitiveBugReport *getBugReport() const { return R; }
+
 private:
   void updateStackPiecesWithMessage(PathDiagnosticPieceRef P,
 const CallWithEntryStack ) const;
@@ -271,8 +274,6 @@
   PathDiagnosticLocation
   ExecutionContinues(llvm::raw_string_ostream ,
  const PathDiagnosticConstruct ) const;
-
-  const PathSensitiveBugReport *getBugReport() const { return R; }
 };
 
 } // namespace
@@ -2874,6 +2875,7 @@
 
 std::unique_ptr
 PathSensitiveBugReporter::generatePathDiagnostics(
+const BugReport *,
 ArrayRef consumers,
 ArrayRef ) {
   assert(!bugReports.empty());
@@ -2889,6 +2891,7 @@
 (*Out)[PC] = std::move(PD);
   }
 }
+exampleReport = PDB->getBugReport();
   }
 
   return Out;
@@ -3062,7 +3065,7 @@
 
 void BugReporter::FlushReport(BugReportEquivClass& EQ) {
   SmallVector bugReports;
-  BugReport *report = findReportInEquivalenceClass(EQ, bugReports);
+  const BugReport *report = findReportInEquivalenceClass(EQ, bugReports);
   if (!report)
 return;
 
@@ -3200,7 +3203,8 @@
 
 std::unique_ptr
 BugReporter::generateDiagnosticForConsumerMap(
-BugReport *exampleReport, ArrayRef consumers,
+const BugReport *,
+ArrayRef consumers,
 ArrayRef bugReports) {
   auto *basicReport = cast(exampleReport);
   auto Out = std::make_unique();
@@ -3272,11 +3276,10 @@
   }
 }
 
-
-
 std::unique_ptr
 PathSensitiveBugReporter::generateDiagnosticForConsumerMap(
-BugReport *exampleReport, ArrayRef consumers,
+const BugReport *,
+ArrayRef consumers,
 ArrayRef bugReports) {
   std::vector BasicBugReports;
   std::vector 

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 393064.
OikawaKirie added a comment.

1. Revert function `CrossTranslationUnitContext::getLookupName`
2. Add length when dumping the CTU index mapping via function 
`createCrossTUIndexString`
3. Remove the assertions during CTU map query process
4. Make function `parseCrossTUIndexItem` more readable


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,25 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp
+
+void importee();
+
+void trigger() {
+  // Call an external function to trigger the parsing process of CTU index.
+  // Refer to file Inputs/ctu-lookup-name-with-space.cpp for more details.
+
+  importee(); // expected-no-diagnostics
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ clang/test/Analysis/ctu-inherited-default-ctor.cpp
@@ -4,7 +4,7 @@
 // RUN: %clang_cc1 -std=c++14 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-inherited-default-ctor-other.cpp.ast \
 // RUN:%S/Inputs/ctu-inherited-default-ctor-other.cpp

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > OikawaKirie wrote:
> > > > steakhal wrote:
> > > > > OikawaKirie wrote:
> > > > > > steakhal wrote:
> > > > > > > 
> > > > > > The source of lookup name of the function being imported is 
> > > > > > function `CrossTranslationUnitContext::getLookupName`. Keeping the 
> > > > > > length in the mapping can avoid parsing the lookup name during 
> > > > > > importing.
> > > > > Okay; you can copy the original StringRef to have that. But by 
> > > > > consuming it on one path makes the code much more readable.
> > > > > 
> > > > The `getAsInterger` call can also check whether the content before the 
> > > > first colon is an integer. Therefore, a sub-string operation is 
> > > > required here.
> > > I don't doubt that your proposed way of doing this works and is efficient.
> > > What I say is that I think there is room for improvement in the 
> > > non-functional aspects, in the readability. However, it's not really a 
> > > blocking issue, more of a personal taste.
> > I know what you are considering, it is clearer and more readable by 
> > consuming the length, then the USR. However, to correctly separate the USR 
> > and file path, the length of `USR-Length` is also required, which makes it 
> > impossible to just *consume* the length at the beginning.
> > 
> > Another way of solving this problem is to re-create the string with the 
> > USR-Length and the USR after parsing, but I think it is not a good solution.
> > 
> > BTW, is it necessary to assert the `USR-Length` to be greater than zero? I 
> > think it is more reasonable to report *invalid format* rather than assert 
> > the value, as it can be provided by the user.
> I think what causes the misunderstanding is the definition of //consume// in 
> the context of `StringRef`.
> ```lang=C++
> const StringRef Copy = Line;
> Line.consumeInteger(...); // Line advances forward by the number of 
> characters that were parsed as an integral value.
> // Copy still refers to the unmodified, original characters.
> // I can use it however I want.
> 
> // `Line` is a suffix of `Copy`, and the `.end()` should be the same, only 
> `.begin()` should differ.
> ```
> 
> I hope that caused the miscommunication.
> 
> ---
> > BTW, is it necessary to assert the USR-Length to be greater than zero? I 
> > think it is more reasonable to report *invalid format* rather than assert 
> > the value, as it can be provided by the user.
> Yeah, sure!
I think I have figured out what have been misunderstood.

In the current patch, I just modify function 
`CrossTranslationUnitContext::getLookupName` by adding a length at the 
beginning. Therefore, the lookup name for the CTU query will have the length 
part. And for the sake of simplicity and efficiency, the length together with 
the USR is stored in the mapping as the key.

To correctly parse the `:` part, I cannot just consume the 
`` at the beginning. Otherwise, I cannot know the length of 
`:` part, which makes it impossible to parse the entire 
`:` part, even though the original `Line` is copied.

I will update the approach of adding the length part. Since the length is only 
used during parsing the CTU index, I will modify function 
`createCrossTUIndexString` to add the length and revert the changes to function 
`CrossTranslationUnitContext::getLookupName` to keep the lookup name for CTU 
query unchanged.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:12-14
+int importee(int X) {
+  return 1 / X;
+}

steakhal wrote:
> Also fixup the return type in the declaration within the main TU. Also add 
> the `// expected-no-diagnostics` comment to the primary TU.
Yes, you are right.
I was misled by myself.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > OikawaKirie wrote:
> > > > steakhal wrote:
> > > > > 
> > > > The source of lookup name of the function being imported is function 
> > > > `CrossTranslationUnitContext::getLookupName`. Keeping the length in the 
> > > > mapping can avoid parsing the lookup name during importing.
> > > Okay; you can copy the original StringRef to have that. But by consuming 
> > > it on one path makes the code much more readable.
> > > 
> > The `getAsInterger` call can also check whether the content before the 
> > first colon is an integer. Therefore, a sub-string operation is required 
> > here.
> I don't doubt that your proposed way of doing this works and is efficient.
> What I say is that I think there is room for improvement in the 
> non-functional aspects, in the readability. However, it's not really a 
> blocking issue, more of a personal taste.
I know what you are considering, it is clearer and more readable by consuming 
the length, then the USR. However, to correctly separate the USR and file path, 
the length of `USR-Length` is also required, which makes it impossible to just 
*consume* the length at the beginning.

Another way of solving this problem is to re-create the string with the 
USR-Length and the USR after parsing, but I think it is not a good solution.

BTW, is it necessary to assert the `USR-Length` to be greater than zero? I 
think it is more reasonable to report *invalid format* rather than assert the 
value, as it can be provided by the user.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:13
+int importee(int X) {
+  return 1 / X;
+}

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > Why do you need to have a div by zero warning?
> > I am not sure whether we should test if an external function can be 
> > correctly imported. Hence, I wrote a div-by-zero bug here. A call to 
> > function `clang_analyzer_warnIfReached` is also OK here.
> > 
> > As the imported lambda expr will not be called, I think I can only test 
> > whether CTU works via another normal function.
> AFAIK importing a function and import-related stuff are orthogonal to 
> actually emitting bug reports produced by the analyzer. That being said, if 
> the `importee()` would have an empty body, the observable behavior would 
> remain the same. And this is what I propose now.
Sorry, but I am not quite clear about your suggestions on this function.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> OikawaKirie wrote:
> > steakhal wrote:
> > > 
> > The source of lookup name of the function being imported is function 
> > `CrossTranslationUnitContext::getLookupName`. Keeping the length in the 
> > mapping can avoid parsing the lookup name during importing.
> Okay; you can copy the original StringRef to have that. But by consuming it 
> on one path makes the code much more readable.
> 
The `getAsInterger` call can also check whether the content before the first 
colon is an integer. Therefore, a sub-string operation is required here.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:7-8
+// CHECK-NOT: multiple definitions are found for the same key in index
+f([](char) -> int { return 42; });
+f([](char) -> bool { return true; });
+  }

steakhal wrote:
> I would rather put these into the `importee()`
The lambda exprs will not be included in the CTU index file if they are 
declared in a normal function.



Comment at: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp:13
+int importee(int X) {
+  return 1 / X;
+}

steakhal wrote:
> Why do you need to have a div by zero warning?
I am not sure whether we should test if an external function can be correctly 
imported. Hence, I wrote a div-by-zero bug here. A call to function 
`clang_analyzer_warnIfReached` is also OK here.

As the imported lambda expr will not be called, I think I can only test whether 
CTU works via another normal function.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping

What do you think about this patch? Can it be landed now? Or I should debug the 
crash in the Windows version detected with the previous version of my patch.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:154-172
+  // Find the length delimiter.
+  const size_t LengthDelimiter = LineRef.find(':');
+  if (StringRef::npos == LengthDelimiter)
+return false;
+
+  // Parse the length of LookupName as USRLength.
+  size_t USRLength = 0;

steakhal wrote:
> 
The source of lookup name of the function being imported is function 
`CrossTranslationUnitContext::getLookupName`. Keeping the length in the mapping 
can avoid parsing the lookup name during importing.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 391968.
OikawaKirie added a comment.

1. Fix formatting bugs
2. Update lookup name format as `: ` in all 
comments and documents
3. Add the new test case as a part of 
`clang/test/Analysis/func-mapping-test.cpp` to verify the lookup name
4. Change the macros for asserting the lookup name format to an `NDEBUG` 
wrapped function


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
@@ -154,9 +154,9 @@
 
 TEST(CrossTranslationUnit, IndexFormatCanBeParsed) {
   llvm::StringMap Index;
-  Index["a"] = "/b/f1";
-  Index["c"] = "/d/f2";
-  Index["e"] = "/f/f3";
+  Index["1:a"] = "/b/f1";
+  Index["1:c"] = "/d/f2";
+  Index["1:e"] = "/f/f3";
   std::string IndexText = createCrossTUIndexString(Index);
 
   int IndexFD;
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
@@ -48,3 +48,15 @@
   float uf;
   const int ui;
 };
+
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+f([](char) -> int { return 42; });
+// CHECK-DAG: 41:c:@S@G@F@G#@Sa@F@operator int (*)(char)#1
+f([](char) -> bool { return true; });
+// CHECK-DAG: 42:c:@S@G@F@G#@Sa@F@operator bool (*)(char)#1
+  }
+};
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,22 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp 2>&1 | FileCheck --allow-empty importee.cpp
+
+int importee(int);
+
+void trigger() {
+  importee(0); // expected-warn...@importee.cpp:13 {{Division by zero}}
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- 

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.
Herald added a subscriber: manas.

It seems that this issue has better solutions outside the CTU module, I will 
drop this revision as mentioned above.
Sorry for the delay.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102149/new/

https://reviews.llvm.org/D102149

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 390653.
OikawaKirie marked an inline comment as done.
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.
Herald added a subscriber: manas.

It has been a long period since the last discussion, I hope you can still 
remember this bug. And apologize for the delay.

Updated as required, the lookup name generator 
`CrossTranslationUnitContext::getLookupName` and parser `parseCrossTUIndex` are 
modified.
And assertions are added before accessing the CTU index mapping for input 
lookup names to be searched.

Corresponding test cases and documents are also updated.

Please let me know if there are other files to be updated.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

Files:
  clang/docs/analyzer/user-docs/CrossTranslationUnit.rst
  clang/include/clang/Basic/DiagnosticCrossTUKinds.td
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-import.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/Inputs/ctu-other.c.externalDefMap.ast-dump.txt
  clang/test/Analysis/Inputs/ctu-other.cpp.externalDefMap.ast-dump.txt
  
clang/test/Analysis/Inputs/plist-macros-with-expansion-ctu.c.externalDefMap.txt
  clang/test/Analysis/ctu-inherited-default-ctor.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/func-mapping-test.cpp
  clang/unittests/CrossTU/CrossTranslationUnitTest.cpp

Index: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
===
--- clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
+++ clang/unittests/CrossTU/CrossTranslationUnitTest.cpp
@@ -61,7 +61,7 @@
 ASSERT_FALSE(llvm::sys::fs::createTemporaryFile("index", "txt", IndexFD,
 IndexFileName));
 llvm::ToolOutputFile IndexFile(IndexFileName, IndexFD);
-IndexFile.os() << "c:@F@f#I# " << ASTFileName << "\n";
+IndexFile.os() << "9:c:@F@f#I# " << ASTFileName << "\n";
 IndexFile.os().flush();
 EXPECT_TRUE(llvm::sys::fs::exists(IndexFileName));
 
@@ -154,9 +154,9 @@
 
 TEST(CrossTranslationUnit, IndexFormatCanBeParsed) {
   llvm::StringMap Index;
-  Index["a"] = "/b/f1";
-  Index["c"] = "/d/f2";
-  Index["e"] = "/f/f3";
+  Index["1:a"] = "/b/f1";
+  Index["1:c"] = "/d/f2";
+  Index["1:e"] = "/f/f3";
   std::string IndexText = createCrossTUIndexString(Index);
 
   int IndexFD;
Index: clang/test/Analysis/func-mapping-test.cpp
===
--- clang/test/Analysis/func-mapping-test.cpp
+++ clang/test/Analysis/func-mapping-test.cpp
@@ -3,10 +3,10 @@
 int f(int) {
   return 0;
 }
-// CHECK-DAG: c:@F@f#I#
+// CHECK-DAG: 9:c:@F@f#I#
 
 extern const int x = 5;
-// CHECK-DAG: c:@x
+// CHECK-DAG: 4:c:@x
 
 // Non-const variables should not be collected.
 int y = 5;
@@ -18,29 +18,29 @@
   int a;
 };
 extern S const s = {.a = 2};
-// CHECK-DAG: c:@s
+// CHECK-DAG: 4:c:@s
 
 struct SF {
   const int a;
 };
 SF sf = {.a = 2};
-// CHECK-DAG: c:@sf
+// CHECK-DAG: 5:c:@sf
 
 struct SStatic {
   static const int a = 4;
 };
 const int SStatic::a;
-// CHECK-DAG: c:@S@SStatic@a
+// CHECK-DAG: 14:c:@S@SStatic@a
 
 extern int const arr[5] = { 0, 1 };
-// CHECK-DAG: c:@arr
+// CHECK-DAG: 6:c:@arr
 
 union U {
   const int a;
   const unsigned int b;
 };
 U u = {.a = 6};
-// CHECK-DAG: c:@u
+// CHECK-DAG: 4:c:@u
 
 // No USR can be generated for this.
 // Check for no crash in this case.
Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,22 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp 2>&1 | FileCheck --allow-empty importee.cpp
+
+int importee(int);
+
+void trigger() {
+  importee(0); // expected-warn...@importee.cpp:13 {{Division by zero}}
+}
Index: clang/test/Analysis/ctu-inherited-default-ctor.cpp
===
--- clang/test/Analysis/ctu-inherited-default-ctor.cpp
+++ 

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347581.
OikawaKirie added a comment.

Update the test case to avoid a crash in the Windows version of the 
`c-index-test` tool.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

Files:
  clang/lib/Index/USRGeneration.cpp
  clang/test/Index/USR/MemberFunctionPtr.cpp


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test -index-file %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: name: V | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: name: f | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test -index-file %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: name: f | USR: c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: name: ftd | USR: c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: name: fus | USR: c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: name: V | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: name: f | USR: c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D101763#2777821 , @steakhal wrote:

> It seems like everything passes. Yeey, good job!
> Shall I commit this tomorrow on your behalf?

I do not have commit access to the code base. It would be appreciated if you 
could commit it on my behalf (Ella Ma ).
Thanks to all reviewers again for your suggestions.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347446.
OikawaKirie added a comment.

Re-submit the patch to re-build the code.

Sorry for the spam of submitting an incomplete diff which triggers a patch 
apply failure.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/trigger.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file": "%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Add an empty invocation list to make the on-demand parsing fail and load it again.
+// RUN: echo '' > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/trigger.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// CHECK: {{Opening file invocations.yaml: 1}}
+// CHECK-NOT: {{Opening file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Opening file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347441.
OikawaKirie added a comment.

Re-submit the patch to re-build the code.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/trigger.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file": "%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Add an empty invocation list to make the on-demand parsing fail and load it again.
+// RUN: echo '' > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/trigger.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// CHECK: {{Opening file invocations.yaml: 1}}
+// CHECK-NOT: {{Opening file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Opening file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 347327.
OikawaKirie added a comment.
Herald added a subscriber: manas.

In D101763#2768549 , @martong wrote:

> First of all, thank you for the patch!
> We had a meeting with my colleges (@steakhal, @gamesh411) and we agreed in 
> the following. This is indeed an issue and the fix is okay. About the test, 
> we'd like to ask if you could create a small test lib with its own 'open' 
> function that would simply call the system's open and would print 'open'. And 
> then with LD_PREOLOAD you could use that lib in the lit test. (If that 
> approach is not working for some reason then we may just mark the test with 
> XFAIL.)

Updated as suggested based on a previous submit 
https://reviews.llvm.org/D101763?id=343346. An empty invocation list is used to 
trigger a parsing failure, and name `invocations.yaml` is used as the file name 
for logging and checking.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/trigger.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file": "%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Add an empty invocation list to make the on-demand parsing fail and load it again.
+// RUN: echo '' > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/trigger.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// CHECK: {{Opening file invocations.yaml: 1}}
+// CHECK-NOT: {{Opening file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Opening file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return 

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added inline comments.



Comment at: clang/lib/Index/USRGeneration.cpp:897
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";

akyrtzi wrote:
> A bit better to do `VisitTagDecl(MPT->getClass())`, what do you think?
This was my first version of the patch, which was 
`VisitTagDecl(MPT->getMostRecentCXXRecordDecl())`. However, it cannot handle 
the `TemplateTypeParmType` type, which is presented in the last two CHECKs of 
the test case. Call to method `MemberPointerType::getMostRecentCXXRecordDecl` 
will dereference a nullptr in the function (`getClass()->getAsCXXRecordDecl()` 
returns nullptr) . To fully handle all circumstances, I finally choose to use 
the `VisitType` method.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102614/new/

https://reviews.llvm.org/D102614

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102149#2768541 , @martong wrote:

> Thank you for the patch!
>
> Though, the idea is nice, there is a serious technical obstacle here: we 
> cannot use the clangTooling lib as a dependency of the CTU lib because that 
> would introduce a circular dependency. Actually, this was the reason for 
> introducing the invocation list yaml file; we could not use the compilation 
> database implementation from tooling 
> (https://reviews.llvm.org/D75665?id=260238#inline-722328).

According to my recent experiences on using clang-check, converting the 
compilation database to an invocation list is also an acceptable solution.
Currently, a better solution may be adding a tool to handle the conversion, 
just as what has been mentioned in the revision you presented. Although, it 
would be a very simple python script such as:

  a = dict()
  for i in json.load(open("/path/to/compile_commands.json")):
  a[os.path.abspath(os.path.join(i['directory'], i['file']))] = \
  (shlex.split(i['command']) if 'command' in i else i['arguments']) + \
  ['-Xclang', '-working-directory=' + i['directory']]
  print(yaml.dump(a));

If you agree with this idea, I will follow the way to create a tool for the 
conversion and drop this revision.

Actually, I am writing a makefile to schedule the ctu-index generation as well 
as analyzing the code with the clang-check tool. And the python script snippet 
presented above is just a part of my makefile.
Maybe I can add my makefile to as a part of the analyzer.




Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:711-712
+
+  List.emplace_back("-Xclang");
+  List.emplace_back("-working-directory=" + CC.Directory);
+}

The approach of converting a compilation database to an invocation list is just 
simply adding the `-working-directory` argument for the `"directory"` entry for 
each compile command object. Therefore, a script doing the conversion can make 
it simpler and easier to use.

The main reason why this patch is submitted is I previously do not know the 
`-working-directory` argument. Without this argument, it would be very 
difficult and complex to convert a compilation database to an invocation list.

If you agree that my approach of converting the database (presented in the for 
loop) is OK, I will continue with making a tool for the conversion.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102149/new/

https://reviews.llvm.org/D102149

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102062: [analyzer][ctu] Append ctu-dir to ctu-invocation-list for non-absolute paths

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.

In D102062#2768569 , @martong wrote:

> First of all, thank you for the patch!
> We had a meeting with my colleges (@steakhal, @gamesh411) and we agreed in 
> the following.
>
> We'd like to keep the current behavior because this way the behavior is 
> similar that we got used to with any other command line tools.

Ok, if you all agree with using `ctu-invocation-list` with a path related to 
CWD rather than `ctu-dir`, I will drop this revision here.
Although, I still believe that it would be better to make the behavior similar 
to the ctu index file, which is related to the `ctu-dir` (see the comments in 
the code).

Thanks for your suggestions on this revision.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102062/new/

https://reviews.llvm.org/D102062

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102149#2765402 , @martong wrote:

> Thank you @OikawaKirie for working on this many CTU related patches! I am 
> going to find time for a thorough review and going to pursue @gamesh411 as 
> well to do the same! On the other hand, it would be really useful if you 
> could build a "Stack" from these patches, I mean could you please set which 
> patch depends from which other (see "Edit related revisions")?

To make it easier to review, all these recently submitted patches are based on 
the `main` branch. Their code does not depend on each other.

Logically, they can be separated into two branches: invocation list and ctu 
index.
The invocation list related patches target on constructing the invocation list 
in ASTLoader from a compilation database. And the problems of using invocation 
lists are also fixed.
Whereas the ctu index related patches mainly focus on the delimiter character 
in the ctu index, which is generated by the `clang-extdef-mapping` tool.

If the "stack" means the logical relations, I will update that later. 
Otherwise, it is OK to start with any of the patches.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102149/new/

https://reviews.llvm.org/D102149

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie marked an inline comment as done.
OikawaKirie added a comment.

In D102669#2765233 , @steakhal wrote:

> I'm not really familiar with the extdefmap part, but I'm surprised that we 
> are using spaces as separators.
> Shouldn't we consider using a different character?

I prefer the idea of changing the delimiter character, but it may lead to 
modifying a lot of test cases.
I think we'd better make this change in another revision in the future if we do 
want to change it.




Comment at: clang/test/Analysis/ctu-lookup-name-with-space.cpp:7
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > 
%t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", 
"file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c 
%t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+

steakhal wrote:
> Probably splitting this up into multiple lines would result in a more 
> readable solution.
> 
> Something along these lines should work:
> ```
> cat >%t/compile_commands.json < line 1
> line 2
> ...
> EOL
> ```
The suggestion is great, however I cannot find a way to write the `RUN` 
commands.
Could you please tell me how to write the commands in this way? It is also 
useful to help me merging the test case into one file.



Comment at: clang/test/Analysis/ctu-lookup-name-with-space.cpp:11-23
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   trigger.cpp 2>&1 | FileCheck importee.cpp
+

steakhal wrote:
> Why do you need two separate invocations? Couldn't you just merge these?
> I've seen cases where `-verify` was used in conjunction with `FileCheck`.
I forgot the `--allow-empty` argument during writing this test case. I will 
merge them in an update.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102669/new/

https://reviews.llvm.org/D102669

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102159: [index][analyzer][ctu] Eliminate white spaces in the CTU lookup name.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.

It seems impossible and not so reasonable to eliminate all white space 
characters in the USR as mentioned in the test case of revision D102669 
.
This patch is split to revision D102669  to 
fix the wrongly parsed CTU index file and revision D102614 
 to handle the member pointer type mentioned 
here.
Please continue with these two following revisions and I will close this one.

Thanks to all reviewers.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102159/new/

https://reviews.llvm.org/D102159

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102669: [analyzer][ctu] Fix wrong 'multiple definitions' errors caused by space characters in lookup names when parsing the ctu index file

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: gamesh411, martong, hgabii.
OikawaKirie added a project: clang.
Herald added subscribers: steakhal, ASDenysPetrov, dkrupp, donat.nagy, 
Szelethus, arphaman, mikhail.ramalho, a.sidorin, rnkovacs, szepet, 
baloghadamsoftware, xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

This error was found when analyzing MySQL with CTU enabled.

When there are space characters in the lookup name, the current delimiter 
searching strategy will make the file path wrongly parsed.
And when two lookup names have the same prefix before their first space 
characters, a 'multiple definitions' error will be wrongly reported.

e.g. The lookup names for the two lambda exprs in the test case are 
`c:@S@G@F@G#@Sa@F@operator int (*)(char)#1` and `c:@S@G@F@G#@Sa@F@operator bool 
(*)(char)#1` respectively. And their prefixes are both 
`c:@S@G@F@G#@Sa@F@operator` when using the first space character as the 
delimiter.

This patch solves the problem by replacing `find` with `rfind` for the 
delimiter, as the chance of file paths having space characters seems to be far 
less than the lookup name.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102669

Files:
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
  clang/test/Analysis/ctu-lookup-name-with-space.cpp


Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,29 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > 
%t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", 
"file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c 
%t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   trigger.cpp 2>&1 | FileCheck importee.cpp
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   -verify trigger.cpp
+
+int importee(int);
+
+void trigger() {
+  importee(0); // expected-warn...@importee.cpp:13 {{Division by zero}}
+}
Index: clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/Inputs/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,14 @@
+void f(int (*)(char));
+void f(bool (*)(char));
+
+struct G {
+  G() {
+// CHECK-NOT: error: multiple definitions are found for the same key in 
index
+f([](char) -> int { return 42; });
+f([](char) -> bool { return true; });
+  }
+};
+
+int importee(int X) {
+  return 1 / X;
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -157,7 +157,7 @@
   unsigned LineNo = 1;
   while (std::getline(ExternalMapFile, Line)) {
 StringRef LineRef{Line};
-const size_t Delimiter = LineRef.find(' ');
+const size_t Delimiter = LineRef.rfind(' ');
 if (Delimiter > 0 && Delimiter != std::string::npos) {
   StringRef LookupName = LineRef.substr(0, Delimiter);
 


Index: clang/test/Analysis/ctu-lookup-name-with-space.cpp
===
--- /dev/null
+++ clang/test/Analysis/ctu-lookup-name-with-space.cpp
@@ -0,0 +1,29 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+
+// RUN: cp %s %t/trigger.cpp
+// RUN: cp %S/Inputs/ctu-lookup-name-with-space.cpp %t/importee.cpp
+// RUN: echo '"%t/importee.cpp" : ["g++", "-c", "%t/importee.cpp"]' > %t/invocations.yaml
+// RUN: echo '[{"directory": "%t", "command": "g++ -c %t/importee.cpp", "file": "%t/importee.cpp"}, {"directory": "%t", "command": "g++ -c %t/trigger.cpp", "file": "%t/trigger.cpp"}]' > %t/compile_commands.json
+
+// RUN: %clang_extdef_map -p %t %t/importee.cpp > %t/externalDefMap.txt
+
+// RUN: cd %t && %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// 

[PATCH] D102614: [index] Add support for type of pointers to class members

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added a reviewer: akyrtzi.
OikawaKirie added a project: clang.
Herald added a subscriber: arphaman.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Required in D102159  that we should add 
support for the unhandled items instead of workaround them.
In this patch, support is added for indexing the type of pointers to class 
members. Such usages are found in MySQL.
The format is `::*`.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102614

Files:
  clang/lib/Index/USRGeneration.cpp
  clang/test/Index/USR/MemberFunctionPtr.cpp


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test core -print-source-symbols -- %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: static-property/C++ | V | c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: instance-method/C++ | f | c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';


Index: clang/test/Index/USR/MemberFunctionPtr.cpp
===
--- /dev/null
+++ clang/test/Index/USR/MemberFunctionPtr.cpp
@@ -0,0 +1,33 @@
+// RUN: c-index-test core -print-source-symbols -- %s | FileCheck %s
+
+struct C {
+  int X;
+  void f(char);
+};
+
+void f(int C::*) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*I#
+void f(void (C::*)(char)) {}
+// CHECK: function/C | f | c:@F@f#$@S@C::*Fv(#C)#
+
+typedef int C::*Xtd;
+void ftd(Xtd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*I#
+typedef void (C::*Ftd)(char);
+void ftd(Ftd) {}
+// CHECK: function/C | ftd | c:@F@ftd#$@S@C::*Fv(#C)#
+
+using Xus = int C::*;
+void fus(Xus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*I#
+using Fus = void (C::*)(char);
+void fus(Fus) {}
+// CHECK: function/C | fus | c:@F@fus#$@S@C::*Fv(#C)#
+
+template  struct S;
+template  struct S {
+  static const bool V = true;
+  // CHECK: static-property/C++ | V | c:@SP>2#T#T@S>#t0.1::*t0.0@V
+  void f() {}
+  // CHECK: instance-method/C++ | f | c:@SP>2#T#T@S>#t0.1::*t0.0@F@f#
+};
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -893,6 +893,12 @@
   T = AT->getElementType();
   continue;
 }
+if (const MemberPointerType *MPT = T->getAs()) {
+  VisitType(QualType(MPT->getClass(), 0));
+  Out << "::*";
+  T = MPT->getPointeeType();
+  continue;
+}
 
 // Unhandled type.
 Out << ' ';
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102159: [index][analyzer][ctu] Eliminate white spaces in the CTU lookup name.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102159#2749033 , @akyrtzi wrote:

> Maybe we could also handle this kind of type instead of leaving it 
> 'unhandled'? What `Type` is it?

The member function pointer type, see the test case.

Although it would be perfect to handle this kind of type, I mean the white 
spaces should still be removed from the USR.
Currently, the white space character is used as the separator between the index 
string and the file path in the output of `clang-extdef-mapping`.
And it is difficult to determine when the index string with white space 
characters ends when parsing the output of `clang-extdef-mapping`.
Therefore, IMO the white space character had better not be used in the index 
string.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102159/new/

https://reviews.llvm.org/D102159

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102159: [index][analyzer][ctu] Eliminate white spaces in the CTU lookup name.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: akyrtzi, martong, steakhal.
OikawaKirie added a project: clang.
Herald added subscribers: ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
arphaman, mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware, 
xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

In the analyzer, the CTU definition index file requires there are no white 
spaces in the CTU lookup name, aka. the USR of a function decl or a global 
variable decl. Otherwise, the index file will be wrongly parsed.
However, it is difficult for the analyzer to know whether a white space `' '` 
is the separator between the index string and the file path, or it is just a 
part of the index string or the path. It means that using either the first or 
the last white space as the separator could not be a good idea. As it is valid 
for a file path to have white spaces, a better choice seems to be eliminating 
all the white spaces in the index string.
In this patch, the white space `' '` for an unsupported type is replaced with a 
question mark `'?'`, which solves the problem as well as makes the index more 
reasonable as far as I am thinking.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102159

Files:
  clang/lib/Index/USRGeneration.cpp
  clang/test/Index/unsupported.cpp


Index: clang/test/Index/unsupported.cpp
===
--- /dev/null
+++ clang/test/Index/unsupported.cpp
@@ -0,0 +1,10 @@
+// RUN: %clang_extdef_map %s -- | FileCheck %s
+
+struct X {
+  bool f(char *);
+};
+
+typedef bool (X::*XFP)(char *);
+
+// CHECK-NOT: {{ [^ ]+ }}
+void f(XFP xfp) {}
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -895,7 +895,7 @@
 }
 
 // Unhandled type.
-Out << ' ';
+Out << '?';
 break;
   } while (true);
 }


Index: clang/test/Index/unsupported.cpp
===
--- /dev/null
+++ clang/test/Index/unsupported.cpp
@@ -0,0 +1,10 @@
+// RUN: %clang_extdef_map %s -- | FileCheck %s
+
+struct X {
+  bool f(char *);
+};
+
+typedef bool (X::*XFP)(char *);
+
+// CHECK-NOT: {{ [^ ]+ }}
+void f(XFP xfp) {}
Index: clang/lib/Index/USRGeneration.cpp
===
--- clang/lib/Index/USRGeneration.cpp
+++ clang/lib/Index/USRGeneration.cpp
@@ -895,7 +895,7 @@
 }
 
 // Unhandled type.
-Out << ' ';
+Out << '?';
 break;
   } while (true);
 }
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102149: [analyzer][ctu] Allow loading invocation list from a compilation database automatically detected from the ctu-dir

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: steakhal, gamesh411, martong, balazske.
OikawaKirie added a project: clang.
Herald added subscribers: ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware, xazax.hun, 
mgorny.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

It seems to be simpler and more convenient to reuse the compilation database 
generated for ClangTool when using clang-check to run the analyzer.
In this patch, when the invocation list file is failed to be loaded, the 
ASTLoader will try to find a compilation database from the `ctu-dir` and 
convert the detected database to an invocation list.

For each compile command objects in the database:

- the "directory" entry is converted to option `-working-directory` appended to 
the invocation commands;
- the "file" entry is updated to an absolute path based on the "directory" 
entry as the index;
- and the "command" entry is converted to the invocation commands.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102149

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CMakeLists.txt
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing.c
  clang/test/Analysis/ctu-on-demand-parsing.cpp

Index: clang/test/Analysis/ctu-on-demand-parsing.cpp
===
--- clang/test/Analysis/ctu-on-demand-parsing.cpp
+++ clang/test/Analysis/ctu-on-demand-parsing.cpp
@@ -13,6 +13,7 @@
 //
 // RUN: cd "%t" && %clang_extdef_map Inputs/ctu-chain.cpp Inputs/ctu-other.cpp > externalDefMap.txt
 //
+// Run with invocation list.
 // RUN: cd "%t" && %clang_analyze_cc1 \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
@@ -26,6 +27,19 @@
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
 // RUN:   -analyzer-config display-ctu-progress=true ctu-on-demand-parsing.cpp 2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
 //
+// Run with compilation database.
+// RUN: rm %t/invocations.yaml
+// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -verify ctu-on-demand-parsing.cpp
+// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config display-ctu-progress=true ctu-on-demand-parsing.cpp 2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
+//
 // CHECK: CTU loaded AST file: {{.*}}ctu-other.cpp
 // CHECK: CTU loaded AST file: {{.*}}ctu-chain.cpp
 //
Index: clang/test/Analysis/ctu-on-demand-parsing.c
===
--- clang/test/Analysis/ctu-on-demand-parsing.c
+++ clang/test/Analysis/ctu-on-demand-parsing.c
@@ -11,6 +11,7 @@
 //
 // RUN: cd "%t" && %clang_extdef_map "%t/ctu-other.c" > externalDefMap.txt
 //
+// Run with invocation list.
 // RUN: cd "%t" && %clang_cc1 -fsyntax-only -std=c89 -analyze \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
@@ -18,6 +19,14 @@
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
 // RUN:   -verify ctu-on-demand-parsing.c
 //
+// Run with compilation database.
+// RUN: rm %t/invocations.yaml
+// RUN: cd "%t" && %clang_cc1 -fsyntax-only -std=c89 -analyze \
+// RUN:   -analyzer-checker=core,debug.ExprInspection \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -verify ctu-on-demand-parsing.c
+//
 // FIXME: Path handling should work on all platforms.
 // REQUIRES: system-linux
 
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -19,6 +19,8 @@
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/TextDiagnosticPrinter.h"
 #include "clang/Index/USRGeneration.h"
+#include "clang/Tooling/CompilationDatabase.h"
+#include "clang/Tooling/Tooling.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/Triple.h"
@@ -119,14 +121,14 @@
 case index_error_code::invocation_list_ambiguous:
   return "Invocation list file contains multiple references to the same "
  "source file.";
-case index_error_code::invocation_list_file_not_found:
-  return "Invocation list file is not found.";
 case index_error_code::invocation_list_empty:
   return "Invocation list file is empty.";
 case 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

> FileCheck error: '' is empty.

It fails again on the build bot, another dead end. Maybe I need a Debian server 
to reproduce the problem encountered in the test case.

In D101763#2745918 , @steakhal wrote:

> In D101763#2745802 , @OikawaKirie 
> wrote:
>
>> I do not know why the test case always fails on the build server, it runs 
>> perfectly on my computer. : (
>
> I've locally run your patch and seemed good to me, I regret not having a look 
> at the actual build bot.

I have tried the test case on a Ubuntu 20.04 server with kernel 5.4.0 and gcc 
9.3.0, as well as a CentOS 7 server with kernel 3.10.0 and gcc 7.1.0. Both of 
them work well.

> In D101763#2745900 , @OikawaKirie 
> wrote:
>
>> Monitor function `open` together with `openat` to fix the failure in the 
>> test case.
>> As some distros actually call function `open` rather than forward to 
>> function `openat`, both functions should be monitored by `strace`.
>>
>> But I am still worrying about whether function `open` will call `openat` and 
>> vice versa. If this circumstance is possible, the test case may still fail.
>> Although, I have not found any pieces of evidence for such invocations.
>
> Uh, that's a fair point. Now, I'm also worried xD
> To be fair, I don't have any better idea testing this without overwhelming 
> effort.
> If anything goes wrong, we can simply revert this change. This is 'just' a 
> performance fix.

I will try to reproduce and fix the problem of the test case next week. If I 
cannot find other ways to overcome the problem by the next weekend, we can have 
a try on landing this patch.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343818.
OikawaKirie added a comment.

Monitor function `open` together with `openat` to fix the failure in the test 
case.
As some distros actually call function `open` rather than forward to function 
`openat`, both functions should be monitored by `strace`.

But I am still worrying about whether function `open` will call `openat` and 
vice versa. If this circumstance is possible, the test case may still fail.
Although, I have not found any pieces of evidence for such invocations.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.c

Index: clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/ctu-on-demand-parsing-multiple-invocation-list-parsing.c
@@ -0,0 +1,30 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: strace -e trace=open,openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+void bar();
+
+void foo() {
+  // It finds the invocation yaml, then it tries to parse it which expected to fail.
+  bar();
+  // CHECK: open{{(at)?}}({{[^"]*}}"{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+
+  // We should remember that the parsing failed, we shouldn't try to reopen and parse the yaml again.
+  bar();
+  // CHECK-NOT: open{{(at)?}}({{[^"]*}}"{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code PreviousParsingResult = index_error_code::success;
   };
 
   /// Maintain number of AST loads and 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D101763#2741629 , @steakhal wrote:

> Okay, so you 'just' want an indication for the given open call. What about 
> using `strace`?
> `strace -e trace=openat %clang_cc1 ... 2>&1 | grep '"invocations.yaml"' | 
> FileCheck %s`

I do not know why the test case always fails on the build server, it runs 
perfectly on my computer. : (
Do any reviewers have any ideas?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102062: [analyzer][ctu] Append ctu-dir to ctu-invocation-list for non-absolute paths

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D102062#2744267 , @steakhal wrote:

> On second thought the current behavior is reasonable.

The behavior of all CTU affairs are related to the `ctu-dir` rather than CWD, 
such as loading the external function map file and looking for the AST dumps 
and source files to be imported.
If a relative path is provided for these files, the `ctu-dir` will be added to 
the path when trying to open the file (see the comments in the code).
Therefore, IMO the behavior of `ctu-invocation-list` and its contents should 
also be relative to the `ctu-dir` unless an absolute path is provided.

> We call clang from a command line, so I think it's fair to expect that 
> relative paths are resolved using the CWD.

I do not know how other people use CSA, but I prefer using CSA via 
`clang-check`.
It works with the relative path to the "directory" option in the compilation 
database of each source file, which could make the CWD not unique.

> AFAIK if one does not supply the `ctu-invocation-list`, then it would be 
> substituted by `ctu-dir/invocations.yaml` anyway.

But we cannot currently distinguish the difference between setting 
`ctu-invocation-list` to `invocations.yaml` and not setting it. As the default 
value of `ctu-invocation-list` is `invocations.yaml` rather than an empty 
string.
Therefore, adding the `ctu-dir` to a relative `ctu-invocation-list` path by 
default seems to be a better choice.




Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:470
   else
 llvm::sys::path::append(IndexFile, IndexName);
 

Loading the external function map.



Comment at: clang/lib/CrossTU/CrossTranslationUnit.cpp:516
 Path = CTUDir;
 llvm::sys::path::append(Path, PathStyle, Identifier);
   }

Loading the ASTUnit from external AST dumps or source files.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D102062/new/

https://reviews.llvm.org/D102062

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D102062: [analyzer][ctu] Append ctu-dir to ctu-invocation-list for non-absolute paths

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: gamesh411, steakhal, martong, balazske.
OikawaKirie added a project: clang.
Herald added subscribers: ASDenysPetrov, dkrupp, donat.nagy, Szelethus, 
mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware, xazax.hun.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

This patch fixes the problem mentioned in D101763#2743984 
.
When the `ctu-invocation-list` is not an absolute path and the current working 
directory is not the `ctu-dir`, the invocation list will fail to be loaded.
In this patch, the `ctu-dir` is appended to the `ctu-invocation-list` if the 
`ctu-invocation-list` is not an absolute path.
And the original test cases for on-demand parsing are reused to test this 
feature.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102062

Files:
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/ctu-on-demand-parsing.c
  clang/test/Analysis/ctu-on-demand-parsing.cpp


Index: clang/test/Analysis/ctu-on-demand-parsing.cpp
===
--- clang/test/Analysis/ctu-on-demand-parsing.cpp
+++ clang/test/Analysis/ctu-on-demand-parsing.cpp
@@ -11,20 +11,20 @@
 //
 // RUN: echo '{"%t/Inputs/ctu-chain.cpp": ["g++", "%t/Inputs/ctu-chain.cpp"], 
"%t/Inputs/ctu-other.cpp": ["g++", "%t/Inputs/ctu-other.cpp"]}' | sed -e 
's/\\//g' > %t/invocations.yaml
 //
-// RUN: cd "%t" && %clang_extdef_map Inputs/ctu-chain.cpp Inputs/ctu-other.cpp 
> externalDefMap.txt
+// RUN: %clang_extdef_map -p %t %t/Inputs/ctu-chain.cpp 
%t/Inputs/ctu-other.cpp > %t/externalDefMap.txt
 //
-// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN: %clang_analyze_cc1 \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
-// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-dir=%t \
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
-// RUN:   -verify ctu-on-demand-parsing.cpp
-// RUN: cd "%t" && %clang_analyze_cc1 \
+// RUN:   -verify %t/ctu-on-demand-parsing.cpp
+// RUN: %clang_analyze_cc1 \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
-// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-dir=%t \
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
-// RUN:   -analyzer-config display-ctu-progress=true ctu-on-demand-parsing.cpp 
2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
+// RUN:   -analyzer-config display-ctu-progress=true 
%t/ctu-on-demand-parsing.cpp 2>&1 | FileCheck %t/ctu-on-demand-parsing.cpp
 //
 // CHECK: CTU loaded AST file: {{.*}}ctu-other.cpp
 // CHECK: CTU loaded AST file: {{.*}}ctu-chain.cpp
Index: clang/test/Analysis/ctu-on-demand-parsing.c
===
--- clang/test/Analysis/ctu-on-demand-parsing.c
+++ clang/test/Analysis/ctu-on-demand-parsing.c
@@ -7,16 +7,16 @@
 // compile_commands.json is only needed for extdef_mapping, not for the 
analysis itself.
 // RUN: echo '[{"directory":"%t","command":"gcc -std=c89 -Wno-visibility 
ctu-other.c","file":"ctu-other.c"}]' | sed -e 's/\\//g' > 
%t/compile_commands.json
 //
-// RUN: echo '"%t/ctu-other.c": ["gcc", "-std=c89", "-Wno-visibility", 
"ctu-other.c"]' | sed -e 's/\\//g' > %t/invocations.yaml
+// RUN: echo '"%t/ctu-other.c": ["gcc", "-std=c89", "-Wno-visibility", 
"%t/ctu-other.c"]' | sed -e 's/\\//g' > %t/invocations.yaml
 //
-// RUN: cd "%t" && %clang_extdef_map "%t/ctu-other.c" > externalDefMap.txt
+// RUN: %clang_extdef_map -p %t "%t/ctu-other.c" > %t/externalDefMap.txt
 //
-// RUN: cd "%t" && %clang_cc1 -fsyntax-only -std=c89 -analyze \
+// RUN: %clang_cc1 -fsyntax-only -std=c89 -analyze \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
-// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-dir=%t \
 // RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
-// RUN:   -verify ctu-on-demand-parsing.c
+// RUN:   -verify %t/ctu-on-demand-parsing.c
 //
 // FIXME: Path handling should work on all platforms.
 // REQUIRES: system-linux
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -668,8 +668,14 @@
   if (InvocationList)
 return llvm::Error::success();
 
+  SmallString<256> InvocationListFileAbsolutePath = CTUDir;
+  if (llvm::sys::path::is_absolute(InvocationListFilePath))
+InvocationListFileAbsolutePath = InvocationListFilePath;
+  else
+llvm::sys::path::append(InvocationListFileAbsolutePath,
+InvocationListFilePath);
   llvm::ErrorOr> 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343626.
OikawaKirie added a comment.

In D101763#2743984 , @steakhal wrote:

> By inspecting the output that is piped to the `count`, I have a suspicion:
>
>   openat(AT_FDCWD, "invocations.yaml", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No 
> such file or directory)

According to the output of the failed test, it seems that CTU is not working 
during the test case execution (`Expected 1 lines, got 0.`). Otherwise, it 
should be matched by grep.
There seems to be something wrong with the test case, although it works well in 
my environment.

> My guess is that `ctu-invocation-list` is not using the `ctu-dir`. It should 
> be a different bug though.
> To workaround this issue, simply use an absolute path for the invocation yaml.

It seems that you are correct, I will fix this problem later.
I can just `cd` to the `ctu-dir` before running CSA as a workaround. Just as 
the author of on-demand parsing did (refer to 
`clang/test/Analysis/ctu-on-demand-parsing.c`).

> `// CHECK: openat(AT_FDCWD, "{{[^"]+}}invocations.yaml", O_RDONLY|O_CLOEXEC) 
> = {{[0-9]+}}`

I am not quite sure whether we need to verify the flags (`AT_FDCWD`, `O_RDONLY` 
and `O_CLOEXEC`) as well. Therefore, I removed them in this update.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.c

Index: clang/test/Analysis/multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.c
@@ -0,0 +1,30 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: cd %t && \
+// RUN: strace -e trace=openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=. \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+void bar();
+
+void foo() {
+  // It finds the invocation yaml, then it tries to parse it which expected to fail.
+  bar();
+  // CHECK: openat({{[^,]+}}, "{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+
+  // We should remember that the parsing failed, we shouldn't try to reopen and parse the yaml again.
+  bar();
+  // CHECK-NOT: openat({{[^,]+}}, "{{[^"]*/?}}invocations.yaml", {{[^)]+}}) = {{[0-9]+}}
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343585.
OikawaKirie added a comment.

Replace `wc -l` with `count`.

Sorry for the spam.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.c

Index: clang/test/Analysis/multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.c
@@ -0,0 +1,26 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: strace -e trace=openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | count 1
+
+// REQUIRES: shell, system-linux
+
+void bar();
+
+void foo() {
+  // Import twice to see whether the second import will open the invocation
+  // list again.
+  bar();
+  bar();
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code PreviousParsingResult = index_error_code::success;
   };
 
   /// Maintain number of AST loads and check for reaching the load limit.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343582.
OikawaKirie added a comment.

- Update the test case as suggested.
- Add a case branch for the `index_error_code::success` error code in function 
`IndexErrorCategory::message` to silent the compiler warnings.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.c

Index: clang/test/Analysis/multiple-invocation-list-parsing.c
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.c
@@ -0,0 +1,29 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// RUN: strace -e trace=openat \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %s 2>&1 | grep 'invocations\.yaml' | wc -l | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// Check the log for the second open of the invocation list.
+// CHECK: {{1}}
+
+void bar();
+
+void foo() {
+  // Import twice to see whether the second import will open the invocation
+  // list again.
+  bar();
+  bar();
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -92,6 +92,10 @@
 
   std::string message(int Condition) const override {
 switch (static_cast(Condition)) {
+case index_error_code::success:
+  // There should not be a success error. Jump to unreachable directly.
+  // Add this case to make the compiler stop complaining.
+  break;
 case index_error_code::unspecified:
   return "An unknown error has occurred.";
 case index_error_code::missing_index_file:
@@ -667,12 +671,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +687,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  success = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code PreviousParsingResult = index_error_code::success;
   };
 
   /// Maintain number of AST loads and check for reaching the load limit.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343346.
OikawaKirie added a comment.

In D101763#2741536 , @steakhal wrote:

> Overall, it looks promising. But I don't quite get this test.
> There is no invocation yaml in the temp directory. So, you are probably not 
> testing the right thing.
> You wanted to test if the invocation yaml exists, and could be opened **but 
> the parsing fails**.
> You should demonstrate that when a parsing error happens, the error code has 
> recoded and it won't try to reparse the invocation yaml again and again.

The main idea of the test case is that the mocked `open` function will dump a 
log every time when the invocation list file is opened to be read. If a second 
open operation is detected in the log, it means the list is parsed again.

As the behavior of missing the invocation list file or failing to parse it are 
the same (both returns the error), the previous version checks whether function 
`open` will be called again after a file-not-found error.
In this update, I use an empty invocation list to really trigger an 
`invocation_list_empty` error and check whether the invocation list file will 
be read again via function `open`.
Compared with the previous version, opening the invocation list will not be 
manually failed, and a real parsing error is triggered instead.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.cpp
@@ -0,0 +1,56 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// Compile the mocked `open` function.
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/importer.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+// RUN: echo "" > %t/invocations.yaml
+
+// Run the test code.
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/importer.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// Check the log for the second open of the invocation list.
+// CHECK: {{Mocking file invocations.yaml: 1}}
+// CHECK-NOT: {{Mocking file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // Log how many times the invocation list is opened.
+  if ("invocations.yaml" == string(name)) {
+static unsigned N = 0;
+cout << "Mocking file invocations.yaml: " << ++N << endl;
+  }
+
+  // The original open function will be called to open the files.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,15 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult = index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +683,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for 

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 343293.
OikawaKirie added a comment.

Add a regression test case by mocking the `open` function. When this function 
is called with the file name of the invocation list, the mocked `open` function 
will reject the open operation and dump a log. And we will then check how many 
times are the invocation list opened. (Thanks to the ideas by steakhal in 
another patch of mocking a library function :-).)

Identifiers renamed as suggested.

Unfortunately, we cannot store and copy a `llvm::Error` object. Therefore, the 
new version still uses the error code to store the previous parsing results.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D101763/new/

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp
  clang/test/Analysis/multiple-invocation-list-parsing.cpp

Index: clang/test/Analysis/multiple-invocation-list-parsing.cpp
===
--- /dev/null
+++ clang/test/Analysis/multiple-invocation-list-parsing.cpp
@@ -0,0 +1,59 @@
+// RUN: rm -rf %t
+// RUN: mkdir -p %t
+
+// Compile the mocked `open` function.
+// RUN: %host_cxx %s -fPIC -shared -o %t/mock_open.so
+
+// RUN: echo "void bar(); void foo() { bar(); bar(); }" > %t/importer.c
+// RUN: echo "void bar() {}" > %t/importee.c
+// RUN: echo '[{"directory":"%t", "command":"cc -c %t/importee.c", "file":"%t/importee.c"}]' > %t/compile_commands.json
+// RUN: %clang_extdef_map -p %t "%t/importee.c" > %t/externalDefMap.txt
+
+// Run the test code.
+// RUN: LD_PRELOAD=%t/mock_open.so \
+// RUN: %clang_cc1 -fsyntax-only -analyze \
+// RUN:   -analyzer-checker=core \
+// RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
+// RUN:   -analyzer-config ctu-dir=%t \
+// RUN:   -analyzer-config ctu-invocation-list=invocations.yaml \
+// RUN:   %t/importer.c | FileCheck %s
+
+// REQUIRES: shell, system-linux
+
+// Check the log for the second open of the invocation list.
+// CHECK: {{Mocking file invocations.yaml: 1}}
+// CHECK-NOT: {{Mocking file invocations.yaml: 2}}
+
+#define _GNU_SOURCE 1
+#include 
+#include 
+
+#include 
+#include 
+#include 
+using namespace std;
+
+extern "C" int open(const char *name, int flag, ...) {
+  // If the opened function is the invocation list, reject opening and return
+  // an error.
+  // Log how many times the open operation have been rejected.
+  string sname(name);
+  if ("invocations.yaml" == sname) {
+static unsigned N = 0;
+cout << "Mocking file invocations.yaml: " << ++N << endl;
+return -1;
+  }
+
+  // For other files, the original open function will be called.
+  using open_t = int (*)(const char *, int, mode_t);
+  static open_t o_open = nullptr;
+  if (!o_open)
+o_open = reinterpret_cast(dlsym(RTLD_NEXT, "open"));
+  assert(o_open && "Cannot find function `open'.");
+
+  va_list vl;
+  va_start(vl, flag);
+  auto mode = va_arg(vl, mode_t);
+  va_end(vl);
+  return o_open(name, flag, mode);
+}
Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,16 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::success != PreviousParsingResult)
+return llvm::make_error(PreviousParsingResult);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+PreviousParsingResult =
+index_error_code::invocation_list_file_not_found;
+return llvm::make_error(PreviousParsingResult);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +684,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](const IndexError ) { PreviousParsingResult = E.getCode(); });
+return llvm::make_error(PreviousParsingResult);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Ping @sammccall again for landing this patch on my behalf.
Thanks.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D101763: [analyzer][ctu] Avoid parsing invocation list again and again during on-demand parsing of CTU

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: gamesh411, balazske, martong, xazax.hun.
OikawaKirie added a project: clang.
Herald added subscribers: steakhal, ASDenysPetrov, dkrupp, donat.nagy, 
Szelethus, mikhail.ramalho, a.sidorin, rnkovacs, szepet, baloghadamsoftware.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

During CTU, the *on-demand parsing* will read and parse the invocation list to 
know how to compile the file being imported. However, it seems that the 
invocation list will be parsed again if a previous parsing has failed. Then, 
parse again and fail again. This patch tries to overcome the problem by storing 
the error code during the first parsing, and re-create the stored error during 
the later parsings.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D101763

Files:
  clang/include/clang/CrossTU/CrossTranslationUnit.h
  clang/lib/CrossTU/CrossTranslationUnit.cpp


Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,16 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::no_error != InvocationListParsingError)
+return llvm::make_error(InvocationListParsingError);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+InvocationListParsingError =
+index_error_code::invocation_list_file_not_found;
+return llvm::make_error(InvocationListParsingError);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer 
"
   "should not be nullptr.");
@@ -680,8 +684,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](IndexError ) { InvocationListParsingError = E.getCode(); });
+return llvm::make_error(InvocationListParsingError);
+  }
 
   InvocationList = *ExpectedInvocationList;
 
Index: clang/include/clang/CrossTU/CrossTranslationUnit.h
===
--- clang/include/clang/CrossTU/CrossTranslationUnit.h
+++ clang/include/clang/CrossTU/CrossTranslationUnit.h
@@ -38,6 +38,7 @@
 namespace cross_tu {
 
 enum class index_error_code {
+  no_error = 0,
   unspecified = 1,
   missing_index_file,
   invalid_index_format,
@@ -253,6 +254,7 @@
 /// In case of on-demand parsing, the invocations for parsing the source
 /// files is stored.
 llvm::Optional InvocationList;
+index_error_code InvocationListParsingError = index_error_code::no_error;
   };
 
   /// Maintain number of AST loads and check for reaching the load limit.


Index: clang/lib/CrossTU/CrossTranslationUnit.cpp
===
--- clang/lib/CrossTU/CrossTranslationUnit.cpp
+++ clang/lib/CrossTU/CrossTranslationUnit.cpp
@@ -667,12 +667,16 @@
   /// Lazily initialize the invocation list member used for on-demand parsing.
   if (InvocationList)
 return llvm::Error::success();
+  else if (index_error_code::no_error != InvocationListParsingError)
+return llvm::make_error(InvocationListParsingError);
 
   llvm::ErrorOr> FileContent =
   llvm::MemoryBuffer::getFile(InvocationListFilePath);
-  if (!FileContent)
-return llvm::make_error(
-index_error_code::invocation_list_file_not_found);
+  if (!FileContent) {
+InvocationListParsingError =
+index_error_code::invocation_list_file_not_found;
+return llvm::make_error(InvocationListParsingError);
+  }
   std::unique_ptr ContentBuffer = std::move(*FileContent);
   assert(ContentBuffer && "If no error was produced after loading, the pointer "
   "should not be nullptr.");
@@ -680,8 +684,13 @@
   llvm::Expected ExpectedInvocationList =
   parseInvocationList(ContentBuffer->getBuffer(), PathStyle);
 
-  if (!ExpectedInvocationList)
-return ExpectedInvocationList.takeError();
+  // Handle the error to store the code for next call to this function.
+  if (!ExpectedInvocationList) {
+llvm::handleAllErrors(
+ExpectedInvocationList.takeError(),
+[&](IndexError ) { InvocationListParsingError = E.getCode(); });
+return llvm::make_error(InvocationListParsingError);
+  }
 
   InvocationList = 

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

If this patch is OK, could you please commit it on my behalf (Ella Ma 
)?
Thx.
And I will continue working on the `-Xanalyzer` option for clang-check, as it 
is too complex when adding analyzer options via the `-extra-arg` option.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D83660#2675064 , @mikhail.ramalho 
wrote:

> Indeed it looks like a copy & paste error, I'm surprised no one found it 
> earlier.
>
> Regarding the tests, we used to have `make check-clang-analysis-z3` (or 
> something similar) that would run only the analyzer's tests, but using Z3 as 
> the constraint solver. It looks like this change broke it: 
> https://reviews.llvm.org/D62445

I add `VERBOSE=1` during execution, and command 
`/path/to/llvm-project/build/./bin/llvm-lit -sv --param USE_Z3_SOLVER=0 
/path/to/llvm-project/clang/test/Analysis/z3` is used to execute lit for 
testing.
And this test case is not executed.

Should the execution requirements be changed to make it run if z3 is enabled? 
Or just keep it as it is now?

If there are no other suggestions for this patch, I'd like to see it landed 
ASAP. I think it is a far too long period for a fix of a copy & paste error.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

ping @sammccall again

Do I need to add other reviewers if you are busy doing your job?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D83660#2661369 , @martong wrote:

>> BTW, I was obstructed by the z3 requirement in the regression test case when 
>> I tried to understand your test case. Even though I set the variables 
>> LLVM_Z3_INSTALL_DIR and LLVM_ENABLE_Z3_SOLVER during CMake, and the CSA can 
>> be correctly compiled with Z3, I still cannot make the test case run during 
>> make check-all. Therefore, this case was manually executed with llvm-lit 
>> -DUSE_Z3_SOLVER=1. Could you please tell me how to enable Z3 during llvm-lit.
>
> I am a bit unhappy that we cannot run the test automatically, but maybe in 
> the future.
> @steakhal, https://reviews.llvm.org/D83677 seems to be related, should we 
> push that?

I am just reporting this phenomenon and I am not sure whether it is caused by 
my bad configurations.
Under my configurations, the last step of `make check-all` will execute 
`llvm-lit` with argument `--param USE_Z3_SOLVER=0`, which will make all test 
cases requiring z3 skipped.
Although, clang is built with z3 under this configuration.

Can we automatically enable all test cases requiring z3 if clang is built with 
z3? I do not think the patch D83677  really 
make the problem fixed.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D83660: [analyzer] Fix a crash for dereferencing an empty llvm::Optional variable in SMTConstraintManager.h.

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 333708.
OikawaKirie added a comment.

In D83660#2597025 , @steakhal wrote:

> I'm somewhat bothered that this patch is not landed yet. :|
>
> I made a **crude** mock to trigger the bug using `LD_PRELOAD`. F15707493: 
> LD_PRELOAD-workaround.patch 
> The test reproduces the issue and passes if you apply the fix from this patch.
>
> However, it requires compiling a shared object with the same compiler you 
> used for building clang. I'm using the `c++` compiler, for now.
> I think it's a good starting point.

Simply replace the c++ compiler with `%host_cxx` compiler to compile the mock 
Z3 solver.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D83660/new/

https://reviews.llvm.org/D83660

Files:
  clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
  clang/test/Analysis/z3/D83660.c
  clang/test/Analysis/z3/Inputs/MockZ3_solver_check.c


Index: clang/test/Analysis/z3/Inputs/MockZ3_solver_check.c
===
--- /dev/null
+++ clang/test/Analysis/z3/Inputs/MockZ3_solver_check.c
@@ -0,0 +1,28 @@
+#include 
+#include 
+
+#include 
+
+// Mock implementation: return UNDEF for the 5th invocation, otherwise it just
+// returns the result of the real invocation.
+Z3_lbool Z3_API Z3_solver_check(Z3_context c, Z3_solver s) {
+  static Z3_lbool (*OriginalFN)(Z3_context, Z3_solver);
+  if (!OriginalFN) {
+OriginalFN = (Z3_lbool (*)(Z3_context, Z3_solver))dlsym(RTLD_NEXT,
+"Z3_solver_check");
+  }
+
+  // Invoke the actual solver.
+  Z3_lbool Result = OriginalFN(c, s);
+
+  // Mock the 5th invocation to return UNDEF.
+  static unsigned int Counter = 0;
+  if (++Counter == 5) {
+fprintf(stderr, "Z3_solver_check returns a mocked value: UNDEF\n");
+return Z3_L_UNDEF;
+  }
+  fprintf(stderr, "Z3_solver_check returns the real value: %s\n",
+  (Result == Z3_L_UNDEF ? "UNDEF"
+: ((Result == Z3_L_TRUE ? "TRUE" : "FALSE";
+  return Result;
+}
Index: clang/test/Analysis/z3/D83660.c
===
--- /dev/null
+++ clang/test/Analysis/z3/D83660.c
@@ -0,0 +1,23 @@
+// RUN: rm -rf %t && mkdir %t
+// RUN: %host_cxx -shared -fPIC %S/Inputs/MockZ3_solver_check.c -o 
%t/MockZ3_solver_check.so
+//
+// RUN: LD_PRELOAD="%t/MockZ3_solver_check.so" 
  \
+// RUN: %clang_cc1 -analyze -analyzer-constraints=z3 -setup-static-analyzer
  \
+// RUN:   -analyzer-checker=core,debug.ExprInspection %s -verify 2>&1 | 
FileCheck %s
+//
+// REQUIRES: z3, asserts, shell, system-linux
+//
+// Works only with the z3 constraint manager.
+// expected-no-diagnostics
+
+// CHECK:  Z3_solver_check returns the real value: TRUE
+// CHECK-NEXT: Z3_solver_check returns the real value: TRUE
+// CHECK-NEXT: Z3_solver_check returns the real value: TRUE
+// CHECK-NEXT: Z3_solver_check returns the real value: TRUE
+// CHECK-NEXT: Z3_solver_check returns a mocked value: UNDEF
+
+void D83660(int b) {
+  if (b) {
+  }
+  (void)b; // no-crash
+}
Index: 
clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
===
--- clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
+++ clang/include/clang/StaticAnalyzer/Core/PathSensitive/SMTConstraintManager.h
@@ -146,7 +146,7 @@
   Solver->addConstraint(NotExp);
 
   Optional isNotSat = Solver->check();
-  if (!isSat.hasValue() || isNotSat.getValue())
+  if (!isNotSat.hasValue() || isNotSat.getValue())
 return nullptr;
 
   // This is the only solution, store it


Index: clang/test/Analysis/z3/Inputs/MockZ3_solver_check.c
===
--- /dev/null
+++ clang/test/Analysis/z3/Inputs/MockZ3_solver_check.c
@@ -0,0 +1,28 @@
+#include 
+#include 
+
+#include 
+
+// Mock implementation: return UNDEF for the 5th invocation, otherwise it just
+// returns the result of the real invocation.
+Z3_lbool Z3_API Z3_solver_check(Z3_context c, Z3_solver s) {
+  static Z3_lbool (*OriginalFN)(Z3_context, Z3_solver);
+  if (!OriginalFN) {
+OriginalFN = (Z3_lbool (*)(Z3_context, Z3_solver))dlsym(RTLD_NEXT,
+"Z3_solver_check");
+  }
+
+  // Invoke the actual solver.
+  Z3_lbool Result = OriginalFN(c, s);
+
+  // Mock the 5th invocation to return UNDEF.
+  static unsigned int Counter = 0;
+  if (++Counter == 5) {
+fprintf(stderr, "Z3_solver_check returns a mocked value: UNDEF\n");
+return Z3_L_UNDEF;
+  }
+  fprintf(stderr, "Z3_solver_check returns the real value: %s\n",
+  (Result == Z3_L_UNDEF ? "UNDEF"
+: ((Result == Z3_L_TRUE ? "TRUE" : 

[PATCH] D98918: [clang][lit] Allow test cases to use the compiler that are used to compile Clang

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D98918#2636735 , @steakhal wrote:

> 



> Oh, wait a minute. The `config.host_cxx` was already there.
> Where is the symmetry :D It looks much better now, thanks.

Sounds not bad if it matters to you :P

> Regarding D83660 , I'm really looking 
> forward to it. It's a shame that it takes so long to land it.

Please commit this patch on my behalf (Ella Ma ), so 
that I can continue with D83660  with your 
mocked solver.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D98918/new/

https://reviews.llvm.org/D98918

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D98918: [clang][lit] Allow test cases to use the compiler that are used to compile Clang

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: thakis, steakhal.
OikawaKirie added a project: clang.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Required by D83660 . Test cases may want to 
use the host compiler to compile some mocks for the test case. This patch adds 
two substitutions `%host_cc` and `%host_cxx` to use the host compilers set via 
variable `CMAKE_C_COMPILER` and `CMAKE_CXX_COMPILER`.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D98918

Files:
  clang/test/lit.cfg.py
  clang/test/lit.site.cfg.py.in


Index: clang/test/lit.site.cfg.py.in
===
--- clang/test/lit.site.cfg.py.in
+++ clang/test/lit.site.cfg.py.in
@@ -15,6 +15,7 @@
 config.clang_tools_dir = path(r"@CLANG_TOOLS_DIR@")
 config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.target_triple = "@TARGET_TRIPLE@"
+config.host_cc = "@CMAKE_C_COMPILER@"
 config.host_cxx = "@CMAKE_CXX_COMPILER@"
 config.llvm_use_sanitizer = "@LLVM_USE_SANITIZER@"
 config.have_zlib = @LLVM_ENABLE_ZLIB@
Index: clang/test/lit.cfg.py
===
--- clang/test/lit.cfg.py
+++ clang/test/lit.cfg.py
@@ -92,6 +92,9 @@
 ('%hmaptool', "'%s' %s" % (config.python_executable,
  os.path.join(config.clang_tools_dir, 
'hmaptool'
 
+config.substitutions.append(('%host_cc', config.host_cc))
+config.substitutions.append(('%host_cxx', config.host_cxx))
+
 
 # Plugins (loadable modules)
 if config.has_plugins and config.llvm_plugin_ext:


Index: clang/test/lit.site.cfg.py.in
===
--- clang/test/lit.site.cfg.py.in
+++ clang/test/lit.site.cfg.py.in
@@ -15,6 +15,7 @@
 config.clang_tools_dir = path(r"@CLANG_TOOLS_DIR@")
 config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.target_triple = "@TARGET_TRIPLE@"
+config.host_cc = "@CMAKE_C_COMPILER@"
 config.host_cxx = "@CMAKE_CXX_COMPILER@"
 config.llvm_use_sanitizer = "@LLVM_USE_SANITIZER@"
 config.have_zlib = @LLVM_ENABLE_ZLIB@
Index: clang/test/lit.cfg.py
===
--- clang/test/lit.cfg.py
+++ clang/test/lit.cfg.py
@@ -92,6 +92,9 @@
 ('%hmaptool', "'%s' %s" % (config.python_executable,
  os.path.join(config.clang_tools_dir, 'hmaptool'
 
+config.substitutions.append(('%host_cc', config.host_cc))
+config.substitutions.append(('%host_cxx', config.host_cxx))
+
 
 # Plugins (loadable modules)
 if config.has_plugins and config.llvm_plugin_ext:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

@sammccall ping.

Updated as you mentioned in D97265#2581653 



CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 328927.
OikawaKirie added a comment.

Replace concatenating the output from clang-check and plist with separated 
checks to workaround the test failure on Windows.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

Files:
  clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments , StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
@@ -0,0 +1,14 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: cd %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo 
-ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: echo '// CHECK: {{qwerty}}' > %t/cclog-check
+// RUN: clang-check -p "%t" "%t/test.cpp" -analyze 
-analyzer-output-path=%t/qwerty -extra-arg=-v -extra-arg=-Xclang 
-extra-arg=-verify 2>&1 | FileCheck %t/cclog-check
+// RUN: FileCheck %s --input-file=%t/qwerty
+
+// CHECK: DOCTYPE plist
+// CHECK: Division by zero
+int f() {
+  return 1 / 0; // expected-warning {{Division by zero}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments , StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
@@ -0,0 +1,14 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: cd %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo -ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: echo '// CHECK: {{qwerty}}' > %t/cclog-check
+// RUN: clang-check -p "%t" "%t/test.cpp" -analyze -analyzer-output-path=%t/qwerty -extra-arg=-v -extra-arg=-Xclang -extra-arg=-verify 2>&1 | 

[PATCH] D97265: [clang] Allow clang-check to customize analyzer output file or dir name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie updated this revision to Diff 328396.
OikawaKirie retitled this revision from "[clang] Allow clang-check to customize 
output file name" to "[clang] Allow clang-check to customize analyzer output 
file or dir name".
OikawaKirie edited the summary of this revision.
OikawaKirie added a comment.
Herald added subscribers: steakhal, ASDenysPetrov, dkrupp, donat.nagy, 
Szelethus, a.sidorin, baloghadamsoftware.

1. Update option name from `-output` to `-analyzer-output-path`. As it can 
either be a file name or a directory name, the word `path` may not be accurate. 
Please give me your suggestions on this option name.
2. Add more verifications on the output files, rather than verifying the value 
of the `-o` option. For simplicity, the regression test case only checks the 
plist format.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97265/new/

https://reviews.llvm.org/D97265

Files:
  clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments , StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-set-analyzer-output-path.cpp
@@ -0,0 +1,15 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: cd %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo 
-ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: clang-check -p "%t" "%t/test.cpp" -analyze 
-analyzer-output-path=qwerty -extra-arg=-v -extra-arg=-Xclang 
-extra-arg=-verify 2>&1 | cat - qwerty | FileCheck %s
+// FIXME: Make the above easier.
+
+// CHECK: Invocation
+// CHECK: {{qwerty}}
+// CHECK: DOCTYPE plist
+// CHECK: Division by zero
+int f() {
+  return 1 / 0; // expected-warning {{Division by zero}}
+}


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,10 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+AnalyzerOutput("analyzer-output-path",
+   cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +209,19 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  Analyze ? [&](const CommandLineArguments , StringRef File) {
+  auto Ret = getClangStripOutputAdjuster()(Args, File);
+  if (!AnalyzerOutput.empty()) {
+Ret.emplace_back("-o");
+Ret.emplace_back(AnalyzerOutput);
+  }
+  return Ret;
+}
+  : getClangStripOutputAdjuster());
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-set-analyzer-output-path.cpp

[PATCH] D97265: [clang] Allow clang-check to customize output file name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D97265#2581653 , @sammccall wrote:

> 



> It looks like either we're in syntax-only mode (and -o will be ignored 
> regardless) or we're in analyzer mode, and want -o.
> It seems tempting to just stop stripping it in this case, but then we'll end 
> up with -o foo.o -o foo.html, which is no good.
> So the logic of this patch seems sound.

The strip output adjuster is invoked before appending the new `-o` option (line 
215). Therefore, there will be only one `-o` option left if the new `-output` 
option is set when running `clang-check`.

> I wonder about the CLI though, I think the user will have to pass four flags 
> with different styles to use it!
>
> `clang-check -analyze ... -extra-arg=-Xanalyzer 
> -extra-arg=-analyzer-output=html -output=html_output `
>
> (Maybe -analyze should imply -Xanalyzer, and analyzer-output should be 
> promoted to a real flag without the legacy plist default...)

You are right. It is better if we can add the `clang-check` version of 
`-Xanalyzer` options. However, the analyzer options can only determine the 
format of the output, rather than the name of the output. Therefore, we still 
need another option to set the output name.

My original idea is to add a generic way for the `clang-check` users to 
customize the output name. Although there is currently only one user, the 
static analyzer, there may be other users of this option in the future. 
Therefore, I thought that it would be better to add just a generic output 
option here.

According to your suggestions, it seems to be a good idea to use `-Xanalyzer` 
to set the output name, and the users can have a good experience when using 
clang-check to analyze their code. However, the disadvantage of this method is 
that the argument parsing procedure will become more complex. IMO, we can also 
add a `clang-check` option to set the analyzer output format together with the 
analyzer output name, and leave other analyzer options that are not commonly 
used to the `-Xanalyzer` option (refer to the second inline comment).




Comment at: clang/test/Tooling/clang-check-reset-o.cpp:10
+// CHECK: {{qwerty}}
+// CHECK: C++ requires
+invalid;

sammccall wrote:
> it looks like you're checking for the analysis finding in stdout of 
> clang-check, rather than in the generated html file.
> So i'm not sure this test verifies it actually works!
Different kinds of output formats can be either a directory or a file named 
after the `-o` option. It is inconvenient to check the exact output file or 
directory.

This test case is created by imitating the 
`clang/test/Tooling/clang-check-strip-o.cpp` test case, which only checks 
whether the cc1 arguments are correctly set.





Comment at: clang/tools/clang-check/ClangCheck.cpp:80
+static cl::opt
+Output("output", cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));

sammccall wrote:
> I think this should have a name with clear semantics like 
> -analyzer-output-file (and only be used in analyzer mode).
> 
> While it does just append -o to the command-line, I think it will confuse 
> users to treat it as a generic output flag.
> I think this should have a name with clear semantics like 
> -analyzer-output-file (and only be used in analyzer mode).
> While it does just append -o to the command-line, I think it will confuse 
> users to treat it as a generic output flag.

My original idea is to add a generic output flag to the `clang-check` tool, as 
it seems impossible for tool users to customize the output name. However, it 
will probably confuse users with the useless `-o` options set via `-extra-arg` 
or appended after `--` and with other `-fsyntax-only` actions, just as what you 
mentioned.

So, which you think is better? All with `-Xanalyzer` option
```
$ clang-check -analyze -Xanalyzer -analyzer-output-name=html_output -Xanalyzer 
-analyzer-output=html -Xanalyzer -analyzer-other-options=value input1.cc 
input2.cc ...
```
OR `-analyzer-output-name` without `-Xanalyzer` option
```
$  clang-check -analyze -analyzer-output-name=html_output -Xanalyzer 
-analyzer-output=html -Xanalyzer -analyzer-other-options=value input1.cc 
input2.cc ...
```
OR even more aggressively, only non-commonly used options with `-Xanalyzer` 
option
```
$  clang-check -analyze -analyzer-output-name=html_output -analyzer-output=html 
-Xanalyzer -analyzer-other-options=value input1.cc input2.cc ...
```



Comment at: clang/tools/clang-check/ClangCheck.cpp:213-221
+  Tool.appendArgumentsAdjuster(
+  [&](const CommandLineArguments , StringRef File) {
+auto Ret = getClangStripOutputAdjuster()(Args, File);
+if (!Output.empty()) {
+  Ret.emplace_back("-o");
+  Ret.emplace_back(Output);
+}

If we use the `-analyzer-output-name` option, we need to check whether 

[PATCH] D97251: [llvm] Add assertions for the smart pointers with the possibility to be null in ClangAttrEmitter

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie requested review of this revision.
OikawaKirie added a comment.

@aaron.ballman

My only concern is the recursive call to this function on line 1359. If you 
think it is also OK for the recursive call on line 1359, I will update the 
patch as you mentioned above.

BTW, how can I test and debug this tblgen backend?




Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1359
 for (const auto  : llvm::reverse(Bases)) {
   if ((Ptr = createArgument(Arg, Attr, Base.first)))
 break;

The recursive call is here.

The for loop here seems to allow a failed attempt on its bases. Although the 
top-level callers expect the return value of this function is always non-null, 
what about the call here?



Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1364
 
+  assert(Ptr && "Cannot create argument.");
+

aaron.ballman wrote:
> aaron.ballman wrote:
> > OikawaKirie wrote:
> > > Just as what has been suggested by @dexonsmith in D91844, I add the 
> > > assertion here. However, as a recursive function it is (line 1359), I 
> > > still concern whether the assertion here will lead to a crash.
> > > 
> > > What do you think?
> > The intent with this function is that it never returns `nullptr` unless the 
> > programmer messed up their call of the function (passed in the wrong kind 
> > of `Record` kind of problem). All of the callers assume this returns 
> > non-null, so I think it's reasonable to assert that the value is non-null 
> > by this point.
> 
However, there is a recursive call to this function on line 1359, and it seems 
that `nullptr` is allowed to be returned for the call. I am just worrying about 
whether the assertion here will lead to a crash in the recursive calls.

IMO, a better way is to wrap this function and assert the return value of this 
function in the wrapper function.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97251/new/

https://reviews.llvm.org/D97251

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97265: [clang] Allow clang-check to customize output file name

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: hokein, sammccall, kbobyrev, alexfh.
OikawaKirie added a project: clang.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Required by https://stackoverflow.com/questions/58073606

As the output argument is stripped out in the `clang-check` tool, it seems 
impossible for `clang-check` users to customize the output file name, even with 
`-extra-args` and `-extra-arg-before`.

This patch adds the `-output` argument to allow users to adjust the output 
name. And if the argument is not set, the original strip output adjuster will 
remove the output arguments.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D97265

Files:
  clang/test/Tooling/clang-check-reset-o.cpp
  clang/tools/clang-check/ClangCheck.cpp


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,9 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+Output("output", cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +208,18 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  [&](const CommandLineArguments , StringRef File) {
+auto Ret = getClangStripOutputAdjuster()(Args, File);
+if (!Output.empty()) {
+  Ret.emplace_back("-o");
+  Ret.emplace_back(Output);
+}
+return Ret;
+  });
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-reset-o.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-reset-o.cpp
@@ -0,0 +1,11 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo 
-ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: not clang-check -p "%t" "%t/test.cpp" -analyze -output=qwerty 
-extra-arg=-v 2>&1|FileCheck %s
+// FIXME: Make the above easier.
+
+// CHECK: Invocation
+// CHECK: {{qwerty}}
+// CHECK: C++ requires
+invalid;


Index: clang/tools/clang-check/ClangCheck.cpp
===
--- clang/tools/clang-check/ClangCheck.cpp
+++ clang/tools/clang-check/ClangCheck.cpp
@@ -76,7 +76,9 @@
 Analyze("analyze",
 cl::desc(Options.getOptionHelpText(options::OPT_analyze)),
 cl::cat(ClangCheckCategory));
-
+static cl::opt
+Output("output", cl::desc(Options.getOptionHelpText(options::OPT_o)),
+   cl::cat(ClangCheckCategory));
 static cl::opt
 Fixit("fixit", cl::desc(Options.getOptionHelpText(options::OPT_fixit)),
   cl::cat(ClangCheckCategory));
@@ -206,7 +208,18 @@
 
   // Clear adjusters because -fsyntax-only is inserted by the default chain.
   Tool.clearArgumentsAdjusters();
-  Tool.appendArgumentsAdjuster(getClangStripOutputAdjuster());
+
+  // Reset output path if is provided by user.
+  Tool.appendArgumentsAdjuster(
+  [&](const CommandLineArguments , StringRef File) {
+auto Ret = getClangStripOutputAdjuster()(Args, File);
+if (!Output.empty()) {
+  Ret.emplace_back("-o");
+  Ret.emplace_back(Output);
+}
+return Ret;
+  });
+
   Tool.appendArgumentsAdjuster(getClangStripDependencyFileAdjuster());
 
   // Running the analyzer requires --analyze. Other modes can work with the
Index: clang/test/Tooling/clang-check-reset-o.cpp
===
--- /dev/null
+++ clang/test/Tooling/clang-check-reset-o.cpp
@@ -0,0 +1,11 @@
+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: echo '[{"directory":".","command":"clang++ -c %t/test.cpp -o foo -ofoo","file":"%t/test.cpp"}]' | sed -e 's/\\/\//g' > %t/compile_commands.json
+// RUN: cp "%s" "%t/test.cpp"
+// RUN: not clang-check -p "%t" "%t/test.cpp" -analyze -output=qwerty -extra-arg=-v 2>&1|FileCheck %s
+// FIXME: Make the above easier.
+
+// CHECK: Invocation
+// CHECK: {{qwerty}}
+// CHECK: C++ requires
+invalid;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D91844: [llvm][clang] Add checks for the smart pointers with the possibility to be null

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie abandoned this revision.
OikawaKirie added a comment.

Split to

- D97251  
(clang/utils/TableGen/ClangAttrEmitter.cpp)
- D97185  
(llvm/lib/DWARFLinker/DWARFLinker.cpp)
- D97254  
(llvm/lib/LTO/ThinLTOCodeGenerator.cpp)
- D97255  
(llvm/lib/Support/VirtualFileSystem.cpp)
- D97258  (llvm/tools/llvm-link/llvm-link.cpp)

And the last one (llvm/utils/TableGen/OptParserEmitter.cpp) has been fixed by 
others.

Please review the new revisions, and this revision will be closed.

Thanks to all reviewers and sorry for the delay.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D91844/new/

https://reviews.llvm.org/D91844

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97251: [llvm] Add assertions for the smart pointers with the possibility to be null in ClangAttrEmitter

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

As a recursive function it is, the original patch in D91844 
 was adding assertions where the function is 
called other places in the file.

To confirm it will not lead to a crash here, I re-build my code base with clang 
and clang-tools-extra enabled. Everything seems to be good.

What do you think about the assertion? Should it be located just here or where 
the function is called?

Refer to D91844  for more details about the 
previous version.




Comment at: clang/utils/TableGen/ClangAttrEmitter.cpp:1364
 
+  assert(Ptr && "Cannot create argument.");
+

Just as what has been suggested by @dexonsmith in D91844, I add the assertion 
here. However, as a recursive function it is (line 1359), I still concern 
whether the assertion here will lead to a crash.

What do you think?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97251/new/

https://reviews.llvm.org/D97251

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97251: [llvm] Add assertions for the smart pointers with the possibility to be null in ClangAttrEmitter

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: dexonsmith, aaron.ballman, pcc.
OikawaKirie added a project: clang.
OikawaKirie requested review of this revision.
Herald added a subscriber: cfe-commits.

Split from D91844 .

The return value of createArgument in file 
clang/utils/TableGen/ClangAttrEmitter.cpp. Although there are a lot of checks 
in the function, nullptr is still possible to be returned.
In this patch, I added an assertion to confirm that.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D97251

Files:
  clang/utils/TableGen/ClangAttrEmitter.cpp


Index: clang/utils/TableGen/ClangAttrEmitter.cpp
===
--- clang/utils/TableGen/ClangAttrEmitter.cpp
+++ clang/utils/TableGen/ClangAttrEmitter.cpp
@@ -1361,6 +1361,8 @@
 }
   }
 
+  assert(Ptr && "Cannot create argument.");
+
   if (Ptr && Arg.getValueAsBit("Optional"))
 Ptr->setOptional(true);
 


Index: clang/utils/TableGen/ClangAttrEmitter.cpp
===
--- clang/utils/TableGen/ClangAttrEmitter.cpp
+++ clang/utils/TableGen/ClangAttrEmitter.cpp
@@ -1361,6 +1361,8 @@
 }
   }
 
+  assert(Ptr && "Cannot create argument.");
+
   if (Ptr && Arg.getValueAsBit("Optional"))
 Ptr->setOptional(true);
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92634: [Analyzer] Diagnose signed integer overflow

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D92634#2477342 , @danielmarjamaki 
wrote:

>> However, the mainstream compilers like GCC and Clang implement this as the 
>> overflowed value, and some programmers also use this feature to do some 
>> tricky things.
>
> hmm.. you mean if some -fwrapv flag is used right. yes I should disable this 
> checking then.
>
> For information, I am not very worried about signed integer overflow. I am 
> mostly worried about the compiler optimisations. If the compiler determines 
> that there is signed integer overflow in a execution path and removes all 
> code in that path.
>
> I was inspired by this blog post:  https://www.airs.com/blog/archives/120
>
> This function:
>
>   int f(int x) { return 0x7ff0 < x && x + 32 < 0x7fff; }
>
> might be optimized to:
>
>   int f(int x) { return 0; }

Yes, you are right. Currently, in the CSA, the symbolic execution engine will 
return true for this function. Condition `0x7ff0 < x` will assume x to be 
[2147483632, 2147483647], then `x + 32` will be evaluated to [-2147483632, 
-2147483617] which will make the check return true.

But for the example you mentioned, I prefer implementing this check with a 
coding style checker for Clang-Tidy, although the integer overflow checker in 
the CSA is also necessary.
As the evaluation result potentially conflicts with the literal arithmetical 
result, reporting directly this conflict condition expression would be easier 
for programmers to diagnostic the code.
Besides, as far as I am thinking, the compiler optimizes the expression as it 
is literally inferable, i.e. the result should be determinable literally during 
compilation. Otherwise, it will be computed during runtime. Therefore I suggest 
you can do a similar check with the ASTMatcher, since the `and` and `or` 
conjunctions will be removed in the CFG.

BTW, I cannot optimize function `f` to returning zero directly with GCC-10.2.1 
and Clang-10.0.1 under -O3. Should I add any other flags? Or it is version 
specific?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92634/new/

https://reviews.llvm.org/D92634

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92634: [Analyzer] Diagnose signed integer overflow

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D92634#2476161 , @danielmarjamaki 
wrote:

>> Besides, the return value should be the exact value computed from the two 
>> integers, even unknown, rather than undefined. As the developers may 
>> overflow an integer on purpose.
>
> I am not sure what you mean. If there is undefined behavior then the value 
> should be undefined and nothing else.. right?

Exactly, it is undefined behavior in the C++ standard. However, the mainstream 
compilers like GCC and Clang implement this as the overflowed value, and some 
programmers also use this feature to do some tricky things. Therefore I suggest 
the computed value should be "the exact value computed from the two integers". 
Or it can be the `Unknown` `SVal`, but rather than the `Undefined` `SVal`, as 
the `Undefined` `SVal` is used to represent what is read from an uninitialized 
variable.

But I do not favour the `Unknown` solution, as it could also trigger other 
problems in the engine, just as what has been mentioned by steakhal. Or maybe 
it would be no longer a problem if you implement this in a checker, but as a 
non-fatal error it is, you can just leave the overflowed value as it is, and 
report the problem only without terminating the symbol execution on this path. 
There is no need to report this problem all the time.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92634/new/

https://reviews.llvm.org/D92634

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92634: [Analyzer] Diagnose signed integer overflow

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

I think it could be better to implement this check with a checker on 
`PreStmt` and so on. And IMO, checkers have enough 
functionalities to report these problems.

Besides, the return value should be the exact value computed from the two 
integers, even unknown, rather than undefined. As the developers may overflow 
an integer on purpose.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92634/new/

https://reviews.llvm.org/D92634

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Replies from the original author Hao Zhang 

---

Sorry for replying late.

> I think it would be good to understand why the tests failed. Are some clients 
> relying on the name matching the query? If so, why?

I'm currently working on this. Maybe try making `FileManager::getFileRef` 
always canonicalize to an absolute path, as you mentioned.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Replies from the original author Hao Zhang 

--

> ...split the data structures between relative and absolute paths. The 
> existing data structures would only store absolute paths, but there are new 
> ones for relative paths that can be cached and swapped out.

I like the idea that using one cache for absolute paths and multiple caches for 
relative paths. Better than my approach.

> Another point is that FileManager takes the working directory at construction 
> time via FileSystemOptions. There's a FIXME to remove that argument, but (as 
> you've seen) the assumption is baked in that the CWD doesn't change. I think 
> the current patch may not completely fix it, since you can't modify 
> FileSystemOptions.

It's the point. It is the assumption that the CWD won't change leads to the 
problem. I agree with you that `FileSystemOptions` should be removed and grab 
the CWD directly from the VFS.

> Another option here is to assume the VFS never changes directories (no need 
> to store the CWD) ... Then if ClangTool wants to change directories, it would 
> be expected to maintain an appropriate set of VFS layers.

IIUC, does this mean that clients should create new `Filesystem`s when they 
change directories? It looks to me that it might conflict with the design of 
`Filesystem` since it already provides an API to change the CWD.

Another option (which I mentioned in the previous post) is to use absolute 
paths to query the cache, something like this:

  llvm::ErrorOr
  FileManager::getFile(StringRef Filename, bool openFile, bool CacheFailure) {
// Froce using the absolute path to query the cache
llvm::SmallString<128> AbsPath(Filename);
makeAbsolutePath(AbsPath);
auto Result = getFileRef(AbsPath.str(), openFile, CacheFailure);
if (Result)
  return >getFileEntry();
return llvm::errorToErrorCode(Result.takeError());
  }

It is more likely a defensive workaround to this problem. It might not be a 
good solution to the real problem (the assumption that the CWD won't change). 
The above code resulted in failures of some test cases, I haven't looked 
closely into why they failed. In my point of view, `Filename` should only be 
used as the key to query the cache. Logically, if I change it into the absolute 
path, those test cases should pass as usual.

Anyway, your approach (remove `FileSystemOptions`, one cache for absolute 
paths, multiple caches for relative paths) looks good to me.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

Replies from the original author Hao Zhang 
(Sorry for the wrong email address in the previous reply.)

--

Thanks for your reply.

I noticed this problem when I was using clang-check to analyze a project which 
had a `compile_commands.json`. A simplified test case is provided in the patch.

As far as I know this problem only happens in `clang/lib/Tooling/Tooling.cpp`. 
(as @OikawaKirie has explained)

However, it is more likely a bug in `FileManager`, since `FileManager` is not 
aware that whether the current working directory has been changed or not.

One solution could be to add an API, such as 
`notifyCurrentWorkingDirectoryChange`. Clients call if necessary. (Similar to 
the 3rd one in your alternative solution). Now the question is what to do in 
`notifyCurrentWorkingDirectoryChange`:

- In my solution, I use a pretty straightforward approach, which is to have an 
individual cache (for `FileEntry` and any other related things) for each 
working directory, and switch caches between working directories.



- In your alternative solution, if I standerstand it correctly, 
`dropRelativePaths` is called in `notifyCurrentWorkingDirectoryChange`. All 
those `FileEntry` which are based on relative paths are dropped from the cache. 
I think this is a good idea, especially for keeping `FileManager` simple. 
However I'm not sure how many files are based on relative paths. If 
`dropRelativePaths` drops too many files and the working directory is switched 
back to a previous one (this could happen when analyzing with a 
`compile_commands.json`), it might result in many cache misses, thus involving 
more system calls to open files.

There was an another solution I have ever tried, without things like 
`notifyCurrentWorkingDirectoryChange`. The solution is still straightforward, 
which is to use absolute path (instead of filename) to query/store a 
`FileEntry` from/to the cache. Actually I have tried this before but it ended 
up with lots of test cases failed. I don't know where I did wrong. If you think 
this approach is okay, I will continue working on this, and it might take some 
time.

I'm still not sure which solution is more suitable, or if there is a better 
one. Any suggestions are welcome!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

In D92160#2443405 , @dexonsmith wrote:

> 



In D92160#2443413 , @dexonsmith wrote:

> 

Replies on my own:

> Can you tell me more about the scenario this happens?

You can first manually try the regression test case in this patch.
As the `%t/b/config.h` is empty, there should not be any reading operations to 
the file. However, as the length of `%t/a/config.h` is wrongly used for 
`%t/b/config.h` because of the bug, the file does be read, and some warnings 
about reading `'\0'` characters are generated during compilation.

> Why is the FileManager being reused after the working directory change?

It seems to be the problem of ClangTool. All the input source files are handled 
one by one with the same FileManager instance from the ClangTool::Files. (See 
the comments in the file `clang/lib/Tooling/Tooling.cpp`)

> I'm also wondering about an alternate solution that optimizes for the common 
> case, such as the following:
>
> Don't specifically track relative paths / absolute paths.
> Add an API, something like: dropRelativePaths(), which drops all 
> {File,Directory}Entry{,Ref} that are somehow based on relative paths. This 
> can forward to a similar function the stat cache if necessary.
> Update clients to call that if the working directory changes (you'd want to 
> do it only if it actually changed, not always).
> This would keep FileManager from having to growing complexity to track 
> working directory changes.
>
> I realize this may not be clear; what I'm suggesting is that 
> dropRelativePaths() would visit all entries, check if they're relative, and 
> drop them if so.

I am not quite clear with the fixes in this patch, I will forward the replies 
to these questions from the original author Hao Zhang  to 
you when he replies to me.

Thanks




Comment at: clang/lib/Tooling/Tooling.cpp:545
   LLVM_DEBUG({ llvm::dbgs() << "Processing: " << File << ".\n"; });
   ToolInvocation Invocation(std::move(CommandLine), Action, Files.get(),
 PCHContainerOps);

The CWD changes for each input file, but the same FileManager instance is used.

And relative paths are used to index the file entries.

When there are two files in different CWDs having the same relative path, they 
will be mixed up.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92160/new/

https://reviews.llvm.org/D92160

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92639: [analyzer] Add control flow arrows to the analyzer's HTML reports

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie added a comment.

It is really a good idea!

The operations that would not leave an event in the report are now clearly 
printed.

But there are three arrows that confuse me in the example report: the 
assignment `x = 0` (x -> 0 -> x), the function call `dereference(x)` (x -> 
dereference), and the return statement `return *x` (int -> *x). I know the 
arrow is based on the evaluation order of the engine. But from the view of a 
user, I think these arrows are confusing to some extent.

For the first two, I think it would be better to point just the statement 
(maybe a `CFGElement`) without inner arrows (x -> 0 -> x and x -> dereference), 
or point to the location of the operator itself rather than the BeginLoc (e.g. 
x -> 0 -> =). For the third one, an arrow from the function name to the first 
`CFGElement` looks good to me. And an arrow from the returned expr to the 
return type or to a special mark (e.g. ⬅️) can also be added, together with 
function calls (an arrow from the callstmt to a special mark, e.g. ➡️).

By the way, what do you think about adding arrows for data flows of specific 
symbolic values in the future?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D92639/new/

https://reviews.llvm.org/D92639

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D92160: [clang] Fix wrong FDs are used for files with same name in Tooling

[Ella Ma via Phabricator via cfe-commits]

OikawaKirie created this revision.
OikawaKirie added reviewers: dexonsmith, harlanhaskins, rsmith, arphaman, 
MaskRay, bkramer, ilya-biryukov.
OikawaKirie added a project: clang.
Herald added a subscriber: cfe-commits.
OikawaKirie requested review of this revision.

ClangTool will make FileManager mix up two header files with the same relative 
path in different absolute paths.

As the cache of previously opened FileEntry in FileManager is indexed by the 
file name, when relative paths are used as the index, wrong FileEntry may be 
used for the file with the same relative path. With ClangTool, as the current 
working directory will change when parsing multiple files, files with the same 
relative paths but different absolute paths will be mixed up by the FileManager.

Submit on behalf of Hao Zhang , I will forward the reviews and 
his replies.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D92160

Files:
  clang/include/clang/Basic/FileManager.h
  clang/lib/Basic/FileManager.cpp
  clang/lib/Tooling/Tooling.cpp
  
clang/test/Tooling/multiple-source-include-different-header-with-same-relative-path.cpp

Index: clang/test/Tooling/multiple-source-include-different-header-with-same-relative-path.cpp
===
--- /dev/null
+++ clang/test/Tooling/multiple-source-include-different-header-with-same-relative-path.cpp
@@ -0,0 +1,42 @@
+// This test case presents a circumstance that the information related one file
+// is misused by another.
+//
+// The path tree of the test directory is as follors.
+//   .
+//   ├── a
+//   │   ├── a.c
+//   │   └── config.h
+//   ├── b
+//   │   ├── b.c
+//   │   └── config.h
+//   └── compile_commands.json
+//
+// Both source files (a/a.c and b/b.c) includes the config.h file of their own
+// directory with `#include "config.h"`. However, the two config.h files are
+// different. File a/config.h is longer than b/config.h. Both a/a.c and b/b.c
+// are compiled in their own directory, which is recorded in the compilation
+// database.
+//
+// When using ClangTool to parse these two source files one by one, since the
+// file name of both header files are the same, the FileManager will confuse
+// with them and using the file entry of a/config.h for b/config.h. And the
+// wrong file length will lead to a buffer overflow when reading the file.
+//
+// In this test case, to avoid buffer overflow, we use the leading '\0' in an
+// empty buffer to trigger the problem. We set a/config.h as an empty line
+// comment, and leave b/config.h empty. Firstly, a/config.h is read and cached,
+// then when reading b/config.h, if the size of a/config.h is used, the first
+// two chars are read and the first one must be a '\0'.
+
+// RUN: rm -rf %t
+// RUN: mkdir -p %t/a
+// RUN: mkdir -p %t/b
+// RUN: echo '#include "config.h"' > %t/a/a.c
+// RUN: echo '#include "config.h"' > %t/b/b.c
+// RUN: echo '//' > %t/a/config.h
+// RUN: echo ''   > %t/b/config.h
+// RUN: echo '[{"arguments": ["cc", "-c", "-o", "a.o", "a.c"], "directory": "%t/a", "file": "a.c"}, {"arguments": ["cc", "-c", "-o", "b.o", "b.c"], "directory": "%t/b", "file": "b.c"}]' > %t/compile_commands.json
+
+// The following two test RUNs should have no output.
+// RUN: cd %t && clang-check a/a.c b/b.c 2>&1 | count 0
+// RUN: cd %t && clang-check b/b.c a/a.c 2>&1 | count 0
Index: clang/lib/Tooling/Tooling.cpp
===
--- clang/lib/Tooling/Tooling.cpp
+++ clang/lib/Tooling/Tooling.cpp
@@ -513,7 +513,7 @@
   CompileCommand.Directory))
 llvm::report_fatal_error("Cannot chdir into \"" +
  Twine(CompileCommand.Directory) + "\"!");
-
+  Files->notifyCurrentWorkingDirectoryChange();
   // Now fill the in-memory VFS with the relative file mappings so it will
   // have the correct relative paths. We never remove mappings but that
   // should be fine.
@@ -560,6 +560,7 @@
 OverlayFileSystem->setCurrentWorkingDirectory(InitialWorkingDir))
   llvm::errs() << "Error when trying to restore working dir: "
<< EC.message() << "\n";
+Files->notifyCurrentWorkingDirectoryChange();
   }
   return ProcessingFailed ? 1 : (FileSkipped ? 2 : 0);
 }
Index: clang/lib/Basic/FileManager.cpp
===
--- clang/lib/Basic/FileManager.cpp
+++ clang/lib/Basic/FileManager.cpp
@@ -50,12 +50,13 @@
 
 FileManager::FileManager(const FileSystemOptions ,
  IntrusiveRefCntPtr FS)
-: FS(std::move(FS)), FileSystemOpts(FSO), SeenDirEntries(64),
-  SeenFileEntries(64), NextFileUID(0) {
+: FS(std::move(FS)), FileSystemOpts(FSO) {
   // If the caller doesn't provide a virtual file system, just grab the real
   // file system.
   if (!this->FS)
 this->FS = llvm::vfs::getRealFileSystem();
+
+