Re: [clang] 33a745e - [clang][clang-scan-deps] Add support for extracting full module dependencies.
You could hack it to standardize the yaml output to forward slashes with
sed, or clang might wish to do it internally.
My current position is that we should pick the path style and slash
direction closest to the point at which the path will be emitted. In the
past we've tried to make arguments that slashes should be internally one
way or another depending on if it's Windows, MinGW, MSVC, or some other
criteria, but the codebase is inconsistent. Now I lean towards
canonicalizing when printing based on the format in use.
- For example, if emitting codeview, try to standardize to backslashes to
match MSVC.
- If emitting DWARF, maybe use forward slashes since the consumer will
probably be a gnu tool.
- When implementing -print-*-path in the driver, the user is probably a
Makefile, so use forward slashes.
- When emitting this YAML stuff, it's our own format, so do whatever is
convenient.
On Wed, Oct 30, 2019 at 4:03 PM Michael Spencer
wrote:
> On Wed, Oct 30, 2019 at 3:54 PM Reid Kleckner via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
>> I XFAILED the test on Windows in 52194350cfe. The [/\\] regexes
>> might need to match two slashes, and the PREFIX variable ends up having
>> inconsistent slash direction.
>>
>
> Hmm, fixing the slashes should be easy, but I'm not sure there's a good
> way to get the PREFIX right. I feel like FileCheck needs an ignore slash
> direction mode.
>
> - Michael Spencer
>
>
>>
>> On Wed, Oct 30, 2019 at 3:27 PM Michael Spencer via cfe-commits <
>> cfe-commits@lists.llvm.org> wrote:
>>
>>>
>>> Author: Michael Spencer
>>> Date: 2019-10-30T15:27:27-07:00
>>> New Revision: 33a745e6fe7e81d3793f7831d2832aa0785ef327
>>>
>>> URL:
>>> https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327
>>> DIFF:
>>> https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327.diff
>>>
>>> LOG: [clang][clang-scan-deps] Add support for extracting full module
>>> dependencies.
>>>
>>> This is a recommit of d8a4ef0e685c with the nondeterminism fixed.
>>>
>>> This adds experimental support for extracting a Clang module dependency
>>> graph
>>> from a compilation database. The output format is experimental and will
>>> change.
>>> It is currently a concatenation of JSON outputs for each compilation.
>>> Future
>>> patches will change this to deduplicate modules between compilations.
>>>
>>> Differential Revision: https://reviews.llvm.org/D69420
>>>
>>> Added:
>>> clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
>>> clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
>>> clang/test/ClangScanDeps/modules-full.cpp
>>>
>>> Modified:
>>>
>>> clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>>>
>>> clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>>>
>>> clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
>>> clang/lib/Tooling/DependencyScanning/CMakeLists.txt
>>> clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp
>>> clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp
>>> clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp
>>> clang/tools/clang-scan-deps/ClangScanDeps.cpp
>>>
>>> Removed:
>>>
>>>
>>>
>>>
>>>
>>> diff --git
>>> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>>> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>>> index fd8ed80b143c..76edf150dbee 100644
>>> ---
>>> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>>> +++
>>> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>>> @@ -30,15 +30,30 @@ enum class ScanningMode {
>>>MinimizedSourcePreprocessing
>>> };
>>>
>>> +/// The format that is output by the dependency scanner.
>>> +enum class ScanningOutputFormat {
>>> + /// This is the Makefile compatible dep format. This will include all
>>> of the
>>> + /// deps necessary for an implicit modules build, but won't include
>>> any
>>> + /// intermodule dependency information.
>>> + Make,
>>> +
>>> + /// This outputs the full module dependency graph suitable for use for
>>> + /// explicitly building modules.
>>> + Full,
>>> +};
>>> +
>>> /// The dependency scanning service contains the shared state that is
>>> used by
>>> /// the invidual dependency scanning workers.
>>> class DependencyScanningService {
>>> public:
>>> - DependencyScanningService(ScanningMode Mode, bool ReuseFileManager =
>>> true,
>>> + DependencyScanningService(ScanningMode Mode, ScanningOutputFormat
>>> Format,
>>> +bool ReuseFileManager = true,
>>> bool SkipExcludedPPRanges = true);
>>>
>>>ScanningMode getMode() const { return Mode; }
>>>
>>> + ScanningOutputFormat getFormat() const { return Format; }
>>> +
>>>bool
Re: [clang] 33a745e - [clang][clang-scan-deps] Add support for extracting full module dependencies.
On Wed, Oct 30, 2019 at 3:54 PM Reid Kleckner via cfe-commits <
cfe-commits@lists.llvm.org> wrote:
> I XFAILED the test on Windows in 52194350cfe. The [/\\] regexes might need
> to match two slashes, and the PREFIX variable ends up having inconsistent
> slash direction.
>
Hmm, fixing the slashes should be easy, but I'm not sure there's a good way
to get the PREFIX right. I feel like FileCheck needs an ignore slash
direction mode.
- Michael Spencer
>
> On Wed, Oct 30, 2019 at 3:27 PM Michael Spencer via cfe-commits <
> cfe-commits@lists.llvm.org> wrote:
>
>>
>> Author: Michael Spencer
>> Date: 2019-10-30T15:27:27-07:00
>> New Revision: 33a745e6fe7e81d3793f7831d2832aa0785ef327
>>
>> URL:
>> https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327
>> DIFF:
>> https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327.diff
>>
>> LOG: [clang][clang-scan-deps] Add support for extracting full module
>> dependencies.
>>
>> This is a recommit of d8a4ef0e685c with the nondeterminism fixed.
>>
>> This adds experimental support for extracting a Clang module dependency
>> graph
>> from a compilation database. The output format is experimental and will
>> change.
>> It is currently a concatenation of JSON outputs for each compilation.
>> Future
>> patches will change this to deduplicate modules between compilations.
>>
>> Differential Revision: https://reviews.llvm.org/D69420
>>
>> Added:
>> clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
>> clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
>> clang/test/ClangScanDeps/modules-full.cpp
>>
>> Modified:
>>
>> clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>>
>> clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>>
>> clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
>> clang/lib/Tooling/DependencyScanning/CMakeLists.txt
>> clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp
>> clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp
>> clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp
>> clang/tools/clang-scan-deps/ClangScanDeps.cpp
>>
>> Removed:
>>
>>
>>
>>
>>
>> diff --git
>> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>> index fd8ed80b143c..76edf150dbee 100644
>> ---
>> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>> +++
>> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
>> @@ -30,15 +30,30 @@ enum class ScanningMode {
>>MinimizedSourcePreprocessing
>> };
>>
>> +/// The format that is output by the dependency scanner.
>> +enum class ScanningOutputFormat {
>> + /// This is the Makefile compatible dep format. This will include all
>> of the
>> + /// deps necessary for an implicit modules build, but won't include any
>> + /// intermodule dependency information.
>> + Make,
>> +
>> + /// This outputs the full module dependency graph suitable for use for
>> + /// explicitly building modules.
>> + Full,
>> +};
>> +
>> /// The dependency scanning service contains the shared state that is
>> used by
>> /// the invidual dependency scanning workers.
>> class DependencyScanningService {
>> public:
>> - DependencyScanningService(ScanningMode Mode, bool ReuseFileManager =
>> true,
>> + DependencyScanningService(ScanningMode Mode, ScanningOutputFormat
>> Format,
>> +bool ReuseFileManager = true,
>> bool SkipExcludedPPRanges = true);
>>
>>ScanningMode getMode() const { return Mode; }
>>
>> + ScanningOutputFormat getFormat() const { return Format; }
>> +
>>bool canReuseFileManager() const { return ReuseFileManager; }
>>
>>bool canSkipExcludedPPRanges() const { return SkipExcludedPPRanges; }
>> @@ -49,6 +64,7 @@ class DependencyScanningService {
>>
>> private:
>>const ScanningMode Mode;
>> + const ScanningOutputFormat Format;
>>const bool ReuseFileManager;
>>/// Set to true to use the preprocessor optimization that skips
>> excluded PP
>>/// ranges by bumping the buffer pointer in the lexer instead of
>> lexing the
>>
>> diff --git
>> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>> index c950cbe167cd..78b49e4fa0c5 100644
>> ---
>> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>> +++
>> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>> @@ -40,6 +40,7 @@ class DependencyScanningTool {
>> StringRef CWD);
>>
>> private:
>> + const ScanningOutputFormat Format;
>>DependencyScanningWorker Worker;
Re: [clang] 33a745e - [clang][clang-scan-deps] Add support for extracting full module dependencies.
I XFAILED the test on Windows in 52194350cfe. The [/\\] regexes might need
to match two slashes, and the PREFIX variable ends up having inconsistent
slash direction.
On Wed, Oct 30, 2019 at 3:27 PM Michael Spencer via cfe-commits <
cfe-commits@lists.llvm.org> wrote:
>
> Author: Michael Spencer
> Date: 2019-10-30T15:27:27-07:00
> New Revision: 33a745e6fe7e81d3793f7831d2832aa0785ef327
>
> URL:
> https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327
> DIFF:
> https://github.com/llvm/llvm-project/commit/33a745e6fe7e81d3793f7831d2832aa0785ef327.diff
>
> LOG: [clang][clang-scan-deps] Add support for extracting full module
> dependencies.
>
> This is a recommit of d8a4ef0e685c with the nondeterminism fixed.
>
> This adds experimental support for extracting a Clang module dependency
> graph
> from a compilation database. The output format is experimental and will
> change.
> It is currently a concatenation of JSON outputs for each compilation.
> Future
> patches will change this to deduplicate modules between compilations.
>
> Differential Revision: https://reviews.llvm.org/D69420
>
> Added:
> clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
> clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
> clang/test/ClangScanDeps/modules-full.cpp
>
> Modified:
>
> clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
> clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
>
> clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
> clang/lib/Tooling/DependencyScanning/CMakeLists.txt
> clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp
> clang/lib/Tooling/DependencyScanning/DependencyScanningTool.cpp
> clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp
> clang/tools/clang-scan-deps/ClangScanDeps.cpp
>
> Removed:
>
>
>
>
>
> diff --git
> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
> index fd8ed80b143c..76edf150dbee 100644
> ---
> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
> +++
> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
> @@ -30,15 +30,30 @@ enum class ScanningMode {
>MinimizedSourcePreprocessing
> };
>
> +/// The format that is output by the dependency scanner.
> +enum class ScanningOutputFormat {
> + /// This is the Makefile compatible dep format. This will include all
> of the
> + /// deps necessary for an implicit modules build, but won't include any
> + /// intermodule dependency information.
> + Make,
> +
> + /// This outputs the full module dependency graph suitable for use for
> + /// explicitly building modules.
> + Full,
> +};
> +
> /// The dependency scanning service contains the shared state that is
> used by
> /// the invidual dependency scanning workers.
> class DependencyScanningService {
> public:
> - DependencyScanningService(ScanningMode Mode, bool ReuseFileManager =
> true,
> + DependencyScanningService(ScanningMode Mode, ScanningOutputFormat
> Format,
> +bool ReuseFileManager = true,
> bool SkipExcludedPPRanges = true);
>
>ScanningMode getMode() const { return Mode; }
>
> + ScanningOutputFormat getFormat() const { return Format; }
> +
>bool canReuseFileManager() const { return ReuseFileManager; }
>
>bool canSkipExcludedPPRanges() const { return SkipExcludedPPRanges; }
> @@ -49,6 +64,7 @@ class DependencyScanningService {
>
> private:
>const ScanningMode Mode;
> + const ScanningOutputFormat Format;
>const bool ReuseFileManager;
>/// Set to true to use the preprocessor optimization that skips
> excluded PP
>/// ranges by bumping the buffer pointer in the lexer instead of lexing
> the
>
> diff --git
> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
> index c950cbe167cd..78b49e4fa0c5 100644
> ---
> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
> +++
> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningTool.h
> @@ -40,6 +40,7 @@ class DependencyScanningTool {
> StringRef CWD);
>
> private:
> + const ScanningOutputFormat Format;
>DependencyScanningWorker Worker;
>const tooling::CompilationDatabase
> };
>
> diff --git
> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
> b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
> index 45c9fb4f029d..689119330c41 100644
> ---
> a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
> +++
>
