Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: c7920bb4d79695c39c594708667389641397bb60
https://github.com/WebKit/WebKit/commit/c7920bb4d79695c39c594708667389641397bb60
Author: Elliott Williams <[email protected]>
Date: 2026-09-03 (Thu, 03 Sep 2026)
Changed paths:
M CMakeLists.txt
M Source/cmake/OptionsCommon.cmake
A Tools/Scripts/swift/__init__.py
A Tools/Scripts/swift/ninja_build_trace.py
A Tools/Scripts/swift/ninja_build_trace_unittest.py
A Tools/Scripts/swift/swiftc_job_recorder.py
M Tools/Scripts/webkitpy/test/main.py
Log Message:
-----------
[CMake] Add swiftc+ninja tracing tool
https://bugs.webkit.org/show_bug.cgi?id=322859
rdar://186102248
Reviewed by Geoffrey Garen.
In CMake builds, we compile each Swift target as a single invocation of
the driver. This means that ninja sees one long task for e.g.
`WebKit.swiftmodule`, which in turn is a driver invocation that performs
many parallel compilations. It's useful to visualize how the work breaks
down to measure oversubscription and check for duplicated work such as
module cache invalidations.
Add a `ninja_build_trace.py` tool to the tree, which reads ninja's log
format and combines it with information from an instrumented swift
invocation (`swiftc_job_recorder.py`). Configuring the build with
`-DSWIFT_NINJA_TRACE=ON` will build with the instrumentation enabled,
and a final step analyzes the data and produces a trace that can be
dropped into Perfetto.
Workflow looks like:
cmake --preset mac-release -DSWIFT_NINJA_TRACE=ON
cmake --build --preset mac-release --clean-first
WebKitBuild/cmake-mac/Release/swift-trace-finalize.sh -o trace.json
Add a suite of unit tests for the tracing code, and add
`Tools/Scripts/swift` to the testwebkitpy tree.
* CMakeLists.txt:
* Source/cmake/OptionsCommon.cmake:
* Tools/Scripts/swift/__init__.py: Added.
* Tools/Scripts/swift/ninja_build_trace.py: Added.
(NinjaEdge):
(NinjaEdge.dur_ms):
(NinjaEdge.anchor_ms):
(NinjaEdge.label):
(parse_ninja_log):
(NinjaRun):
(NinjaRun.spread_ms):
(NinjaRun.window_ms):
(group_runs):
(FrontendJob):
(FrontendJob.end_us):
(FrontendJob.label):
(load_stats_dir):
(_find_wall_usec):
(RecordedJob):
(RecordedJob.dur_us):
(RecordedJob.label):
(RecordedJob.builds_a_dependency):
(DriverInvocation):
(DriverInvocation.outputs):
(load_jobs_log):
(_recorded_job):
(attribute_invocations):
(anchor_from_invocations):
(refine_anchor):
(_merge_spans):
(is_swift_edge):
(_match_score):
(attribute):
(within_run):
(partition_within_run):
(Span):
(Span.end_us):
(_stats_args):
(_output_names):
(spans_for_edge):
(pack_lanes):
(TidAllocator):
(TidAllocator.__init__):
(TidAllocator.assign):
(Row):
(_edge_span):
(build_rows):
(process_spans):
(emit_trace):
(build_trace):
(validate_trace):
(_metadata):
(spawning_edge_ids):
(_load_transitions):
(_counter_events):
(_counter):
(summarize):
(main):
(merge_runs):
(anchor_is_plausible):
(select_run):
(_select_run):
(_select_run.contained):
(NinjaEdge.anchor_bounds_ms):
(clean_name):
(collapse_batches):
(_whole_microseconds):
* Tools/Scripts/swift/ninja_build_trace_unittest.py: Added.
(write_stats):
(ninja_log):
(write_jobs_log):
(framed):
(make_job):
(ScratchDirectoryTest):
(ScratchDirectoryTest.setUp):
(NinjaLogParsingTest):
(NinjaLogParsingTest.test_multi_output_records_fold_into_one_edge):
(NinjaLogParsingTest.test_header_and_malformed_lines_are_skipped):
(NinjaLogParsingTest.test_edge_anchor_uses_freshest_output_mtime):
(RunSplittingTest):
(RunSplittingTest.test_runs_are_split_by_anchor_gap):
(RunSplittingTest.test_undated_edges_are_held_back_from_every_run):
(RunSplittingTest.test_merge_runs_rebases_each_invocation_to_its_own_wall_time):
(StatsLoadingTest):
(StatsLoadingTest.test_load_stats_dir_reads_start_and_duration):
(StatsLoadingTest.test_whole_module_job_is_labelled_by_module):
(StatsLoadingTest.test_missing_wall_timer_falls_back_to_one_microsecond):
(StatsLoadingTest.test_non_stats_files_are_ignored):
(ClockAlignmentTest):
(ClockAlignmentTest.test_refine_anchor_recovers_a_shifted_anchor):
(ClockAlignmentTest.test_refine_anchor_corrects_a_stale_restat_mtime):
(ClockAlignmentTest.test_refine_anchor_is_a_noop_without_data):
(ClockAlignmentTest.test_jobs_outside_the_run_window_are_separated):
(AttributionTest):
(AttributionTest.test_attribute_prefers_the_edge_naming_the_module):
(AttributionTest.test_attribute_reports_jobs_with_no_containing_edge):
(AttributionTest.test_identical_sources_in_several_targets_go_to_their_own_edges):
(AttributionTest.test_a_containing_clang_edge_does_not_claim_a_swift_job):
(AttributionTest.test_sibling_swift_edges_do_not_pool_two_modules):
(AttributionTest.test_whole_module_job_matches_its_swiftmodule_edge):
(AttributionTest.test_is_swift_edge_recognizes_swift_outputs):
(RecordedJobsTest):
(RecordedJobsTest.test_load_jobs_log_pairs_began_with_finished):
(RecordedJobsTest.test_an_interrupted_recorder_keeps_its_span_and_in_flight_jobs):
(RecordedJobsTest.test_load_jobs_log_separates_concurrent_recorders):
(RecordedJobsTest.test_load_jobs_log_survives_a_torn_line):
(RecordedJobsTest.test_anchor_from_invocations_ignores_a_misleading_mtime):
(RecordedJobsTest.test_anchor_from_invocations_reports_nothing_without_a_match):
(RecordedJobsTest.test_attribute_invocations_matches_on_resolved_output_paths):
(RecordedJobsTest.test_recorded_and_stats_views_of_one_process_do_not_double_count):
(RecordedJobsTest.test_the_in_process_dependency_scan_is_kept):
(RecordedJobsTest.test_module_jobs_reach_the_trace_and_count_as_load):
(TraceEmissionTest):
(TraceEmissionTest.test_pack_lanes_reuses_a_lane_once_it_is_free):
(TraceEmissionTest.test_pack_lanes_handles_fully_concurrent_spans):
(TraceEmissionTest.test_edges_that_spawned_frontends_are_not_counted_twice):
(TraceEmissionTest.test_counter_track_returns_to_zero):
(TraceEmissionTest.test_rows_for_the_same_module_are_disambiguated_by_edge):
(TraceEmissionTest.test_no_tid_doubles_as_a_pid):
(TraceEmissionTest.test_validate_trace_catches_a_shared_track):
(TraceEmissionTest.test_validate_trace_catches_a_pid_used_as_a_tid):
(TraceEmissionTest.test_validate_trace_passes_a_clean_trace):
(TraceEmissionTest.test_no_tid_is_shared_by_two_processes):
(TraceEmissionTest.test_lanes_get_thread_names):
(TraceEmissionTest.test_trace_shape_is_a_valid_catapult_object):
(TraceEmissionTest.test_relative_and_absolute_timestamps_differ_by_the_anchor):
(MainTest):
(MainTest.test_main_writes_a_trace):
(MainTest.test_main_errors_on_an_empty_log):
(RecorderTest):
(RecorderTest.test_recorder_consumes_its_own_flags):
(RecorderTest.test_recorder_defaults_to_swiftc_on_path):
(RecorderTest.test_recorder_reads_framed_messages_and_passes_text_through):
(RecorderTest.test_recorder_first_output_reads_the_path):
(NinjaLogParsingTest.test_edge_anchor_subtracts_the_start_not_the_end):
(RunSplittingTest.test_a_restat_edge_stays_with_its_own_invocation):
(AttributionTest.test_attribute_matches_a_source_whose_name_swift_sanitized):
(RecordedJobsTest.test_batch_constituents_collapse_into_one_process):
(RecordedJobsTest.test_batch_constituents_collapse_into_one_process.began):
(RecordedJobsTest.test_a_sanitized_stats_name_still_merges_with_the_recorded_job):
(TraceEmissionTest.test_emitted_timestamps_are_whole_microseconds):
(RecorderTest.test_batch_key_names_the_process_a_quasi_pid_job_ran_in):
* Tools/Scripts/swift/swiftc_job_recorder.py: Added.
(log_line):
(first_output):
(read_messages):
(parse_args):
(main):
(main.on_message):
(main.on_text):
(batch_key):
* Tools/Scripts/webkitpy/test/main.py:
(main):
Canonical link: https://commits.webkit.org/320460@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
