[swift-dev] External pass pipeline YAML format
Hi all, Please, let me know if I should post this to another list. I'm trying to figure out what is the expected YAML format of the -external-pass-pipeline-filename option. Dumping the pass pipeline under -O and feeding it back to the compiler with this option doesn't work (from swift-3.1-branch). I thought to ask here because I'm not entirely sure that the relevant YAML parsing code from SILPassPipelinePlan::getPassPipelineFromFile() accepts valid YAML input. Thanks, Vasileios ___ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
Re: [swift-dev] External pass pipeline YAML format
> Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, it looks like it might give some hints? I've tried generating a pass pipeline with utils/pass-pipeline/scripts/pipelines_generator.py and use it with the -external-pass-pipeline-filename. However, AFAICT the generated pipeline is not in the correct format that the compiler expects: [ [ "HighLevel", "run_n_times", 2, "SimplifyCFG", ... "GlobalARCOpts" ], [ "EarlyLoopOpt", "run_n_times", 1, "LowerAggregateInstrs", ... "SwiftArrayOpts" ], ... ] Each generated pass pipeline contains the "run_n_times"|"run_to_fixed_point" field, followed by the number of iterations, which is not what the compiler expects. I had no luck even when I tried to re-format the file containing the pipelines to something that I believe the compiler would expect based on the source code of SILPassPipelinePlan::getPassPipelineFromFile(): [ [ "HihLevel", "SimplifyCFG", ... "GlobalARCOpts" ], ... ] or even: [ [ "HighLevel", [ "SimplifyCFG" ], ... [ "GlobalARCOpts" ] ], ... ] I suspect that we don't use the pass pipeline python scripts in our buildbots anymore and the relevant bits, ie. the code in SILPassPipelinePlan::getPassPipelineFromFile and/or the python scripts, have not been kept up-to-date. On Wed, Jun 28, 2017 at 6:18 PM, Daniel Dunbar wrote: > Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, it > looks like it might give some hints? > -- > $ sgit grep external-pass-pipeline-filename > include/swift/Option/FrontendOptions.td:def external_pass_pipeline_filename > : Separate<["-"], "external-pass-pipeline-filename">, > utils/pass-pipeline/scripts/pipelines_build_script.py: > '-external-pass-pipeline-filename\;-Xfrontend\;%s' % data_file] > -- > > - Daniel > > > On Jun 28, 2017, at 5:27 AM, Vasileios Kalintiris via swift-dev < > swift-dev@swift.org> wrote: > > > > Hi all, > > > > Please, let me know if I should post this to another list. > > > > I'm trying to figure out what is the expected YAML format of the > -external-pass-pipeline-filename option. > > > > Dumping the pass pipeline under -O and feeding it back to the compiler > with this option doesn't work (from swift-3.1-branch). > > > > I thought to ask here because I'm not entirely sure that the relevant > YAML parsing code from SILPassPipelinePlan::getPassPipelineFromFile() > accepts valid YAML input. > > > > Thanks, > > Vasileios > > ___ > > swift-dev mailing list > > swift-dev@swift.org > > https://lists.swift.org/mailman/listinfo/swift-dev > > ___ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
Re: [swift-dev] External pass pipeline YAML format
> I suspect that we don't use the pass pipeline python scripts in our buildbots anymore and the relevant bits, ie. the code in SILPassPipelinePlan::getPassPipelineFromFile and/or the python scripts, have not been kept up-to-date. It seems that the following format works for the time being: [ [ "Pipeline1", [ "PassName1" ], [ "Pipeline2", [ "PassName2" ], [ "Pipeline3", [ "PassName3" ], [ ... ] ] ] ] ] However, it works only when LLVM's assertions are disabled. I'll try to come-up with a patch that will accept a saner format with multiple passes per pipeline entry. Against which branch should I perform a pull request? - Vasileios On Wed, Jun 28, 2017 at 7:15 PM, Vasileios Kalintiris wrote: > > Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, > it looks like it might give some hints? > > I've tried generating a pass pipeline with > utils/pass-pipeline/scripts/pipelines_generator.py > and use it with the -external-pass-pipeline-filename. However, AFAICT the > generated pipeline is not in the correct format that the compiler expects: > > [ > [ > "HighLevel", > "run_n_times", > 2, > "SimplifyCFG", > ... > "GlobalARCOpts" > ], > [ > "EarlyLoopOpt", > "run_n_times", > 1, > "LowerAggregateInstrs", > ... > "SwiftArrayOpts" > ], > ... > ] > > Each generated pass pipeline contains the "run_n_times"|"run_to_fixed_point" > field, followed by the number of iterations, which is not what the compiler > expects. > > I had no luck even when I tried to re-format the file containing the > pipelines to something that I believe the compiler would expect based on > the source code of SILPassPipelinePlan::getPassPipelineFromFile(): > > [ > [ > "HihLevel", > "SimplifyCFG", > ... > "GlobalARCOpts" > ], > ... > ] > > or even: > > [ > [ >"HighLevel", >[ "SimplifyCFG" ], >... >[ "GlobalARCOpts" ] > ], > ... > ] > > I suspect that we don't use the pass pipeline python scripts in our > buildbots anymore and the relevant bits, ie. the code in > SILPassPipelinePlan::getPassPipelineFromFile and/or the python scripts, > have not been kept up-to-date. > > > On Wed, Jun 28, 2017 at 6:18 PM, Daniel Dunbar > wrote: > >> Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, >> it looks like it might give some hints? >> -- >> $ sgit grep external-pass-pipeline-filename >> include/swift/Option/FrontendOptions.td:def >> external_pass_pipeline_filename : Separate<["-"], >> "external-pass-pipeline-filename">, >> utils/pass-pipeline/scripts/pipelines_build_script.py: >> '-external-pass-pipeline-filename\;-Xfrontend\;%s' % data_file] >> -- >> >> - Daniel >> >> > On Jun 28, 2017, at 5:27 AM, Vasileios Kalintiris via swift-dev < >> swift-dev@swift.org> wrote: >> > >> > Hi all, >> > >> > Please, let me know if I should post this to another list. >> > >> > I'm trying to figure out what is the expected YAML format of the >> -external-pass-pipeline-filename option. >> > >> > Dumping the pass pipeline under -O and feeding it back to the compiler >> with this option doesn't work (from swift-3.1-branch). >> > >> > I thought to ask here because I'm not entirely sure that the relevant >> YAML parsing code from SILPassPipelinePlan::getPassPipelineFromFile() >> accepts valid YAML input. >> > >> > Thanks, >> > Vasileios >> > ___ >> > swift-dev mailing list >> > swift-dev@swift.org >> > https://lists.swift.org/mailman/listinfo/swift-dev >> >> > ___ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
Re: [swift-dev] External pass pipeline YAML format
I plan on writing a patch for this. Based on which branch should I create the pull request? Is the swift-3.1-branch frozen? On Thu, Jun 29, 2017 at 7:27 PM, Michael Gottesman wrote: > > On Jun 28, 2017, at 10:15 AM, Vasileios Kalintiris via swift-dev < > swift-dev@swift.org> wrote: > > > Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, > it looks like it might give some hints? > > I've tried generating a pass pipeline with > utils/pass-pipeline/scripts/pipelines_generator.py > and use it with the -external-pass-pipeline-filename. However, AFAICT the > generated pipeline is not in the correct format that the compiler expects: > > [ > [ > "HighLevel", > "run_n_times", > 2, > "SimplifyCFG", > ... > "GlobalARCOpts" > ], > [ > "EarlyLoopOpt", > "run_n_times", > 1, > "LowerAggregateInstrs", > ... > "SwiftArrayOpts" > ], > ... > ] > > Each generated pass pipeline contains the "run_n_times"|"run_to_fixed_point" > field, followed by the number of iterations, which is not what the compiler > expects. > > I had no luck even when I tried to re-format the file containing the > pipelines to something that I believe the compiler would expect based on > the source code of SILPassPipelinePlan::getPassPipelineFromFile(): > > [ > [ > "HihLevel", > "SimplifyCFG", > ... > "GlobalARCOpts" > ], > ... > ] > > or even: > > [ > [ >"HighLevel", >[ "SimplifyCFG" ], >... >[ "GlobalARCOpts" ] > ], > ... > ] > > I suspect that we don't use the pass pipeline python scripts in our > buildbots anymore and the relevant bits, ie. the code in > SILPassPipelinePlan::getPassPipelineFromFile and/or the python scripts, > have not been kept up-to-date. > > > Yes. I think that is true. Here is what I would suggest: > > 1. It would be really trivial to change this to use yamltraits. > 2. If you make this change, make sure that a test is added (maybe to > sil-opt) that makes sure that we can roundtrip from the dumper. > > MIchael > > > > On Wed, Jun 28, 2017 at 6:18 PM, Daniel Dunbar > wrote: > >> Have you seen `utils/pass-pipeline/scripts/pipelines_build_script.py`, >> it looks like it might give some hints? >> -- >> $ sgit grep external-pass-pipeline-filename >> include/swift/Option/FrontendOptions.td:def >> external_pass_pipeline_filename : Separate<["-"], >> "external-pass-pipeline-filename">, >> utils/pass-pipeline/scripts/pipelines_build_script.py: >> '-external-pass-pipeline-filename\;-Xfrontend\;%s' % data_file] >> -- >> >> - Daniel >> >> > On Jun 28, 2017, at 5:27 AM, Vasileios Kalintiris via swift-dev < >> swift-dev@swift.org> wrote: >> > >> > Hi all, >> > >> > Please, let me know if I should post this to another list. >> > >> > I'm trying to figure out what is the expected YAML format of the >> -external-pass-pipeline-filename option. >> > >> > Dumping the pass pipeline under -O and feeding it back to the compiler >> with this option doesn't work (from swift-3.1-branch). >> > >> > I thought to ask here because I'm not entirely sure that the relevant >> YAML parsing code from SILPassPipelinePlan::getPassPipelineFromFile() >> accepts valid YAML input. >> > >> > Thanks, >> > Vasileios >> > ___ >> > swift-dev mailing list >> > swift-dev@swift.org >> > https://lists.swift.org/mailman/listinfo/swift-dev >> >> > ___ > swift-dev mailing list > swift-dev@swift.org > https://lists.swift.org/mailman/listinfo/swift-dev > > > ___ swift-dev mailing list swift-dev@swift.org https://lists.swift.org/mailman/listinfo/swift-dev
