Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title John Duarte commented on PUP-9726 Re: Refactor long running acceptance tests to exectute in less than 45 seconds Of the tests that are tagged as "audit:high" only "acceptance/tests/resource/file/symbolic_modes.rb" is not also tagged as a "server" test. Given this context, I agree Josh Cooper that the "server" tests do not warrant refactoring. I would say that reduces the scope of this effort to the point that it is not worth pursuing. Therefore, I am closing this issue. Add Comment This message was sent by Atlassian Jira (v8.5.2#805002-sha1:a66f935) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.310800.1559242752000.313.1582912800113%40Atlassian.JIRA.
Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title Josh Cooper commented on PUP-9726 Re: Refactor long running acceptance tests to exectute in less than 45 seconds The environment, ssl, utf8, and source attribute tests are still very valid and important. I don't mind that they don't complete in less than 45 seconds. The "cve-2013-4761_injection_of_class_names_loading_code" could be modified to not start puppetserver twice. Also puppet acceptance is no longer the slowest part of the pipeline (pxp now is). It might be good for us to resync with Mihai Buzgau to see what the slowest parts of the pipeline are. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.310800.1559242752000.1415.1582761060727%40Atlassian.JIRA.
Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title John Duarte commented on PUP-9726 Re: Refactor long running acceptance tests to exectute in less than 45 seconds PR 8004 opened for symbolic_modes test. Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.310800.1559242752000.1361.1582759860362%40Atlassian.JIRA.
Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title John Duarte assigned an issue to John Duarte Puppet / PUP-9726 Refactor long running acceptance tests to exectute in less than 45 seconds Change By: John Duarte Assignee: John Duarte Add Comment This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-bugs/JIRA.310800.1559242752000.1337.1582759380114%40Atlassian.JIRA.
Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title John Duarte commented on PUP-9726 Re: Refactor long running acceptance tests to exectute in less than 45 seconds Of the above list, only the following have a risk tag of "audit:high". This ticket should be re-scoped to only include the tests below. acceptance/tests/environment/agent_runs_pluginsync_with_proper_environment.rb acceptance/tests/security/cve-2013-4761_injection_of_class_names_loading_code.rb acceptance/tests/ssl/autosign_command.rb acceptance/tests/ssl/certificate_extensions.rb acceptance/tests/utf8/utf8-in-catalog.rb acceptance/tests/resource/file/source_attribute.rb acceptance/tests/resource/file/symbolic_modes.rb Add Comment
Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title
John Duarte updated an issue
Puppet / PUP-9726
Refactor long running acceptance tests to exectute in less than 45 seconds
Change By:
John Duarte
*As a* puppet tester *I want* to keep the execution time of individual acceptance tests under 45 seconds *So that* I can get test feedback fasterAcceptance tests are expensive.The following acceptance tests take longer than 45 seconds to execute on the reference RHEL 7 platform. These tests should be refactored to reduce there execution time below 45 seconds (preferably below 30 seconds). Total execution time for the tests in the sample below was 3506.51 seconds. If all of these tests were successfully executed in 30 secs each, then the overall test suite execution time would be reduced by about 30% (on RHEL 7 as of 2019.05.31). Tests that take longer than 45 seconds to run on RHEL7 along with there execution time:{noformat}acceptance/tests/direct_puppet/cached_catalog_remediate_local_drift.rb 52.35acceptance/tests/direct_puppet/catalog_uuid_correlates_catalogs_with_reports.rb 55.81acceptance/tests/direct_puppet/static_catalog_env_control.rb 114.42acceptance/tests/environment/agent_runs_pluginsync_with_proper_environment.rb 55.61acceptance/tests/environment/custom_type_provider_from_same_environment.rb 47.64acceptance/tests/environment/enc_nonexistent_directory_environment.rb 58.88acceptance/tests/environment/environment_scenario-bad.rb 106.58acceptance/tests/environment/use_agent_environment_when_enc_doesnt_specify.rb 88.39acceptance/tests/environment/use_agent_environment_when_no_enc.rb 81.42acceptance/tests/environment/use_enc_environment.rb 76.63acceptance/tests/environment/use_enc_environment_for_files.rb 46.97acceptance/tests/environment/use_enc_environment_for_pluginsync.rb 67.49acceptance/tests/environment/use_environment_from_environmentpath.rb 71.33acceptance/tests/language/exported_resources.rb 48.25acceptance/tests/language/pcore_generate_env_isolation.rb 63.53acceptance/tests/loader/autoload_from_resource_type_decl.rb 64.57acceptance/tests/lookup/lookup.rb 57.79acceptance/tests/lookup/lookup_rich_values.rb 56.28acceptance/tests/lookup/merge_strategies.rb 78.82acceptance/tests/lookup/v3_config_and_data.rb 68.38acceptance/tests/ordering/master_agent_application.rb 53.52acceptance/tests/parser_functions/puppet_lookup_cmd.rb 171.06acceptance/tests/pluginsync/4420_pluginfacts_should_be_resolvable_on_agent.rb 81.22acceptance/tests/reports/cached_catalog_status_in_report.rb 66.33acceptance/tests/reports/corrective_change_outside_puppet.rb 82.52acceptance/tests/reports/corrective_change_via_puppet.rb 81.63acceptance/tests/reports/failover_master.rb 118.09acceptance/tests/reports/submission.rb 79.67acceptance/tests/security/cve-2013-4761_injection_of_class_names_loading_code.rb 54.48acceptance/tests/ssl/autosign_command.rb 116.72acceptance/tests/ssl/certificate_extensions.rb 178.28acceptance/tests/utf8/utf8-in-catalog.rb 158.18acceptance/tests/utf8/utf8-in-function-args.rb 54.22acceptance/tests/i18n/modules/puppet_agent.rb
Jira (PUP-9726) Refactor long running acceptance tests to exectute in less than 45 seconds
Title: Message Title John Duarte created an issue Puppet / PUP-9726 Refactor long running acceptance tests to exectute in less than 45 seconds Issue Type: Task Assignee: Unassigned Created: 2019/05/30 11:59 AM Priority: Normal Reporter: John Duarte As a puppet tester I want to keep the execution time of individual acceptance tests under 45 seconds So that I can get test feedback faster Acceptance tests are expensive. The following acceptance tests take longer than 45 seconds to execute on the reference RHEL 7 platform. These tests should be refactored to reduce there execution time below 45 seconds (preferably below 30 seconds). Tests that take longer than 45 seconds to run on RHEL7 along with there execution time: acceptance/tests/direct_puppet/cached_catalog_remediate_local_drift.rb 52.35 acceptance/tests/direct_puppet/catalog_uuid_correlates_catalogs_with_reports.rb 55.81 acceptance/tests/direct_puppet/static_catalog_env_control.rb 114.42 acceptance/tests/environment/agent_runs_pluginsync_with_proper_environment.rb 55.61
