The GitHub Actions job "CMake" on subversion.git/trunk has succeeded. Run started by GitHub user asf-gitbox-commits (triggered by asf-gitbox-commits).
Head commit for run: 85707b2acbe4dac50d1c5e0ee72022c1ea5363ad / Timofei Zhakov <[email protected]> Switch command-line programs to UTF-8 mode. Merging from branch utf8-cmdline. The Subversion library internally works with UTF-8 encoded strings which simplifies it a lot. However, commands-line programs receive arguments in locale encoding (cstring). Before this change, we used to convert encoding as we handle each individual argument. But this approach has several limitations; 1) It's easy to miss some conversions since we need to constantly keep it in mind that the arguments are in non-UTF-8 encoding, unlike the majority of codebase assumes. Let's keep it consistent and simple. 2) Even though we could perfectly handle Non-ASCII (Unicode) strings in UTF-8 encoding under the hood, cstring squishes them to allow ASCII-only data. Switching command-line to UTF-8 would remove that layer of converting input data back and forth, which means direct conversions from UTF-16 to UTF-8 without any complications. Recently, we added a family of svn_cmdline__get_cstring_argv() function to resolve an issue with inconsistent encoding of what we assumed was cstring. Now, it's time to get into UTF-8, so we are introducing svn_cmdline__get_utf8_argv() for that. It uses svn_utf__win32_utf16_to_utf8() on Windows, and svn_utf_cstring_to_utf8() on Unix. This branch should not bring any behaviour changes on Unix, while Windows code now supports Unicode command-line arguments. * subversion/include/private/svn_cmdline_private.h (svn_cmdline__win32_get_utf8_argv, svn_cmdline__default_get_utf8_argv): Declare methods. (svn_cmdline__get_utf8_argv): New define. * subversion/include/svn_client.h (svn_client_args_to_target_array3): Declare new functions that do not perform UTF-8 conversion unlike its old variant. (svn_client_args_to_target_array2): Deprecate. * subversion/include/svn_opt.h (svn_opt_args_to_target_array4, svn_opt_parse_revprop2): Declare new functions that do not perform UTF-8 conversion unlike their old variant. (svn_opt_args_to_target_array3, svn_opt_parse_revprop): Deprecate. * subversion/libsvn_client/client.h (svn_client__process_target_array): Declare new private function to put common code of target collection to later use in either UTF8 or non-UTF8 versions. * subversion/libsvn_client/cmdline.c (svn_client__process_target_array): Implement function, using body of former svn_client_args_to_target_array2() function. (svn_client_args_to_target_array3): Implement by forwarding targets to svn_client__process_target_array() as they are without any conversions. * subversion/libsvn_client/deprecated.c (svn_client_args_to_target_array2): Implement backward compatibility function in a similar way as svn_client_args_to_target_array3(), but also converting targets to UTF8 encoding. * subversion/libsvn_subr/cmdline.c (svn_cmdline__win32_get_utf8_argv): New function that receives a collection of argv in locale Windows encoding and converts them to UTF8 using svn_utf__win32_utf16_to_utf8. (svn_cmdline__default_get_utf8_argv): New function that ensures the exact same behaviour in terms of encoding conversions performed as our command-line code used to do it before on Unix platforms since it is backed by svn_utf_cstring_to_utf8(). * subversion/libsvn_subr/deprecated.c (svn_opt_args_to_target_array3): Use svn_opt_parse_all_args() method to back the implementation. (svn_opt_parse_revprop): Implement wrapper over svn_opt_parse_revprop2 with UTF8 conversion. * subversion/libsvn_subr/opt.c (svn_opt_parse_revprop2): Bump version and remove UTF8 conversion. * subversion/svn/auth-cmd.c * subversion/svn/propdel-cmd.c * subversion/svn/propedit-cmd.c * subversion/svn/propget-cmd.c * subversion/svn/propset-cmd.c * subversion/svn/shelf-cmd.c * subversion/svn/shelf2-cmd.c (svn_cl__auth, svn_cl__changelist, svn_cl__propdel, svn_cl__propedit, svn_cl__propget, svn_cl__propedit, svn_cl__propget, svn_cl__propset, get_next_argument): Remove conversion to UTF8 encoding of target arguments. * subversion/svn/svn.c (parse_compatible_version): Do not convert the encoding of value of the opt_compatible_version argument. (sub_main): Use svn_cmdline__get_utf8_argv instead of svn_cmdline__get_cstring_argv to get normalized arguments. (sub_main): Read opt_arg directly to utf8_opt_arg and remove all conversion from cstring to UTF8. (sub_main): For some options which want cstring, convert utf8_opt_arg back. Those include --diff-cmd, --merge-cmd, and --editor-cmd. (sub_main): Use a newer version of svn_opt_parse_revprop, svn_opt_parse_revprop which does not convert strings to UTF8. (sub_main): Do not utf8-ize first_arg. (sub_main): Do not convert the message to UTF8. * subversion/svn/util.c subversion/svn/util.c (svn_cl__make_log_msg_baton): Consider --encoding argument only when the message is read from a file, keeping it in UTF8 when --message is utilised instead. (svn_cl__args_to_target_array_print_reserved): Use svn_client_args_to_target_array3 instead of svn_client_args_to_target_array2, which doesn't convert result to utf8, since we are working with already converted 'os'. * subversion/svnadmin/svnadmin.c (parse_args): No need to command-line arguments to UTF8. (set_revprop): Explicitly state UTF8 encoding when invoking the svn_subst_translate_string2() since the arguments should already be in this encoding. This ensures that no double conversion would be done. (subcommand_lslocks): Use svn_opt_args_to_target_array4() instead of svn_opt_args_to_target_array3() to retrieve targets which does not convert encodings. (sub_main): Use svn_cmdline__get_utf8_argv() instead of svn_cmdline__get_cstring_argv() to get UTF8 argv. (sub_main): Read opt_arg directly to utf8_opt_arg and remove all conversion from cstring to UTF8. (sub_main): Do not utf8-ize first_arg. * subversion/svnbench/svnbench.c (sub_main): Convert args straight to UTF8, remove in-place conversions, and silently migrate to handle the other arguments as UTF8. * subversion/svnbench/null-list-cmd.c (includes): Remove svn_utf.h to make sure no more UTF8 conversions are done. * subversion/svnbench/util.c (svn_cl__args_to_target_array_print_reserved): Use newer version of svn_client_args_to_target_array(), which doesn't perform UTF8 conversion (sub_main): Directly convert argv to UTF8 and remove all following conversions. * subversion/svnlook/svnlook.c (print_diff_tree): Don't convert diff_options to UTF8 twice. (sub_main): Convert args straight to UTF8, remove in-place conversions, and handle the other arguments as UTF8. * subversion/svnfsfs/svnfsfs.c * subversion/svnmucc/svnmucc.c * subversion/svndumpfilter/svndumpfilter.c * subversion/svnrdump/svnrdump.c * subversion/svnserve/svnserve.c * subversion/svnversion/svnversion.c * tools/client-side/svn-mergeinfo-normalizer/svn-mergeinfo-normalizer.c * tools/client-side/svnconflict/svnconflict.c * tools/dev/svnraisetreeconflict/svnraisetreeconflict.c * tools/dev/wc-ng/svn-wc-db-tester.c * tools/server-side/svnauthz.c (includes): Remove svn_utf8.h where possible as we no longer use its functions. (sub_main): Convert all args straight to UTF8 and remove later conversion. * subversion/svnsync/svnsync.c (initialize_cmd, synchronize_cmd, copy_revprops_cmd, info_cmd): Use svn_opt_args_to_target_array4() instead of svn_opt_args_to_target_array3() and remove svn_utf_cstring_to_utf8() calls. (sub_main): Convert args straight to UTF8, remove in-place conversions, and handle the other arguments as UTF8. * subversion/tests/cmdline/svntest/testcase.py (_detect_utf8_locale): New local helper. (RequireUtf8_deco): Add RequireUtf8 decorator that allows test to be run in UTF8 environment to fix failures on Unix platforms. * subversion/tests/cmdline/basic_tests.py (): Import RequireUtf8 decorator from testcase.py (argv_with_best_fit_chars): Adapt the unit test to the arguments in utf-8 bytes. (unicode_arguments_test): Use that RequireUtf8 decorator to force the test to run in the UTF8 environment. * subversion/tests/libsvn_subr/opt-test.c (includes): Add svn_hash.h. (test_svn_opt_parse_revprop): Add new test. (test_funcs): Run the test. Reviewed by: ivan git-svn-id: https://svn.apache.org/repos/asf/subversion/trunk@1935602 13f79535-47bb-0310-9956-ffa450edef68 Report URL: https://github.com/apache/subversion/actions/runs/28049475345 With regards, GitHub Actions via GitBox
