Author: kotkov
Date: Sat Feb 13 14:30:10 2021
New Revision: 1886490
URL: http://svn.apache.org/viewvc?rev=1886490&view=rev
Log:
In the update editor, stream data to both pristine and working files.
Previously, the working files were installed in a separate step. That step
happened per-directory and involved copying and possibly translating the
pristine contents into new working files. For transports like HTTP having
a separate step with nothing being read from the connection puts an implicit
limit on the size of the directory that can be checked out without hitting
a timeout.
Assuming the default HTTP timeout = 60 seconds of httpd 2.4.x and a relatively
fast disk, the limit was around 6 GB for any directory.
If we stream data to both pristine and the (projected) working file, the
actual install can happen as an atomic rename. Then we don't have to stop
reading from the connection, and the timeouts should no longer be an issue.
The new approach also has several nice properties, such as not having to
re-read the pristine files, not interfering with the network-level buffering,
TCP slow starts, and etc. The implementation of the new approach gives up to
25% reduction in the amount of write I/O during checkouts. On Windows, it
uses the existing mechanism of installing the files by handle, without having
to reopen them.
Several quick benchmarks:
- Checking out subversion/trunk over https://
Total time: 3.861 s → 2.812 s
Read IO: 57322 KB → 316 KB
Write IO: 455013 KB → 359977 KB
- Checking out 4 large binary files (7.4 GB) over https://
Total time: 91.594 s → 70.214 s
Read IO: 7798883 KB → 19 KB
Write IO: 15598167 KB → 15598005 KB
>From the implementation standpoint, we factor out a utility layer that can
prepare and install working files, svn_wc__working_file_writer_t, and use it
in the part of the workqueue that handles OP_FILE_INSTALL. We then rework
the update editor to make it stream the data to the working file during
apply_textdelta(), by using the introduced file writer utility layer.
* subversion/include/private/svn_io_private.h
(svn_stream__install_stream_set_read_only,
svn_stream__install_stream_set_executable,
svn_stream__install_stream_set_affected_time,
svn_io__win_set_file_basic_info): Declare.
* subversion/libsvn_subr/io.c
(FILE_BASIC_INFO, FileBasicInfo): Declare these Win32 types for older SDKs.
(APR_DELTA_EPOCH_IN_USEC): Unless defined, declare a copy of the constant
from apr/arch/win32/apr_arch_atime.h.
(svn_io__win_set_file_basic_info): New function that sets the basic file
information using an existing file handle.
* subversion/libsvn_subr/stream.c
(install_baton_t): Add new fields to remember if the target of the install
stream should have specific mtime, read-only attribute or the executable
bit.
(svn_stream__create_for_install): Initialize the new fields to their
default values.
(svn_stream__install_stream_set_read_only,
svn_stream__install_stream_set_executable,
svn_stream__install_stream_set_affected_time): Update the new fields.
(svn_stream__install_stream): Adjust the expected mtime and attributes of
the to-be-installed file, if necessary.
(svn_stream__install_get_info): If we are to set a specific mtime on the
target, return it. Otherwise, stat the file as before.
* subversion/libsvn_wc/working_file_writer.h: New file.
(svn_wc__working_file_writer_t,
svn_wc__working_file_writer_open,
svn_wc__working_file_writer_get_stream,
svn_wc__working_file_writer_get_info,
svn_wc__working_file_writer_install,
svn_wc__working_file_writer_close): Declare the new utility layer.
* subversion/libsvn_wc/working_file_writer.c: New file.
(svn_wc__working_file_writer_t,
cleanup_file_writer,
svn_wc__working_file_writer_open,
svn_wc__working_file_writer_get_stream,
svn_wc__working_file_writer_get_info,
svn_wc__working_file_writer_install,
svn_wc__working_file_writer_close): Implement the new utility layer.
Make use of the svn_stream__install_stream's new functionality.
* subversion/libsvn_wc/workqueue.c
(): Include working_file_writer.h.
(run_file_install): Reimplement by creating and delegating most of the
work to the working file writer.
* subversion/libsvn_wc/wc_db.c
(): Include working_file_writer.h.
(svn_wc__db_base_add_file): Accept an optional working file writer.
If it's passed in, this function will atomically update the db and complete
the installation of the file. Accept the new "record_fileinfo" parameter.
If it's set, also record the fileinfo of the installed file.
* subversion/libsvn_wc/wc_db.c
(db_record_fileinfo): Move this function so that it can be used without
a forward declaration.
(install_working_file): New helper. Completes an installation of the
working file and optionally records its fileinfo.
(svn_wc__db_base_add_file): If the working file writer is passed in,
atomically update the db and complete the installation of the file.
If the "record_fileinfo" parameter is set, also record the fileinfo
of the installed file.
* subversion/libsvn_wc/update_editor.c
(): Include working_file_writer.h.
(struct file_baton): Add new fields to hold the cached base properties
of the file and an optional working file writer.
(window_handler): In case of an error, explicitly close the working file
writer.
(get_file_base_props): New helper, mostly factored out from the
close_file() function. Returns the, possibly cached, base properties
for a file context.
(open_working_file_writer): New helper. Opens a working file writer
according to the properties in the specified file context.
(lazy_open_target): If necessary, open the working file writer.
Configure the incoming data to be written to both the pristine and
the provisioned working file. Introduce the local variable "stream"
and rename the output argument to "stream_p" for clarity.
(close_file): If we need to install the working file, ensure that we have
an appropriate working file writer. The writer may have been provisioned
during apply_textdelta(), but also may need to be opened on-the-go.
Pass the file writer to svn_wc__db_base_add_file() to complete the
installation of the working file. Don't schedule OP_FILE_INSTALL
workqueue items.
* subversion/tests/libsvn_subr/io-test.c
(test_install_stream_set_read_only,
test_install_stream_set_affected_time): New tests.
(test_funcs): Run the new tests.
* subversion/tests/libsvn_wc/db-test.c
(test_inserting_nodes): Adjust the call to svn_wc__db_base_add_file().
* subversion/tests/cmdline/update_tests.py
(update_add_missing_local_add): No longer fails. Add comment.
Added:
subversion/trunk/subversion/libsvn_wc/working_file_writer.c (with props)
subversion/trunk/subversion/libsvn_wc/working_file_writer.h (with props)
Modified:
subversion/trunk/subversion/include/private/svn_io_private.h
subversion/trunk/subversion/libsvn_subr/io.c
subversion/trunk/subversion/libsvn_subr/stream.c
subversion/trunk/subversion/libsvn_wc/update_editor.c
subversion/trunk/subversion/libsvn_wc/wc_db.c
subversion/trunk/subversion/libsvn_wc/wc_db.h
subversion/trunk/subversion/libsvn_wc/workqueue.c
subversion/trunk/subversion/tests/cmdline/update_tests.py
subversion/trunk/subversion/tests/libsvn_subr/io-test.c
subversion/trunk/subversion/tests/libsvn_wc/db-test.c
Modified: subversion/trunk/subversion/include/private/svn_io_private.h
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/include/private/svn_io_private.h?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/include/private/svn_io_private.h (original)
+++ subversion/trunk/subversion/include/private/svn_io_private.h Sat Feb 13
14:30:10 2021
@@ -102,6 +102,24 @@ svn_stream__create_for_install(svn_strea
apr_pool_t *result_pool,
apr_pool_t *scratch_pool);
+/* Configure value of the read-only attribute that will be set when
+ the stream is installed. */
+void
+svn_stream__install_stream_set_read_only(svn_stream_t *install_stream,
+ svn_boolean_t read_only);
+
+/* Configure value of the executable bit that will be set when the
+ stream is installed. */
+void
+svn_stream__install_stream_set_executable(svn_stream_t *install_stream,
+ svn_boolean_t executable);
+
+/* Configure value of the last modification time that will be set
+ when the stream is installed. */
+void
+svn_stream__install_stream_set_affected_time(svn_stream_t *install_stream,
+ apr_time_t mtime);
+
/* Installs a stream created with svn_stream__create_for_install in its final
location FINAL_ABSPATH, potentially using platform specific optimizations.
@@ -172,6 +190,18 @@ svn_io__win_rename_open_file(apr_file_t
const char *to_path,
apr_pool_t *pool);
+/* This Windows-specific function sets the basic file information using an
+ existing file handle. If SET_MTIME is non-negative, it will be set as
+ the new LastWriteTime value. If SET_READ_ONLY is non-zero, the file
+ attributes will be updated to include FILE_ATTRIBUTE_READONLY.
+ Return SVN_ERR_UNSUPPORTED_FEATURE if not supported by OS. */
+svn_error_t *
+svn_io__win_set_file_basic_info(apr_file_t *file,
+ const char *path,
+ apr_time_t set_mtime,
+ svn_boolean_t set_read_only,
+ apr_pool_t *pool);
+
#endif /* WIN32 */
#ifdef __cplusplus
Modified: subversion/trunk/subversion/libsvn_subr/io.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_subr/io.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_subr/io.c (original)
+++ subversion/trunk/subversion/libsvn_subr/io.c Sat Feb 13 14:30:10 2021
@@ -144,6 +144,14 @@
#ifdef WIN32
#if _WIN32_WINNT < 0x600 /* Does the SDK assume Windows Vista+? */
+typedef struct _FILE_BASIC_INFO {
+ LARGE_INTEGER CreationTime;
+ LARGE_INTEGER LastAccessTime;
+ LARGE_INTEGER LastWriteTime;
+ LARGE_INTEGER ChangeTime;
+ DWORD FileAttributes;
+} FILE_BASIC_INFO, *PFILE_BASIC_INFO;
+
typedef struct _FILE_RENAME_INFO {
BOOL ReplaceIfExists;
HANDLE RootDirectory;
@@ -160,6 +168,7 @@ typedef struct _FILE_ATTRIBUTE_TAG_INFO
DWORD ReparseTag;
} FILE_ATTRIBUTE_TAG_INFO, *PFILE_ATTRIBUTE_TAG_INFO;
+#define FileBasicInfo 0
#define FileRenameInfo 3
#define FileDispositionInfo 4
#define FileAttributeTagInfo 9
@@ -2319,6 +2328,62 @@ svn_io__win_rename_open_file(apr_file_t
}
return SVN_NO_ERROR;
+}
+
+/* Number of micro-seconds between the beginning of the Windows epoch
+ * (Jan. 1, 1601) and the Unix epoch (Jan. 1, 1970)
+ */
+#ifndef APR_DELTA_EPOCH_IN_USEC
+#define APR_DELTA_EPOCH_IN_USEC APR_TIME_C(11644473600000000)
+#endif
+
+svn_error_t *
+svn_io__win_set_file_basic_info(apr_file_t *file,
+ const char *path,
+ apr_time_t set_mtime,
+ svn_boolean_t set_read_only,
+ apr_pool_t *pool)
+{
+ FILE_BASIC_INFO info;
+ HANDLE hFile;
+ apr_status_t status;
+
+ apr_os_file_get(&hFile, file);
+
+ if (set_read_only)
+ {
+ status = win32_get_file_information_by_handle(hFile, FileBasicInfo,
+ &info, sizeof(info));
+ if (status)
+ {
+ return svn_error_wrap_apr(status, _("Can't get attributes of '%s'"),
+ svn_dirent_local_style(path, pool));
+ }
+ }
+
+ info.CreationTime.QuadPart = 0;
+ info.LastAccessTime.QuadPart = 0;
+ info.ChangeTime.QuadPart = 0;
+
+ if (set_mtime >= 0)
+ info.LastWriteTime.QuadPart = (set_mtime + APR_DELTA_EPOCH_IN_USEC) * 10;
+ else
+ info.LastWriteTime.QuadPart = 0;
+
+ if (set_read_only)
+ info.FileAttributes |= FILE_ATTRIBUTE_READONLY;
+ else
+ info.FileAttributes = 0;
+
+ status = win32_set_file_information_by_handle(hFile, FileBasicInfo,
+ &info, sizeof(info));
+ if (status)
+ {
+ return svn_error_wrap_apr(status, _("Can't set attributes of '%s'"),
+ svn_dirent_local_style(path, pool));
+ }
+
+ return SVN_NO_ERROR;
}
#endif /* WIN32 */
Modified: subversion/trunk/subversion/libsvn_subr/stream.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_subr/stream.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_subr/stream.c (original)
+++ subversion/trunk/subversion/libsvn_subr/stream.c Sat Feb 13 14:30:10 2021
@@ -2172,6 +2172,9 @@ struct install_baton_t
{
struct baton_apr baton_apr;
const char *tmp_path;
+ svn_boolean_t set_read_only;
+ svn_boolean_t set_executable;
+ apr_time_t set_mtime;
};
#ifdef WIN32
@@ -2313,6 +2316,9 @@ svn_stream__create_for_install(svn_strea
(*install_stream)->baton = ib;
ib->tmp_path = tmp_path;
+ ib->set_read_only = FALSE;
+ ib->set_executable = FALSE;
+ ib->set_mtime = -1;
/* Don't close the file on stream close; flush instead */
svn_stream_set_close(*install_stream, install_close);
@@ -2320,6 +2326,33 @@ svn_stream__create_for_install(svn_strea
return SVN_NO_ERROR;
}
+void
+svn_stream__install_stream_set_read_only(svn_stream_t *install_stream,
+ svn_boolean_t read_only)
+{
+ struct install_baton_t *ib = install_stream->baton;
+
+ ib->set_read_only = read_only;
+}
+
+void
+svn_stream__install_stream_set_executable(svn_stream_t *install_stream,
+ svn_boolean_t executable)
+{
+ struct install_baton_t *ib = install_stream->baton;
+
+ ib->set_executable = executable;
+}
+
+void
+svn_stream__install_stream_set_affected_time(svn_stream_t *install_stream,
+ apr_time_t mtime)
+{
+ struct install_baton_t *ib = install_stream->baton;
+
+ ib->set_mtime = mtime;
+}
+
svn_error_t *
svn_stream__install_stream(svn_stream_t *install_stream,
const char *final_abspath,
@@ -2331,23 +2364,38 @@ svn_stream__install_stream(svn_stream_t
SVN_ERR_ASSERT(svn_dirent_is_absolute(final_abspath));
#ifdef WIN32
- err = svn_io__win_rename_open_file(ib->baton_apr.file, ib->tmp_path,
- final_abspath, scratch_pool);
- if (make_parents && err && APR_STATUS_IS_ENOENT(err->apr_err))
- {
- svn_error_t *err2;
-
- err2 = svn_io_make_dir_recursively(svn_dirent_dirname(final_abspath,
- scratch_pool),
- scratch_pool);
+ err = SVN_NO_ERROR;
- if (err2)
- return svn_error_trace(svn_error_compose_create(err, err2));
- else
- svn_error_clear(err);
+ /* Windows doesn't have an executable bit. */
+ if (ib->set_mtime >= 0 || ib->set_read_only)
+ {
+ err = svn_io__win_set_file_basic_info(ib->baton_apr.file,
+ ib->tmp_path,
+ ib->set_mtime,
+ ib->set_read_only,
+ scratch_pool);
+ }
+ if (!err)
+ {
err = svn_io__win_rename_open_file(ib->baton_apr.file, ib->tmp_path,
final_abspath, scratch_pool);
+ if (make_parents && err && APR_STATUS_IS_ENOENT(err->apr_err))
+ {
+ svn_error_t *err2;
+
+ err2 = svn_io_make_dir_recursively(svn_dirent_dirname(final_abspath,
+ scratch_pool),
+ scratch_pool);
+
+ if (err2)
+ return svn_error_trace(svn_error_compose_create(err, err2));
+ else
+ svn_error_clear(err);
+
+ err = svn_io__win_rename_open_file(ib->baton_apr.file, ib->tmp_path,
+ final_abspath, scratch_pool);
+ }
}
/* ### rhuijben: I wouldn't be surprised if we later find out that we
@@ -2372,6 +2420,13 @@ svn_stream__install_stream(svn_stream_t
/* Close temporary file. */
SVN_ERR(svn_io_file_close(ib->baton_apr.file, scratch_pool));
+ if (ib->set_read_only)
+ SVN_ERR(svn_io_set_file_read_only(ib->tmp_path, FALSE, scratch_pool));
+ if (ib->set_executable)
+ SVN_ERR(svn_io_set_file_executable(ib->tmp_path, TRUE, FALSE,
scratch_pool));
+ if (ib->set_mtime >= 0)
+ SVN_ERR(svn_io_set_file_affected_time(ib->set_mtime, ib->tmp_path,
scratch_pool));
+
err = svn_io_file_rename2(ib->tmp_path, final_abspath, FALSE, scratch_pool);
/* A missing directory is too common to not cover here. */
@@ -2409,8 +2464,10 @@ svn_stream__install_get_info(apr_time_t
apr_int32_t wanted;
apr_finfo_t finfo;
+ /* If we are instructed to set a specific timestamp, return it.
+ For everything else, stat the file to get the information. */
wanted = 0;
- if (mtime_p)
+ if (mtime_p && ib->set_mtime < 0)
wanted |= APR_FINFO_MTIME;
if (size_p)
wanted |= APR_FINFO_SIZE;
@@ -2422,7 +2479,9 @@ svn_stream__install_get_info(apr_time_t
return svn_error_wrap_apr(status, NULL);
}
- if (mtime_p)
+ if (mtime_p && ib->set_mtime >= 0)
+ *mtime_p = ib->set_mtime;
+ else if (mtime_p)
*mtime_p = finfo.mtime;
if (size_p)
Modified: subversion/trunk/subversion/libsvn_wc/update_editor.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/update_editor.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/update_editor.c (original)
+++ subversion/trunk/subversion/libsvn_wc/update_editor.c Sat Feb 13 14:30:10
2021
@@ -48,6 +48,7 @@
#include "conflicts.h"
#include "translate.h"
#include "workqueue.h"
+#include "working_file_writer.h"
#include "private/svn_subr_private.h"
#include "private/svn_wc_private.h"
@@ -765,6 +766,12 @@ struct file_baton
/* The tree conflict to install once the node is really edited */
svn_skel_t *edit_conflict;
+
+ /* Cached copy of the base properties for this file. */
+ apr_hash_t *base_props;
+
+ /* Writer with the provisioned content of the working file. May be NULL. */
+ svn_wc__working_file_writer_t *file_writer;
};
@@ -1014,6 +1021,12 @@ window_handler(svn_txdelta_window_t *win
svn_error_clear(svn_wc__db_pristine_install_abort(hb->install_data,
hb->pool));
}
+
+ if (fb->file_writer)
+ {
+ svn_error_clear(svn_wc__working_file_writer_close(fb->file_writer));
+ fb->file_writer = NULL;
+ }
}
else
{
@@ -3560,6 +3573,112 @@ open_file(const char *path,
return SVN_NO_ERROR;
}
+/* Return the, possibly cached, base properties for file associated
+ with the delta editor baton FB. */
+static svn_error_t *
+get_file_base_props(apr_hash_t **props_p,
+ struct file_baton *fb,
+ apr_pool_t *scratch_pool)
+{
+ if (!fb->base_props)
+ {
+ apr_hash_t *props;
+
+ if (fb->add_existed)
+ {
+ /* This node already exists. Grab the current pristine properties. */
+ SVN_ERR(svn_wc__db_read_pristine_props(&props,
+ fb->edit_baton->db,
+ fb->local_abspath,
+ fb->pool, scratch_pool));
+ }
+ else if (!fb->adding_file)
+ {
+ /* Get the BASE properties for proper merging. */
+ SVN_ERR(svn_wc__db_base_get_props(&props,
+ fb->edit_baton->db,
+ fb->local_abspath,
+ fb->pool, scratch_pool));
+ }
+ else
+ props = apr_hash_make(fb->pool);
+
+ fb->base_props = props;
+ }
+
+ *props_p = fb->base_props;
+ return SVN_NO_ERROR;
+}
+
+/* Create the writer of the working file based on the state of
+ the file associated with the delta editor baton FB. */
+static svn_error_t *
+open_working_file_writer(svn_wc__working_file_writer_t **writer_p,
+ struct file_baton *fb,
+ apr_pool_t *result_pool,
+ apr_pool_t *scratch_pool)
+{
+ apr_hash_t *base_props;
+ apr_hash_t *props;
+ const char *lock_token;
+ const char *temp_dir_abspath;
+ const char *cmt_rev_str;
+ svn_revnum_t cmt_rev;
+ const char *cmt_date_str;
+ apr_time_t cmt_date;
+ const char *cmt_author;
+ apr_time_t final_mtime;
+
+ SVN_ERR(get_file_base_props(&base_props, fb, scratch_pool));
+ props = svn_prop__patch(base_props, fb->propchanges, scratch_pool);
+
+ cmt_rev_str = svn_prop_get_value(props, SVN_PROP_ENTRY_COMMITTED_REV);
+ if (cmt_rev_str)
+ {
+ apr_int64_t rev;
+ SVN_ERR(svn_cstring_atoi64(&rev, cmt_rev_str));
+ cmt_rev = (svn_revnum_t)rev;
+ }
+ else
+ cmt_rev = SVN_INVALID_REVNUM;
+
+ cmt_date_str = svn_prop_get_value(props, SVN_PROP_ENTRY_COMMITTED_DATE);
+ if (cmt_date_str)
+ SVN_ERR(svn_time_from_cstring(&cmt_date, cmt_date_str, scratch_pool));
+ else
+ cmt_date = 0;
+
+ cmt_author = svn_prop_get_value(props, SVN_PROP_ENTRY_LAST_AUTHOR);
+
+ if (fb->edit_baton->use_commit_times && cmt_date)
+ final_mtime = cmt_date;
+ else
+ final_mtime = -1;
+
+ lock_token = svn_prop_get_value(props, SVN_PROP_ENTRY_LOCK_TOKEN);
+
+ SVN_ERR(svn_wc__db_temp_wcroot_tempdir(&temp_dir_abspath,
+ fb->edit_baton->db,
+ fb->edit_baton->wcroot_abspath,
+ scratch_pool, scratch_pool));
+
+ SVN_ERR(svn_wc__working_file_writer_open(writer_p,
+ temp_dir_abspath,
+ final_mtime,
+ props,
+ cmt_rev,
+ cmt_date,
+ cmt_author,
+ lock_token != NULL,
+ fb->adding_file,
+ fb->edit_baton->repos_root,
+ fb->new_repos_relpath,
+ fb->pool,
+ scratch_pool));
+
+ return SVN_NO_ERROR;
+}
+
/* Implements svn_stream_lazyopen_func_t. */
static svn_error_t *
lazy_open_source(svn_stream_t **stream,
@@ -3580,29 +3699,61 @@ lazy_open_source(svn_stream_t **stream,
/* Implements svn_stream_lazyopen_func_t. */
static svn_error_t *
-lazy_open_target(svn_stream_t **stream,
+lazy_open_target(svn_stream_t **stream_p,
void *baton,
apr_pool_t *result_pool,
apr_pool_t *scratch_pool)
{
struct handler_baton *hb = baton;
- svn_wc__db_install_data_t *install_data;
+ struct file_baton *fb = hb->fb;
+ svn_wc__db_install_data_t *pristine_install_data;
+ svn_stream_t *pristine_install_stream;
+ svn_wc__working_file_writer_t *file_writer;
+ svn_stream_t *stream;
/* By convention return value is undefined on error, but we rely
on HB->INSTALL_DATA value in window_handler() and abort
INSTALL_STREAM if is not NULL on error.
So we store INSTALL_DATA to local variable first, to leave
HB->INSTALL_DATA unchanged on error. */
- SVN_ERR(svn_wc__db_pristine_prepare_install(stream,
- &install_data,
+ SVN_ERR(svn_wc__db_pristine_prepare_install(&pristine_install_stream,
+ &pristine_install_data,
&hb->new_text_base_sha1_checksum,
NULL,
- hb->fb->edit_baton->db,
- hb->fb->dir_baton->local_abspath,
+ fb->edit_baton->db,
+ fb->dir_baton->local_abspath,
result_pool, scratch_pool));
- hb->install_data = install_data;
+ if (fb->shadowed || fb->obstruction_found || fb->edit_obstructed)
+ {
+ /* No need to provision the working file. */
+ file_writer = NULL;
+ }
+ else
+ {
+ SVN_ERR(open_working_file_writer(&file_writer, fb, fb->pool,
+ scratch_pool));
+ }
+
+ hb->install_data = pristine_install_data;
+ fb->file_writer = file_writer;
+ /* Configure the incoming data to be written to both the pristine and the
+ provisioned working file. This allows us to properly stream data and
+ avoid having to re-read and copy the contents of the pristine file as
+ some post-op. Streaming is essential if we want the underlying transport
+ such as HTTP to work without timeouts and without any limits on the
+ sizes of the directories that can be checked out. */
+ stream = pristine_install_stream;
+ if (file_writer)
+ {
+ stream = svn_stream_tee(
+ pristine_install_stream,
+ svn_wc__working_file_writer_get_stream(file_writer),
+ result_pool);
+ }
+
+ *stream_p = stream;
return SVN_NO_ERROR;
}
@@ -4214,6 +4365,9 @@ close_file(void *file_baton,
svn_boolean_t keep_recorded_info = FALSE;
const svn_checksum_t *new_checksum;
apr_array_header_t *iprops = NULL;
+ svn_boolean_t install_pristine;
+ const char *install_from = NULL;
+ svn_boolean_t record_fileinfo;
if (fb->skip_this)
{
@@ -4318,22 +4472,8 @@ close_file(void *file_baton,
if (local_actual_props == NULL)
local_actual_props = apr_hash_make(scratch_pool);
- if (fb->add_existed)
- {
- /* This node already exists. Grab the current pristine properties. */
- SVN_ERR(svn_wc__db_read_pristine_props(¤t_base_props,
- eb->db, fb->local_abspath,
- scratch_pool, scratch_pool));
- current_actual_props = local_actual_props;
- }
- else if (!fb->adding_file)
- {
- /* Get the BASE properties for proper merging. */
- SVN_ERR(svn_wc__db_base_get_props(¤t_base_props,
- eb->db, fb->local_abspath,
- scratch_pool, scratch_pool));
- current_actual_props = local_actual_props;
- }
+ SVN_ERR(get_file_base_props(¤t_base_props, fb, scratch_pool));
+ current_actual_props = local_actual_props;
/* Note: even if the node existed before, it may not have
pristine props (e.g a local-add) */
@@ -4348,9 +4488,6 @@ close_file(void *file_baton,
if (! fb->shadowed)
{
- svn_boolean_t install_pristine;
- const char *install_from = NULL;
-
/* Merge the 'regular' props into the existing working proplist. */
/* This will merge the old and new props into a new prop db, and
write <cp> commands to the logfile to install the merged
@@ -4419,29 +4556,8 @@ close_file(void *file_baton,
content_state = svn_wc_notify_state_unchanged;
}
- if (install_pristine)
- {
- svn_boolean_t record_fileinfo;
-
- /* If we are installing from the pristine contents, then go ahead and
- record the fileinfo. That will be the "proper" values. Installing
- from some random file means the fileinfo does NOT correspond to
- the pristine (in which case, the fileinfo will be cleared for
- safety's sake). */
- record_fileinfo = (install_from == NULL);
-
- SVN_ERR(svn_wc__wq_build_file_install(&work_item,
- eb->db,
- fb->local_abspath,
- install_from,
- eb->use_commit_times,
- record_fileinfo,
- scratch_pool, scratch_pool));
- all_work_items = svn_wc__wq_merge(all_work_items, work_item,
- scratch_pool);
- }
- else if (lock_state == svn_wc_notify_lock_state_unlocked
- && !fb->obstruction_found)
+ if (lock_state == svn_wc_notify_lock_state_unlocked
+ && !fb->obstruction_found)
{
/* If a lock was removed and we didn't update the text contents, we
might need to set the file read-only.
@@ -4460,20 +4576,6 @@ close_file(void *file_baton,
/* It is safe to keep the current recorded timestamp and size */
keep_recorded_info = TRUE;
}
-
- /* Clean up any temporary files. */
-
- /* Remove the INSTALL_FROM file, as long as it doesn't refer to the
- working file. */
- if (install_from != NULL
- && strcmp(install_from, fb->local_abspath) != 0)
- {
- SVN_ERR(svn_wc__wq_build_file_remove(&work_item, eb->db,
- fb->local_abspath, install_from,
- scratch_pool, scratch_pool));
- all_work_items = svn_wc__wq_merge(all_work_items, work_item,
- scratch_pool);
- }
}
else
{
@@ -4505,6 +4607,8 @@ close_file(void *file_baton,
content_state = svn_wc_notify_state_changed;
else
content_state = svn_wc_notify_state_unchanged;
+
+ install_pristine = FALSE;
}
/* Insert/replace the BASE node with all of the new metadata. */
@@ -4558,6 +4662,54 @@ close_file(void *file_baton,
svn_hash_sets(eb->wcroot_iprops, fb->local_abspath, NULL);
}
+ /* We have to install the working file from the pristine, but we did not
+ receive it as the part of this edit. Or we did receive it, but have to
+ install from a different file. Either way, reset the writer and copy
+ appropriate contents to it. */
+ if (install_pristine && (fb->file_writer == NULL || install_from))
+ {
+ svn_stream_t *src_stream;
+ svn_stream_t *dst_stream;
+
+ if (!fb->file_writer)
+ {
+ SVN_ERR(open_working_file_writer(&fb->file_writer, fb, fb->pool,
+ scratch_pool));
+ }
+
+ dst_stream = svn_wc__working_file_writer_get_stream(fb->file_writer);
+ SVN_ERR(svn_stream_reset(dst_stream));
+
+ if (install_from)
+ {
+ apr_file_t *file;
+ /* Open an unbuffered file, as svn_wc__db_pristine_read() does. */
+ SVN_ERR(svn_io_file_open(&file, install_from, APR_READ,
+ APR_OS_DEFAULT, scratch_pool));
+ src_stream = svn_stream_from_aprfile2(file, FALSE, scratch_pool);
+ }
+ else
+ {
+ SVN_ERR(svn_wc__db_pristine_read(&src_stream, NULL, eb->db,
+ eb->wcroot_abspath, new_checksum,
+ scratch_pool, scratch_pool));
+ }
+
+ SVN_ERR(svn_stream_copy3(src_stream, dst_stream, NULL, NULL,
scratch_pool));
+
+ /* Remove the INSTALL_FROM file, as long as it doesn't refer to the
+ working file. */
+ if (install_from && strcmp(install_from, fb->local_abspath) != 0)
+ SVN_ERR(svn_io_remove_file2(install_from, TRUE, scratch_pool));
+ }
+
+ /* If we are installing from the pristine contents, then go ahead and
+ record the fileinfo. That will be the "proper" values. Installing
+ from some random file means the fileinfo does NOT correspond to
+ the pristine (in which case, the fileinfo will be cleared for
+ safety's sake). */
+ record_fileinfo = (install_from == NULL);
+
SVN_ERR(svn_wc__db_base_add_file(eb->db, fb->local_abspath,
eb->wcroot_abspath,
fb->new_repos_relpath,
@@ -4580,9 +4732,17 @@ close_file(void *file_baton,
keep_recorded_info,
(fb->shadowed && fb->obstruction_found),
conflict_skel,
+ install_pristine ? fb->file_writer : NULL,
+ record_fileinfo,
all_work_items,
scratch_pool));
+ if (fb->file_writer)
+ {
+ SVN_ERR(svn_wc__working_file_writer_close(fb->file_writer));
+ fb->file_writer = NULL;
+ }
+
if (conflict_skel && eb->conflict_func)
SVN_ERR(svn_wc__conflict_invoke_resolver(eb->db, fb->local_abspath,
svn_node_file,
Modified: subversion/trunk/subversion/libsvn_wc/wc_db.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc_db.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc_db.c (original)
+++ subversion/trunk/subversion/libsvn_wc/wc_db.c Sat Feb 13 14:30:10 2021
@@ -1752,6 +1752,59 @@ svn_wc__db_base_add_incomplete_directory
}
+/* Record RECORDED_SIZE and RECORDED_TIME into top layer in NODES */
+static svn_error_t *
+db_record_fileinfo(svn_wc__db_wcroot_t *wcroot,
+ const char *local_relpath,
+ apr_int64_t recorded_size,
+ apr_int64_t recorded_time,
+ apr_pool_t *scratch_pool)
+{
+ svn_sqlite__stmt_t *stmt;
+ int affected_rows;
+
+ SVN_ERR(svn_sqlite__get_statement(&stmt, wcroot->sdb,
+ STMT_UPDATE_NODE_FILEINFO));
+ SVN_ERR(svn_sqlite__bindf(stmt, "isii", wcroot->wc_id, local_relpath,
+ recorded_size, recorded_time));
+ SVN_ERR(svn_sqlite__update(&affected_rows, stmt));
+
+ SVN_ERR_ASSERT(affected_rows == 1);
+
+ return SVN_NO_ERROR;
+}
+
+
+/* Install the working file provided by FILE_WRITER, optionally
+ recording its fileinfo. */
+static svn_error_t *
+install_working_file(svn_wc__db_wcroot_t *wcroot,
+ const char *local_relpath,
+ svn_wc__working_file_writer_t *file_writer,
+ svn_boolean_t record_fileinfo,
+ apr_pool_t *scratch_pool)
+{
+ const char *local_abspath;
+
+ if (record_fileinfo)
+ {
+ apr_time_t mtime;
+ apr_off_t size;
+
+ SVN_ERR(svn_wc__working_file_writer_get_info(&mtime, &size, file_writer,
+ scratch_pool));
+ SVN_ERR(db_record_fileinfo(wcroot, local_relpath, size, mtime,
+ scratch_pool));
+ }
+
+ local_abspath = svn_dirent_join(wcroot->abspath, local_relpath,
scratch_pool);
+ SVN_ERR(svn_wc__working_file_writer_install(file_writer, local_abspath,
+ scratch_pool));
+
+ return SVN_NO_ERROR;
+}
+
+
svn_error_t *
svn_wc__db_base_add_file(svn_wc__db_t *db,
const char *local_abspath,
@@ -1773,6 +1826,8 @@ svn_wc__db_base_add_file(svn_wc__db_t *d
svn_boolean_t keep_recorded_info,
svn_boolean_t insert_base_deleted,
const svn_skel_t *conflict,
+ svn_wc__working_file_writer_t *file_writer,
+ svn_boolean_t record_fileinfo,
const svn_skel_t *work_items,
apr_pool_t *scratch_pool)
{
@@ -1828,9 +1883,22 @@ svn_wc__db_base_add_file(svn_wc__db_t *d
ibb.conflict = conflict;
ibb.work_items = work_items;
- SVN_WC__DB_WITH_TXN(
- insert_base_node(&ibb, wcroot, local_relpath, scratch_pool),
- wcroot);
+ if (file_writer)
+ {
+ /* Atomically update the db and install the file (installation
+ * of the file is atomic, because it happens as a rename). */
+ SVN_WC__DB_WITH_TXN2(
+ insert_base_node(&ibb, wcroot, local_relpath, scratch_pool),
+ install_working_file(wcroot, local_relpath, file_writer,
+ record_fileinfo, scratch_pool),
+ wcroot);
+ }
+ else
+ {
+ SVN_WC__DB_WITH_TXN(
+ insert_base_node(&ibb, wcroot, local_relpath, scratch_pool),
+ wcroot);
+ }
/* If this used to be a directory we should remove children so pass
* depth infinity. */
@@ -6045,28 +6113,6 @@ svn_wc__db_op_add_symlink(svn_wc__db_t *
return SVN_NO_ERROR;
}
-
-/* Record RECORDED_SIZE and RECORDED_TIME into top layer in NODES */
-static svn_error_t *
-db_record_fileinfo(svn_wc__db_wcroot_t *wcroot,
- const char *local_relpath,
- apr_int64_t recorded_size,
- apr_int64_t recorded_time,
- apr_pool_t *scratch_pool)
-{
- svn_sqlite__stmt_t *stmt;
- int affected_rows;
-
- SVN_ERR(svn_sqlite__get_statement(&stmt, wcroot->sdb,
- STMT_UPDATE_NODE_FILEINFO));
- SVN_ERR(svn_sqlite__bindf(stmt, "isii", wcroot->wc_id, local_relpath,
- recorded_size, recorded_time));
- SVN_ERR(svn_sqlite__update(&affected_rows, stmt));
-
- SVN_ERR_ASSERT(affected_rows == 1);
-
- return SVN_NO_ERROR;
-}
svn_error_t *
Modified: subversion/trunk/subversion/libsvn_wc/wc_db.h
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/wc_db.h?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/wc_db.h (original)
+++ subversion/trunk/subversion/libsvn_wc/wc_db.h Sat Feb 13 14:30:10 2021
@@ -49,6 +49,8 @@
#include "svn_private_config.h"
+#include "working_file_writer.h"
+
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
@@ -503,6 +505,10 @@ svn_wc__db_base_add_incomplete_directory
Unless KEEP_RECORDED_INFO is set to TRUE, recorded size and timestamp values
will be cleared.
+ If FILE_WRITER is not NULL, atomically complete installation of the file to
+ LOCAL_ABSPATH together with updating the DB. If RECORD_FILEINFO is
non-zero,
+ also record the size and timestamp values of the installed file.
+
All temporary allocations will be made in SCRATCH_POOL.
*/
svn_error_t *
@@ -526,6 +532,8 @@ svn_wc__db_base_add_file(svn_wc__db_t *d
svn_boolean_t keep_recorded_info,
svn_boolean_t insert_base_deleted,
const svn_skel_t *conflict,
+ svn_wc__working_file_writer_t *file_writer,
+ svn_boolean_t record_fileinfo,
const svn_skel_t *work_items,
apr_pool_t *scratch_pool);
Added: subversion/trunk/subversion/libsvn_wc/working_file_writer.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/working_file_writer.c?rev=1886490&view=auto
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/working_file_writer.c (added)
+++ subversion/trunk/subversion/libsvn_wc/working_file_writer.c Sat Feb 13
14:30:10 2021
@@ -0,0 +1,223 @@
+/*
+ * working_file_writer.c : utility to prepare and install working files
+ *
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ */
+
+#include "svn_props.h"
+#include "svn_subst.h"
+#include "svn_time.h"
+#include "svn_path.h"
+
+#include "private/svn_io_private.h"
+
+#include "working_file_writer.h"
+
+struct svn_wc__working_file_writer_t
+{
+ apr_pool_t *pool;
+ const char *tmp_abspath;
+ svn_boolean_t is_special;
+ svn_stream_t *install_stream;
+ svn_stream_t *write_stream;
+};
+
+static apr_status_t
+cleanup_file_writer(void *baton)
+{
+ svn_wc__working_file_writer_t *writer = baton;
+
+ if (writer->install_stream)
+ {
+ svn_error_clear(svn_stream__install_delete(writer->install_stream,
+ writer->pool));
+ writer->install_stream = NULL;
+ }
+
+ return APR_SUCCESS;
+}
+
+svn_error_t *
+svn_wc__working_file_writer_open(svn_wc__working_file_writer_t **writer_p,
+ const char *tmp_abspath,
+ apr_time_t final_mtime,
+ apr_hash_t *props,
+ svn_revnum_t changed_rev,
+ apr_time_t changed_date,
+ const char *changed_author,
+ svn_boolean_t has_lock,
+ svn_boolean_t is_added,
+ const char *repos_root_url,
+ const char *repos_relpath,
+ apr_pool_t *result_pool,
+ apr_pool_t *scratch_pool)
+{
+ svn_wc__working_file_writer_t *writer;
+ const char *url;
+ svn_boolean_t special;
+ svn_boolean_t executable;
+ svn_boolean_t needs_lock;
+ const char *keywords_propval;
+ apr_hash_t *keywords;
+ const char *eol_propval;
+ svn_subst_eol_style_t eol_style;
+ const char *eol;
+ svn_stream_t *install_stream;
+ svn_stream_t *write_stream;
+
+ url = svn_path_url_add_component2(repos_root_url, repos_relpath,
scratch_pool);
+
+ special = svn_prop_get_value(props, SVN_PROP_SPECIAL) != NULL;
+ executable = svn_prop_get_value(props, SVN_PROP_EXECUTABLE) != NULL;
+ needs_lock = svn_prop_get_value(props, SVN_PROP_NEEDS_LOCK) != NULL;
+
+ eol_propval = svn_prop_get_value(props, SVN_PROP_EOL_STYLE);
+ svn_subst_eol_style_from_value(&eol_style, &eol, eol_propval);
+
+ keywords_propval = svn_prop_get_value(props, SVN_PROP_KEYWORDS);
+ if (keywords_propval)
+ {
+ SVN_ERR(svn_subst_build_keywords3(&keywords, keywords_propval,
+ apr_psprintf(scratch_pool, "%ld",
+ changed_rev),
+ url, repos_root_url, changed_date,
+ changed_author, scratch_pool));
+ if (apr_hash_count(keywords) <= 0)
+ keywords = NULL;
+ }
+ else
+ keywords = NULL;
+
+ SVN_ERR(svn_stream__create_for_install(&install_stream, tmp_abspath,
+ result_pool, scratch_pool));
+
+ if (needs_lock && !is_added && !has_lock)
+ svn_stream__install_stream_set_read_only(install_stream, TRUE);
+ if (executable)
+ svn_stream__install_stream_set_executable(install_stream, TRUE);
+ if (final_mtime >= 0)
+ svn_stream__install_stream_set_affected_time(install_stream, final_mtime);
+
+ write_stream = install_stream;
+
+ if (svn_subst_translation_required(eol_style, eol, keywords,
+ FALSE /* special */,
+ TRUE /* force_eol_check */))
+ {
+ write_stream = svn_subst_stream_translated(write_stream,
+ eol,
+ TRUE /* repair */,
+ keywords,
+ TRUE /* expand */,
+ result_pool);
+ }
+
+ writer = apr_pcalloc(result_pool, sizeof(*writer));
+ writer->pool = result_pool;
+ writer->tmp_abspath = apr_pstrdup(result_pool, tmp_abspath);
+ writer->is_special = special;
+ writer->install_stream = install_stream;
+ writer->write_stream = write_stream;
+
+ apr_pool_cleanup_register(result_pool, writer, cleanup_file_writer,
+ apr_pool_cleanup_null);
+
+ *writer_p = writer;
+ return SVN_NO_ERROR;
+}
+
+svn_stream_t *
+svn_wc__working_file_writer_get_stream(svn_wc__working_file_writer_t *writer)
+{
+ return writer->write_stream;
+}
+
+svn_error_t *
+svn_wc__working_file_writer_get_info(apr_time_t *mtime_p,
+ apr_off_t *size_p,
+ svn_wc__working_file_writer_t *writer,
+ apr_pool_t *scratch_pool)
+{
+ SVN_ERR(svn_stream__install_get_info(mtime_p, size_p, writer->install_stream,
+ scratch_pool));
+
+ return SVN_NO_ERROR;
+}
+
+svn_error_t *
+svn_wc__working_file_writer_install(svn_wc__working_file_writer_t *writer,
+ const char *target_abspath,
+ apr_pool_t *scratch_pool)
+{
+ if (writer->is_special)
+ {
+ const char *temp_path;
+ svn_stream_t *src_stream;
+ svn_stream_t *dst_stream;
+
+ /* Install the current contents to a temporary file, and use it to
+ create the resulting special file. */
+ SVN_ERR(svn_io_open_unique_file3(NULL, &temp_path, writer->tmp_abspath,
+ svn_io_file_del_on_pool_cleanup,
+ scratch_pool, scratch_pool));
+ SVN_ERR(svn_stream__install_stream(writer->install_stream, temp_path,
+ TRUE, scratch_pool));
+ writer->install_stream = NULL;
+
+ /* When this stream is closed, the resulting special file will
+ atomically be created/moved into place at LOCAL_ABSPATH. */
+ SVN_ERR(svn_subst_create_specialfile(&dst_stream, target_abspath,
+ scratch_pool, scratch_pool));
+ SVN_ERR(svn_stream_open_readonly(&src_stream, temp_path, scratch_pool,
+ scratch_pool));
+ SVN_ERR(svn_stream_copy3(src_stream, dst_stream, NULL, NULL,
+ scratch_pool));
+ SVN_ERR(svn_io_remove_file2(temp_path, TRUE, scratch_pool));
+
+ return SVN_NO_ERROR;
+ }
+ else
+ {
+ /* With a single db we might want to install files in a missing
directory.
+ Simply trying this scenario on error won't do any harm and at least
+ one user reported this problem on IRC. */
+ SVN_ERR(svn_stream__install_stream(writer->install_stream,
+ target_abspath, TRUE,
+ scratch_pool));
+ writer->install_stream = NULL;
+ }
+
+ return SVN_NO_ERROR;
+}
+
+svn_error_t *
+svn_wc__working_file_writer_close(svn_wc__working_file_writer_t *writer)
+{
+ if (writer->install_stream)
+ {
+ svn_stream_t *stream = writer->install_stream;
+ /* Do not retry deleting if it fails, as the stream may already
+ be in an invalid state. */
+ writer->install_stream = NULL;
+ SVN_ERR(svn_stream__install_delete(stream, writer->pool));
+ }
+
+ return SVN_NO_ERROR;
+}
Propchange: subversion/trunk/subversion/libsvn_wc/working_file_writer.c
------------------------------------------------------------------------------
svn:eol-style = native
Added: subversion/trunk/subversion/libsvn_wc/working_file_writer.h
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/working_file_writer.h?rev=1886490&view=auto
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/working_file_writer.h (added)
+++ subversion/trunk/subversion/libsvn_wc/working_file_writer.h Sat Feb 13
14:30:10 2021
@@ -0,0 +1,86 @@
+/*
+ * working_file_writer.h : utility to prepare and install working files
+ *
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ */
+
+#ifndef SVN_LIBSVN_WC_WORKING_FILE_WRITER_H
+#define SVN_LIBSVN_WC_WORKING_FILE_WRITER_H
+
+#include <apr_pools.h>
+#include "svn_types.h"
+#include "svn_io.h"
+
+#ifdef __cplusplus
+extern "C" {
+#endif /* __cplusplus */
+
+/* Context of the working file writer. */
+typedef struct svn_wc__working_file_writer_t svn_wc__working_file_writer_t;
+
+/* Create a write context for the (provisioned) working file with the specified
+ properties. The file writer must be given data in the repository-normal
+ form and will handle its translation according to the specified properties.
+ Place the temporary file in the TMP_ABSPATH directory. If FINAL_MTIME is
+ non-negative, it will be set as the last modification time on the installed
+ file. */
+svn_error_t *
+svn_wc__working_file_writer_open(svn_wc__working_file_writer_t **writer_p,
+ const char *tmp_abspath,
+ apr_time_t final_mtime,
+ apr_hash_t *props,
+ svn_revnum_t changed_rev,
+ apr_time_t changed_date,
+ const char *changed_author,
+ svn_boolean_t has_lock,
+ svn_boolean_t is_added,
+ const char *repos_root_url,
+ const char *repos_relpath,
+ apr_pool_t *result_pool,
+ apr_pool_t *scratch_pool);
+
+/* Get the writable stream for WRITER. The returned stream supports reset
+ and is configured to be truncated on seek. */
+svn_stream_t *
+svn_wc__working_file_writer_get_stream(svn_wc__working_file_writer_t *writer);
+
+/* Retrieve file information for WRITER.
+ MTIME_P and SIZE_P both may be NULL to allow for partial queries. */
+svn_error_t *
+svn_wc__working_file_writer_get_info(apr_time_t *mtime_p,
+ apr_off_t *size_p,
+ svn_wc__working_file_writer_t *writer,
+ apr_pool_t *scratch_pool);
+
+/* Atomically install the contents of WRITER to TARGET_ABSPATH. */
+svn_error_t *
+svn_wc__working_file_writer_install(svn_wc__working_file_writer_t *writer,
+ const char *target_abspath,
+ apr_pool_t *scratch_pool);
+
+/* Cleanup WRITER by closing and removing the underlying file. */
+svn_error_t *
+svn_wc__working_file_writer_close(svn_wc__working_file_writer_t *writer);
+
+#ifdef __cplusplus
+}
+#endif /* __cplusplus */
+
+#endif /* SVN_LIBSVN_WC_WORKING_FILE_WRITER_H */
Propchange: subversion/trunk/subversion/libsvn_wc/working_file_writer.h
------------------------------------------------------------------------------
svn:eol-style = native
Modified: subversion/trunk/subversion/libsvn_wc/workqueue.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_wc/workqueue.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_wc/workqueue.c (original)
+++ subversion/trunk/subversion/libsvn_wc/workqueue.c Sat Feb 13 14:30:10 2021
@@ -37,6 +37,7 @@
#include "adm_files.h"
#include "conflicts.h"
#include "translate.h"
+#include "working_file_writer.h"
#include "private/svn_io_private.h"
#include "private/svn_skel.h"
@@ -463,19 +464,22 @@ run_file_install(work_item_baton_t *wqb,
const char *local_abspath;
svn_boolean_t use_commit_times;
svn_boolean_t record_fileinfo;
- svn_boolean_t special;
svn_stream_t *src_stream;
- svn_subst_eol_style_t style;
- const char *eol;
- apr_hash_t *keywords;
const char *temp_dir_abspath;
- svn_stream_t *dst_stream;
apr_int64_t val;
const char *wcroot_abspath;
const char *source_abspath;
const svn_checksum_t *checksum;
apr_hash_t *props;
apr_time_t changed_date;
+ svn_revnum_t changed_rev;
+ const char *changed_author;
+ apr_time_t final_mtime;
+ svn_wc__db_status_t status;
+ svn_wc__db_lock_t *lock;
+ const char *repos_relpath;
+ const char *repos_root_url;
+ svn_wc__working_file_writer_t *file_writer;
local_relpath = apr_pstrmemdup(scratch_pool, arg1->data, arg1->len);
SVN_ERR(svn_wc__db_from_relpath(&local_abspath, db, wri_abspath,
@@ -524,101 +528,54 @@ run_file_install(work_item_baton_t *wqb,
scratch_pool, scratch_pool));
}
- SVN_ERR(svn_stream_open_readonly(&src_stream, source_abspath,
- scratch_pool, scratch_pool));
-
- /* Fetch all the translation bits. */
- SVN_ERR(svn_wc__get_translate_info(&style, &eol,
- &keywords,
- &special, db, local_abspath,
- props, FALSE,
- scratch_pool, scratch_pool));
- if (special)
- {
- /* When this stream is closed, the resulting special file will
- atomically be created/moved into place at LOCAL_ABSPATH. */
- SVN_ERR(svn_subst_create_specialfile(&dst_stream, local_abspath,
- scratch_pool, scratch_pool));
-
- /* Copy the "repository normal" form of the special file into the
- special stream. */
- SVN_ERR(svn_stream_copy3(src_stream, dst_stream,
- cancel_func, cancel_baton,
- scratch_pool));
-
- /* No need to set exec or read-only flags on special files. */
-
- /* ### Shouldn't this record a timestamp and size, etc.? */
- return SVN_NO_ERROR;
- }
-
- if (svn_subst_translation_required(style, eol, keywords,
- FALSE /* special */,
- TRUE /* force_eol_check */))
- {
- /* Wrap it in a translating (expanding) stream. */
- src_stream = svn_subst_stream_translated(src_stream, eol,
- TRUE /* repair */,
- keywords,
- TRUE /* expand */,
- scratch_pool);
- }
-
/* Where is the Right Place to put a temp file in this working copy? */
SVN_ERR(svn_wc__db_temp_wcroot_tempdir(&temp_dir_abspath,
db, wcroot_abspath,
scratch_pool, scratch_pool));
- /* Translate to a temporary file. We don't want the user seeing a partial
- file, nor let them muck with it while we translate. We may also need to
- get its TRANSLATED_SIZE before the user can monkey it. */
- SVN_ERR(svn_stream__create_for_install(&dst_stream, temp_dir_abspath,
- scratch_pool, scratch_pool));
+ SVN_ERR(svn_wc__db_read_info(&status, NULL, NULL, &repos_relpath,
+ &repos_root_url, NULL, &changed_rev, NULL,
+ &changed_author, NULL, NULL, NULL, NULL,
+ NULL, NULL, NULL, &lock, NULL, NULL,
+ NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL,
+ db, local_abspath,
+ scratch_pool, scratch_pool));
+ /* Handle special statuses (e.g. added) */
+ if (!repos_relpath)
+ SVN_ERR(svn_wc__db_read_repos_info(NULL, &repos_relpath,
+ &repos_root_url, NULL,
+ db, local_abspath,
+ scratch_pool, scratch_pool));
- /* Copy from the source to the dest, translating as we go. This will also
- close both streams. */
- SVN_ERR(svn_stream_copy3(src_stream, dst_stream,
- cancel_func, cancel_baton,
- scratch_pool));
+ if (use_commit_times && changed_date)
+ final_mtime = changed_date;
+ else
+ final_mtime = -1;
- /* All done. Move the file into place. */
- /* With a single db we might want to install files in a missing directory.
- Simply trying this scenario on error won't do any harm and at least
- one user reported this problem on IRC. */
- SVN_ERR(svn_stream__install_stream(dst_stream, local_abspath,
- TRUE /* make_parents*/, scratch_pool));
-
- /* Tweak the on-disk file according to its properties. */
-#ifndef WIN32
- if (props && svn_hash_gets(props, SVN_PROP_EXECUTABLE))
- SVN_ERR(svn_io_set_file_executable(local_abspath, TRUE, FALSE,
- scratch_pool));
-#endif
-
- /* Note that this explicitly checks the pristine properties, to make sure
- that when the lock is locally set (=modification) it is not read only */
- if (props && svn_hash_gets(props, SVN_PROP_NEEDS_LOCK))
- {
- svn_wc__db_status_t status;
- svn_wc__db_lock_t *lock;
- SVN_ERR(svn_wc__db_read_info(&status, NULL, NULL, NULL, NULL, NULL, NULL,
- NULL, NULL, NULL, NULL, NULL, NULL, NULL,
- NULL, NULL, &lock, NULL, NULL, NULL, NULL,
- NULL, NULL, NULL, NULL, NULL, NULL,
- db, local_abspath,
+ SVN_ERR(svn_wc__working_file_writer_open(&file_writer,
+ temp_dir_abspath,
+ final_mtime,
+ props,
+ changed_rev,
+ changed_date,
+ changed_author,
+ lock != NULL,
+ status == svn_wc__db_status_added,
+ repos_root_url,
+ repos_relpath,
+ scratch_pool,
+ scratch_pool));
+
+ SVN_ERR(svn_stream_open_readonly(&src_stream, source_abspath,
scratch_pool, scratch_pool));
- if (!lock && status != svn_wc__db_status_added)
- SVN_ERR(svn_io_set_file_read_only(local_abspath, FALSE, scratch_pool));
- }
+ SVN_ERR(svn_stream_copy3(src_stream,
+ svn_wc__working_file_writer_get_stream(file_writer),
+ cancel_func, cancel_baton,
+ scratch_pool));
- if (use_commit_times)
- {
- if (changed_date)
- SVN_ERR(svn_io_set_file_affected_time(changed_date,
- local_abspath,
+ SVN_ERR(svn_wc__working_file_writer_install(file_writer, local_abspath,
scratch_pool));
- }
/* ### this should happen before we rename the file into place. */
if (record_fileinfo)
Modified: subversion/trunk/subversion/tests/cmdline/update_tests.py
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/update_tests.py?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/tests/cmdline/update_tests.py (original)
+++ subversion/trunk/subversion/tests/cmdline/update_tests.py Sat Feb 13
14:30:10 2021
@@ -6840,12 +6840,14 @@ def update_delete_switched(sbox):
svntest.actions.run_and_verify_update(wc_dir, None, None, expected_status,
[], False, sbox.ospath('A'), '-r', 0)
-@XFail()
def update_add_missing_local_add(sbox):
"update adds missing local addition"
sbox.build(read_only=True)
+ ### This used to insert an invalid workqueue item, but the issue vanished
+ ### when the update editor was changed. Annotate this line for more info.
+
# Note that updating 'A' to r0 doesn't reproduce this issue...
sbox.simple_update('', revision='0')
sbox.simple_mkdir('A')
Modified: subversion/trunk/subversion/tests/libsvn_subr/io-test.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/libsvn_subr/io-test.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/tests/libsvn_subr/io-test.c (original)
+++ subversion/trunk/subversion/tests/libsvn_subr/io-test.c Sat Feb 13 14:30:10
2021
@@ -1043,6 +1043,82 @@ test_install_stream_over_readonly_file(a
}
static svn_error_t *
+test_install_stream_set_read_only(apr_pool_t *pool)
+{
+ const char *tmp_dir;
+ const char *final_abspath;
+ svn_stream_t *stream;
+ svn_stringbuf_t *actual_content;
+ apr_finfo_t finfo;
+ svn_boolean_t is_read_only;
+
+ /* Create an empty directory. */
+ SVN_ERR(svn_test_make_sandbox_dir(&tmp_dir,
+ "test_install_stream_set_read_only",
+ pool));
+
+ final_abspath = svn_dirent_join(tmp_dir, "stream1", pool);
+
+ SVN_ERR(svn_stream__create_for_install(&stream, tmp_dir, pool, pool));
+ SVN_ERR(svn_stream_puts(stream, "stream1 content"));
+ SVN_ERR(svn_stream_close(stream));
+ svn_stream__install_stream_set_read_only(stream, TRUE);
+ SVN_ERR(svn_stream__install_stream(stream,
+ final_abspath,
+ TRUE,
+ pool));
+
+ SVN_ERR(svn_stringbuf_from_file2(&actual_content,
+ final_abspath,
+ pool));
+
+ SVN_TEST_STRING_ASSERT(actual_content->data, "stream1 content");
+
+ SVN_ERR(svn_io_stat(&finfo, final_abspath, SVN__APR_FINFO_READONLY, pool));
+ SVN_ERR(svn_io__is_finfo_read_only(&is_read_only, &finfo, pool));
+ SVN_TEST_ASSERT(is_read_only);
+
+ return SVN_NO_ERROR;
+}
+
+static svn_error_t *
+test_install_stream_set_affected_time(apr_pool_t *pool)
+{
+ const char *tmp_dir;
+ const char *final_abspath;
+ svn_stream_t *stream;
+ svn_stringbuf_t *actual_content;
+ apr_finfo_t finfo;
+
+ /* Create an empty directory. */
+ SVN_ERR(svn_test_make_sandbox_dir(&tmp_dir,
+ "test_install_stream_set_affected_time",
+ pool));
+
+ final_abspath = svn_dirent_join(tmp_dir, "stream1", pool);
+
+ SVN_ERR(svn_stream__create_for_install(&stream, tmp_dir, pool, pool));
+ SVN_ERR(svn_stream_puts(stream, "stream1 content"));
+ SVN_ERR(svn_stream_close(stream));
+ svn_stream__install_stream_set_affected_time(stream, 123456789);
+ SVN_ERR(svn_stream__install_stream(stream,
+ final_abspath,
+ TRUE,
+ pool));
+
+ SVN_ERR(svn_stringbuf_from_file2(&actual_content,
+ final_abspath,
+ pool));
+
+ SVN_TEST_STRING_ASSERT(actual_content->data, "stream1 content");
+
+ SVN_ERR(svn_io_stat(&finfo, final_abspath, APR_FINFO_MTIME, pool));
+ SVN_TEST_INT_ASSERT(finfo.mtime, 123456789);
+
+ return SVN_NO_ERROR;
+}
+
+static svn_error_t *
test_file_size_get(apr_pool_t *pool)
{
const char *tmp_dir, *path;
@@ -1254,6 +1330,10 @@ static struct svn_test_descriptor_t test
"test svn_stream__install_stream to long path"),
SVN_TEST_PASS2(test_install_stream_over_readonly_file,
"test svn_stream__install_stream over RO file"),
+ SVN_TEST_PASS2(test_install_stream_set_read_only,
+ "test svn_stream__install_stream_set_read_only"),
+ SVN_TEST_PASS2(test_install_stream_set_affected_time,
+ "test svn_stream__install_stream_set_affected_time"),
SVN_TEST_PASS2(test_file_size_get,
"test svn_io_file_size_get"),
SVN_TEST_PASS2(test_file_rename2,
Modified: subversion/trunk/subversion/tests/libsvn_wc/db-test.c
URL:
http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/libsvn_wc/db-test.c?rev=1886490&r1=1886489&r2=1886490&view=diff
==============================================================================
--- subversion/trunk/subversion/tests/libsvn_wc/db-test.c (original)
+++ subversion/trunk/subversion/tests/libsvn_wc/db-test.c Sat Feb 13 14:30:10
2021
@@ -635,7 +635,7 @@ test_inserting_nodes(apr_pool_t *pool)
1, TIME_1a, AUTHOR_1,
checksum,
NULL, FALSE, FALSE, NULL, NULL, FALSE, FALSE,
- NULL, NULL,
+ NULL, NULL, FALSE, NULL,
pool));
/* Create a new symlink node. */
