Dear Wiki user, You have subscribed to a wiki page or wiki category on "Subversion Wiki" for change notification.
The "MtimePreservation" page has been changed by RickYorgason: http://wiki.apache.org/subversion/MtimePreservation?action=diff&rev1=6&rev2=7 Comment: Integrated suggestions, answered questions, added new 'Rules' and 'Auto-props deficiencies' sections, and various editing changes. ---- = Design: MTime Preservation = + Nearly every modern operating system associates a "modified time" with each file on the system, updating it whenever the file is modified. This metadata can be very important, but Subversion currently offers no way to preserve it. - {{{#!wiki comment - [JAF] Can you add a section describing the overall rules or invariants of the behaviour? I'm thinking particularly of the way that you've defined that 'checkout' does not add a text-time prop even where 'use-text-times' says there should be one, but rather 'commit' adds it. Therefore it looks like there may be a general rule in this design, something like: - * The text-time prop is not necessarily expected to be present during the life of the node in the WC, and so (I guess) when it ''is'' present it's not necessarily expected to be up-to-date. The svn:text-time is only relevant when the node is in the repository and at the moment of going into or coming out of the repo. + This design proposes adding a new 'svn:text-time' property to store the modified time of each file or directory in UTC time using the format 'YYYY-MM-DDTHH:MM:SS.UUUUUU'. The name of this property was chosen because it is already used in an identical manner by FSVS and svntar. + == Rules == + * The svn:text-time property is only relevant when the node is in the repository and at the moment of going into or out of the repository. + * Conceptually, svn:text-time is only updated when a node is going ''into'' the repository, and mtime is only updated when a node is coming ''out of'' the repository. + * When mtime changes locally, svn:text-time is not expected to remain up-to-date; it merely indicates that the svn:text-time property should be updated during the next commit. + * The presence of svn:text-time on a node indicates that mtime preservation is expected for that node. It may be blank, indicating that it should be updated at the next commit. In this way, auto-props can be used to specify which nodes should preserve mtime. + * If svn:text-time differs from BASE.svn:text-time, this counts as a local modification, as always. + * If svn:text-time differs from mtime, this also counts as a local modification, even for mtime-only modifications. + * Note: This is ''not'' true if mtime-usage is set to 'now' or 'commit-time'. See the 'commit' operation for details. + * Not all operating systems can support the resolution of the svn:text-time format specified. When setting mtime, the time should always be rounded down to the nearest supported resolution first. When comparing mtime against svn:text-time, svn:text-time should always be rounded down first. + * As svn:text-time is managed by Subversion, it should never conflict, and will always auto-resolve as necessary. - I'm not sure I got that rule 100% right but it looks like there is some such rule, and we should be explicit about the rules so that we can decide how each individual command should behave in all its variations to remain in agreement with the rules. - }}} - - Nearly every modern operating system associated a "modified time" with each file on the system, updating it whenever the file is modified. This metadata can be very important, but Subversion currently offers no way to preserve it. - - This design proposes two new properties: - - svn:text-time:: This property stores the modified time of each file or directory in UTC time using the format 'YYYY-MM-DDTHH:MM:SS.UUUUUU'. It is automatically generated by Subversion during commit. The name of this property was chosen because it is already used in an identical manner by FSVS and svntar. - svn:use-text-times:: The user defines this property on a directory as a list of file types for which svn:text-time should be applied. Its syntax is identical to svn:ignore. - - {{{#!wiki comment - [JAF] Here you have two things interacting to specify whether text-time is used: the presence of svn:text-time property on the node, and the matching of svn:use-text-times on its parent dir. They're both specified by the repository, but affect only the behaviour of a user's local copy. By contrast, 'use-commit-times' is a user-specified option (in user config file or cmd line). - - As we have the 'auto-props' mechanism already, I think we should rely on auto-props to add the 'svn:text-time' property to new files initially, and not invent a new way of doing that. Auto-props could set it with an empty value, and then the rest of the behaviour would fill in a valid value some time later. - - Then perhaps you could simplify this proposal, with the behaviour just depending on whether a svn:text-time property is present but not on svn:use-text-times. - - Auto-props is rather limited and would really benefit from being upgraded to e.g. be settable on the parent dir, like your 'svn:use-text-times', but that's a separate issue. - - In the cases where svn:text-time is supposed to be enabled for a given path, but the current proposal calls for the prop to be deleted in order to signal that it is not up to date at the moment, we could set it to an empty value instead. - }}} == Definitions == mtime:: Modified time. In this specification, mtime will always refer to the modified time of a file as stored by the operating system. Under this specification, Subversion will sometimes alter the mtime to equal the value stored in svn:text-time. stime:: System time. The current time according to the operating system at the time of the operation in question; aka now(). ctime:: Commit time. The timestamp currently stored by Subversion indicating when the file or directory was committed. - == Interaction with use-commit-times == - Wherever use-commit-times conflicts with this specification, use-commit-times takes precedence. This will be defined in more detail in the following operations. + == mtime-usage == + A new client-side configuration option named 'mtime-usage' should be introduced to specify how Subversion treats a node's mtime during an update operation. It can be one of the following options: + * text-time-or-now (default; sets mtime=svn:text-time if possible, else sets mtime=stime) + * text-time-or-commit-time (sets mtime=svn:text-time if possible, else sets mtime=ctime) + * now (sets mtime=stime; identical to current default behaviour) + * commit-time (sets mtime=commit-time; identical to use-commit-times) + mtime-usage obsoletes use-commit-times. If mtime-usage is unset and use-commit-times is true, the behaviour is identical to if mtime-usage was set to commit-time. - {{{#!wiki comment - [JAF] I think an API user should be able to choose the operating mode from: - * mtime = text-time on paths where defined, else commit-time - * mtime = text-time on paths where defined, else 'now' - * mtime = commit time - * mtime = 'now' - This can be expressed as adding a "use text-times where defined?" option that overrides the use-commit-times option. - - The first of these is not in your proposal. We need not necessarily provide all those options to the user of 'svn', but we should think of the design in those terms. - }}} == Operations == * Any operations not listed are presumed to be unaffected by this proposal. - * For the sake of being succinct, 'svn:' will be elided from 'svn:text-time' and 'svn:use-text-times' in the following operations. + * For the sake of being succinct, 'svn:' will be elided from 'svn:text-time' in the following operations. - * use-commit-times is used here as a boolean variable. It is true if the user has set use-commit-times=yes in their configuration. * The operations below apply to individual files or directories. For operations that support multiple nodes, the operation is simply repeated for each node. === svn commit === - * If use-text-times matches and (mtime differs from text-time or text-time is undefined), text-time = mtime + * If local modifications exist and text-time is defined, text-time = mtime + * Note: local modifications include mtime-only modifications iff mtime-usage is set to one of the two text-time-or- options. Otherwise every update operation would cause a local modification. - {{{#!wiki comment - [JAF] If use-text-times does ''not'' match and text-time is defined, then delete text-time? I'm thinking your proposal seems to have Subversion (rather than the user directly) in control of creating and deleting the text-time prop, at least in most cases (this should be specified in the Rules section), and it would be inconsistent if it isn't deleted by Subversion in this case. - }}} - - {{{#!wiki comment - [JAF] If mtime differs from text-time and that's the only local modification, do we still commit it or should we ignore this? - }}} + === svn update === + (The following logic is applied after the file has been updated.) + * If mtime-usage == now, mtime = stime + * Else if mtime-usage == commit-time, mtime = ctime + * Else if text-time is defined: + * If no local modifications exist, mtime = text-time + * Else, BASE.text-time = text-time and mtime = stime + * Else: + * If text-time was defined locally, delete it. + * If mtime-usage == text-time-or-now, repeat assuming mtime-usage == now + * Else if mtime-usage == text-time-or-commit-time, repeat assuming mtime-usage == commit-time === svn checkout === + * Identical to an update from r0 - * If use-commit-times, mtime = ctime - * Else if use-text-times matches and text-time is defined, mtime = text-time - * Else, mtime = stime - - (Checkout is implemented as update from r0 --philip) - - (What happens when mtime cannot represent text-time? We try to set mtime, get some mtime' that is slightly different, store mtime' in BASE.last-mode-time, and then compare mtime against last-mod-time to determine if the mtime has changed. --philip) - - === svn update === - * If use-commit-times, mtime = ctime - * Else if use-text-times matches: - * If local modifications exist, mtime = stime and text-time is deleted. - * (This is to prevent mtime-only conflicts; text-time will be reapplied in the next commit.) - * Else if text-time is defined, mtime = text-time - * Else, mtime = stime - - (Deleting text-time looks wrong, that's a property mod. Better to update BASE.text-time but not attempt to set mtime. --philip) - - {{{#!wiki comment - [JAF] "If local mods exist..." before or after the update? Usually if mods exist before, then mods exist after, but not always. - }}} === svn merge === * If no conflicts occur, behaviour is identical to svn update. + * Else, no changes are made to either mtime or text-time. - * Else if use-text-times matches, mtime=stime and text-time is deleted. - * (This is to avoid mtime-only conflicts; on commit, text-time will be reapplied.) === svn resolve === - * The rules above should ensure that, if a conflict occurred, the text-time property was deleted. * If (resolved with base, mine-full, or theirs-full) and the file chosen has text-time defined, text-time is copied and mtime=text-time. === svn copy === - * text-time is not copied. - - (That doesn't sound right, it makes cheap copies into expensive copies. An URL copy will preserve the property so a WC copy should do the same. --philip) + * Identical to current behaviour: + * In WC copies, mtime = stime + * text-time is preserved. === svn rename === + * Identical to current behaviour: - * mtime is unchanged. + * mtime is unchanged. - * text-time is preserved. + * text-time is preserved. - * (This may not be possible until true-renames are supported. Until that tile, treating a rename as a copy+delete is acceptable.) - == Setting and Comparing Dates == - Not all operating systems can support the resolution of the svn:text-time format specified. When setting mtime, the time should always be rounded down to the nearest supported resolution first. When comparing mtime against text-time, text-time should always be rounded down first. + === svn revert === + * Following the revert, mtime = now, commit-time, or text-time, as defined by mtime-usage. == Use cases == === Control cases === @@ -119, +84 @@ ==== make ==== The build system 'make' defines a specific set of output files for a specific set of input files. Whenever the output files are built, their mtime is set to stime. Whenever rebuilding, it checks to see if mtime of the the input files is newer than the mtime of the output files and rebuilds them if that's the case. - For this case to work, the output files must not be under version control, and use-text-times must not match the input files. + Implementation: + * The output files must not be under version control. + * The input files must not have text-time set. ==== Log files ==== Log files are often opened and resaved, even when nothing is written to them. This is not expected to count as a modification. - For this case to work, use-text-times must not match the log files. + Implementation: + * text-time must not be set on the log files. ==== Compiler-generated files ==== When building software, the compiler may be configured to output a compiler-generated file. These often get re-generated with identical output, only modifying the mtime. This is not expected to count as a modification. - For this case to work, use-text-times must not match the output files. To reduce the number of unnecessary generations, it also should not match the input files. + Implementation: + * text-time must not be set on the output files. + * To reduce the number of unnecessary generations, it should also not be set on the input files. === Motivating use cases === These are some of the reasons that users require this feature. @@ -137, +107 @@ ==== Image compression ==== In this scenario, the user stores uncompressed images in one folder and then runs a script to compress the images and dump them in another folder. The compression tool can generate slightly different bitstreams on different computers. Both folders are required to be under version control, and should avoid recompressing images that have not changed. - ==== Image compression ==== - In this scenario, the user stores uncompressed images in one folder and then runs a script to compress the images and dump them in another folder. The compression tool can generate slightly different bitstreams on different computers. Both folders are required to be under version control, and should avoid recompressing images that have not changed. - + Implementation: + * Auto-props should be set up to add text-time to both the input and output image files. + * Any existing files must have text-time added manually. - For this case to work, use-text-times must match both the input and output files. The script must also set the mtime of each output file to the mtime of the matching input file and only recompress when the mtimes differ. + * The script must set the mtime of each output file to the mtime of the matching input file and only recompress when the mtimes differ. ==== Subversion as a backup tool ==== In this scenario, Subversion is primarily used to detect modified files and upload them to an off-site server; version control is only used in rare cases, such as a file being backup up after it was corrupted. - For this case to work, use-text-times should be set to * recursively. All unversioned files must be added whenever committing, and unversioned directories must also have their use-text-times property set to *. The latter step could be eliminated if something similar to Issue #1054 (Specify default properties on a per repository or per directory basis) is implemented. + Implementation: + * Auto-props should be set up to add text-time to all files. + * Any existing file must have text-time added manually. ==== Finding documents by publication date ==== Some users maintain their mtime to refer to the publication date of a file. It should be possible to locate a document based on its mtime, even if that document was committed weeks after it was modified. - For this case to work, use-text-times must match the document types. In an unmixed-environment, matching '*' would be sufficient. + Implementation: + * Auto-props should be set up to add text-time to all document files. + * In an un-mixed environment, matching '*' would be sufficient. + * Any existing files must have text-time added manually. ==== Web server ==== Web servers expose the mtime as the 'Last-Modified-Date' header in http requests, which is used in a variety of ways. Clients use it to determine whether or not the file needs to be re-cached, reducing bandwidth for the server. Search engines use it to determine if the page needs to be reindexed, or when the user requests to find a page matching a certain date range. Users use it when they need to determine if the information on a page is obsolete. Web servers also offer index pages, which show all the files in a directory and allow the user to sort by mtime. - For this to work, use-text-times must match the documents. Web sites are often a mixed-environment, so some file types may have to be left out of use-text-times to ensure tools like 'make' function properly. + Implementation: + * Auto-props should be set up to add text-time to all document files. + * Web sites are often mixed environments, so some file types may have to be excluded to ensure tools like 'make' continue to work. + * Any existing document files must have text-time added manually. + + == Auto-props deficiencies == + This proposal relies heavily on auto-props, and as a result, betrays some of the deficiencies extant in the auto-props system. + + Most of the motivating use cases would benefit from having all users share the same auto-props. TortoiseSVN users can do this today using tsvn:autoprops, but there is no standard way to accomplish this. See [[http://subversion.tigris.org/issues/show_bug.cgi?id=1974|Issue 1974: server-side config which 'broadcasts' to clients]] + + Auto-props does not apply to directories. This means that directories cannot preserve mtimes without manually adding the svn:text-time property to all directories. See [[http://subversion.tigris.org/issues/show_bug.cgi?id=1989|Issue 1989: make auto-props affect directories too]] + + Auto-props only applies properties to newly imported/added files, so any existing files will have to have svn:text-time added manually when adopting the new feature. One solution would be to extend auto-props to allow the user to specify 'important' rules that should be applied on commit, rather than just on import/add. Another solution would be to add a command to svn that recursively applies the auto-props rules to a selected directory. I am not aware of any filed issues that request these features. == References == * [[http://subversion.tigris.org/issues/show_bug.cgi?id=1256|Issue 1256: Ability to preserve last modification time (mtime) of files under version control]] * [[http://svn.haxx.se/dev/archive-2012-02/0440.shtml|@dev discussion]] * [[http://svn.haxx.se/dev/archive-2010-02/0127.shtml|Previous proposal by Edmund Wong]] - * [[http://subversion.tigris.org/issues/show_bug.cgi?id=1054|Issue 1054: Specify default properties on a per repository or per directory basis]] ---- ''The content of this page was written by Rick Yorgason: see <''http://svn.haxx.se/dev/archive-2012-02/0492.shtml>.
