Author: danielsh Date: Thu Mar 24 23:37:13 2022 New Revision: 1899186 URL: http://svn.apache.org/viewvc?rev=1899186&view=rev Log: [in site/staging]
Add a rendered version of the issue #525 user guide, maintained in markdown on the pristines-on-demand-on-mwf branch, so we can point people not to the markdown source but to the rendered HTML. * docs/i525-user-guide.html: Rendered version of i525-user-guide.md. See keyword expansions within. . Please don't set svn:keywords on this file :) * docs/release-notes/1.15.html (#bare-working-copies): Point to it. Added: subversion/site/staging/docs/i525-user-guide.html (with props) Modified: subversion/site/staging/docs/release-notes/1.15.html Added: subversion/site/staging/docs/i525-user-guide.html URL: http://svn.apache.org/viewvc/subversion/site/staging/docs/i525-user-guide.html?rev=1899186&view=auto ============================================================================== --- subversion/site/staging/docs/i525-user-guide.html (added) +++ subversion/site/staging/docs/i525-user-guide.html Thu Mar 24 23:37:13 2022 @@ -0,0 +1,368 @@ +<p>This is a detailed user guide to the "i525pod" feature.</p> + +<p>$LastChangedDate: 2022-03-24 23:26:24 +0000 (Thu, 24 Mar 2022) $</p> + +<p>$LastChangedRevision: 1899185 $</p> + +<p>$URL: https://svn.apache.org/repos/asf/subversion/branches/pristines-on-demand-on-mwf/notes/i525/i525-user-guide.md $</p> + +<h1>Terminology</h1> + +<p>Place-holders:</p> + +<ul> +<li>"i525pod" stands for the name of the feature documented here, that is +as implemented on branch 'pristines-on-demand-on-mwf' on 2022-03-08. +(Not any other interpretation of what Issue #525 discusses.)</li> +<li>"bare" stands for the state of a WC in which the feature "i525pod" is +enabled, and so contains only some of the pristine copies.</li> +</ul> + +<p>Terminology:</p> + +<ul> +<li>"pristine copy" or "pristine" or "text base": a copy of a file's content +matching the corresponding base revision in the repository. For any file +type, not necessarily text format. Enables e.g. local diff and revert, +and delta update and commit. Stored in the WC metadata area. The term +herein refers only to file content, although Subversion also stores the +pristine copy of properties and of tree structure.</li> +<li>"hydrate" a pristine copy: to fetch the pristine copy from the +repository and store it in the WC metadata area.</li> +<li>"dehydrate" a pristine copy: to remove the pristine copy from the WC +metadata area, while remembering that it may be needed again.</li> +<li>"sync scope": the set of WC paths in which a Subversion operation will +check for pristines that need to be hydrated or dehydrated. This is a +superset of the pristines that the operation will actually need.</li> +<li>"operation": a high level Subversion operation, such as "diff" or +"merge" or "update"; e.g. a subcommand of the 'svn' program.</li> +</ul> + +<h1>"i525pod" User Guide</h1> + +<h2>Functional and Timing Differences</h2> + +<p>This section details the functional and timing differences when the +"i525pod" feature is used.</p> + +<p>In a WC where "i525pod" is enabled (see other sections for how), basic usage +differs from that found in previous versions of Subversion (1.14 in case of +doubt) in the following ways.</p> + +<p>Each of the following operations, that previously were <em>offline</em> operations, +will now contact the repository to "hydrate" pristine copies before +beginning its function, if (and only if) any pristines within the "sync scope" +are found to be locally modified and currently "dehydrated".</p> + +<ul> +<li><code>svn cat</code> (default case: base version)</li> +<li><code>svn diff</code> (default case: base against working)</li> +<li><code>svn resolve</code> (also conflicts resolver in <code>merge</code>, <code>update</code>)</li> +<li><code>svn revert</code></li> +</ul> + +<p>Notes on previously <em>offline</em> operations,:</p> + +<ul> +<li>Contacting the repository may require authentication.</li> +<li>If contact or authorization fails, the operation will error out and not +be available.</li> +<li>This contact may be needed as a result of a file being modified that is +not of interest in the current operation, as the "sync scope" is a +superset of the pristines that this operation will actually need.</li> +<li>The "hydrating" phase may take a long time, and (currently) gives no +progress feedback, before the operation begins its usual (previous) +behaviour.</li> +</ul> + +<p>[TODO: update that if we add progress feedback]</p> + +<p>Each of the following operations, that previously were <em>online</em> operations, +also will now require the same.</p> + +<ul> +<li><code>svn diff</code> (comparing repository to WC)</li> +<li><code>svn merge</code></li> +<li><code>svn switch</code></li> +<li><code>svn update</code></li> +<li><code>svn checkout --force</code> (similar to update)</li> +</ul> + +<p>Additional notes on previously <em>online</em> operations:</p> + +<ul> +<li>The "hydrating" phase requires its own connection and authentication to +the repository, which is not [currently] shared with the main part of +the operation. This may mean a password would have to be entered an +additional time, for example, depending on the configuration.</li> +<li>In some edge cases, there may be some difference in the outcome of the +operation. A possible example is if repository path authorization has +been withdrawn from a path that now needs to be hydrated; this +particular case is still under discussion in issue #????.</li> +</ul> + +<p>[TODO: check/eliminate the additional authentication? Issue filed?]</p> + +<h2>Disk Space Usage Differences</h2> + +<p>This section details the disk space usage differences when the feature is +used.</p> + +<p>Summary:</p> + +<ul> +<li>Without "i525pod" feature, a Subversion WC typically occupies +approximately twice the size of the working files, because there is a +pristine copy of every unique working file, plus some other metadata.</li> +<li>The "i525pod" feature enables a WC to occupy a disk space little greater +than the size of the working files, in cases where only a small +proportion (size wise) of the files are locally modified at any one +time.</li> +<li>Cases where the feature will not provide much disk space benefit +include: +<ul> +<li>where a large proportion of WC files are duplicates (because there was +only one pristine copy in the first place for each set of duplicates);</li> +<li>where the WC is arranged to contain only (predominantly) the files +that are to be modified in a given work flow.</li> +</ul></li> +</ul> + +<p>Initial WC size, from checkout/upgrade:</p> + +<ul> +<li>on a fresh checkout, no pristines are stored;</li> +<li>on enabling the feature in an existing WC (by WC upgrade), pristines are +removed for unmodified files;</li> +<li>[TODO] check: are pristines removed in fact for all files not just +unmodified files?</li> +</ul> + +<p>Size increase and decrease:</p> + +<ul> +<li>the operations listed in the 'Functional differences' section may grow +and/or shrink the disk space used, as detailed there;</li> +<li>operations such as editing a file (without Subversion's control), and +editing its Subversion properties, will not cause any change to the +pristine storage;</li> +<li>'commit' will remove the pristines for files it commits that had locally +modified content;</li> +</ul> + +<p>Disk Full:</p> + +<p>Failure modes when disk becomes full during an operation...</p> + +<ul> +<li>[TODO] Investigate.</li> +</ul> + +<h2>Support, Compatibility, Enabling</h2> + +<p>In brief:</p> + +<p>"i525pod" is an optional feature in Subversion 1.15. It can be enabled +separately for each WC. By default "i525pod" is not enabled and WCs remain +compatible with Subversion 1.8 through 1.14. To enable "i525pod" for a given +WC, check out or upgrade the WC to 1.15's format. A WC in +1.15's format is no longer compatible with older Subversion +clients.</p> + +<p>For details: see "Working Copy Format Upgrade and Compatibility" section.</p> + +<p>[TODO: update if/when we use a specific feature-enable flag.]</p> + +<h2>User Guide: Principle Of Operation</h2> + +<p>Quoting the original 'BRANCH-README':</p> + +<p>"The core idea is that we start to maintain the following invariant: only + the modified files have their pristine text-base files available on the + disk."</p> + +<p>When "i525pod" is enabled in a given WC, Subversion stores pristine copies +according to the following principle:</p> + +<ul> +<li>It stores the pristine copy of each file that is currently in a locally +modified state.</li> +<li>It does not store the pristine copy of each file that is not currently +in a locally modified state.</li> +<li>At certain points in its operations, Subversion checks the modified +state of certain files, and fetches (from the repository) or removes +their pristine copy accordingly if the state has changed.</li> +</ul> + +<p>Subversion updates the pristine storage to match this principle at certain +times. + - To get into the appropriate state at the beginning of the operation, we + walk through the current text-base info in the db and check if the + corresponding working files are modified. The missing text-bases are + fetched using the 'svn_ra' layer. The operations also include a final + step during which the no longer required text-bases are removed from + disk.</p> + +<ul> +<li>The operations that don't need to access the text-bases (such as "svn +ls" or the updated "svn st") do not perform this walk and do not +synchronize the text-base state.</li> +</ul> + +<p>Subversion updates the pristine storage to match this principle at certain +times.</p> + +<ul> +<li>At the beginning of any high level operation that may need to access +pristine copies [1], Subversion first checks for files [1] that are now +locally modified and whose pristine copies are not already present, and +connects to the repository and fetches them.</li> +<li>At the beginning and/or the end of those operations, Subversion checks +for files that are now unmodified, and removes their pristine copies. +That includes files that became unmodified before the operation, and, +for some operations (such as commit) also files that became unmodified +because of the action of the operation.</li> +<li>It does not matter how a file became modified or unmodified: whether a +Subversion operation caused that to be the case, or whether the user had +externally edited the file into that state some time before.</li> +<li>[1] See "Which operations?"</li> +<li>[2] See "Which files?" about the "sync scope".</li> +</ul> + +<p>The pristine storage state does not immediately change to match the +principle:</p> + +<ul> +<li>NOT whenever the user edits a file outside of Subversion's control,</li> +<li>NOR for files outside the "sync scope" of a given operation,</li> +<li>NOR during any Subversion operation other than those listed.</li> +</ul> + +<p>The definition of "locally modified" takes into account Subversion's local +"translation" options such as "keywords" and "eol-style". A working file +that differs from the repository copy only in keywords and/or EOL-style is +not regarded as locally modified. When Subversion needs to access a pristine +copy of such a file, Subversion makes a temporary pristine copy by +"detranslating" the working file. It may store this in temporary disk space +for the duration of the operation, but does not keep this indefinitely.</p> + +<h3>Which operations will "hydrate" or "dehydrate" pristines?</h3> + +<p>The operations deemed to need the pristine copies of locally modified files, +and which therefore "hydrate" (and "dehydrate") the pristine copies of +locally modified files, are:</p> + +<ul> +<li><code>H</code> <code>D</code> <code>svn cat</code> (default case: base version)</li> +<li><code>-</code> <code>D</code> <code>svn commit</code> (dehydrate only)</li> +<li><code>H</code> <code>D</code> <code>svn diff</code> (default case: base against working)</li> +<li><code>H</code> <code>D</code> <code>svn resolve</code> (also conflicts resolver in <code>merge</code>, <code>update</code>)</li> +<li><code>H</code> <code>D</code> <code>svn revert</code></li> +<li><code>H</code> <code>D</code> <code>svn switch</code></li> +<li><code>H</code> <code>D</code> <code>svn update</code></li> +<li><code>-</code> <code>D</code> <code>svn upgrade --compatible-version=1.15</code> (upgrade to 1.15's WC format enables "i525pod")</li> +</ul> + +<h3>Which files does Subversion "hydrate" or "dehydrate" ("sync scope")?</h3> + +<p>Which files does Subversion "hydrate" or "dehydrate", whenever an eligible +operation has the chance to do so?</p> + +<p>The files that a given operation "hydrates" and/or "dehydrates" are neither +all possible files in the working copy, nor the minimal subset necessary for +the particular operation.</p> + +<p>Not maximal:</p> + +<ul> +<li>It considers files to "hydrate" or "dehydrate" within one or more +sub-trees that encompass all the target paths passed to the operation. +Depending on the operation, this sub-tree may span anything from the +whole WC (typical for "update", for example), right down to one specific +file of interest, or even a directory containing no files at all.</li> +</ul> + +<p>Not minimal:</p> + +<ul> +<li>The operation in the end may not look at all the files in "sync scope": +for example, because of filtering options (such as <code>--depth</code>, +<code>--changelist</code>), or because the operation terminated early (for +example, <code>svn resolve</code>... and choose <code>quit</code>).</li> +<li>The operation in the end may not read the pristine copy of every file it +processes: for example, <code>svn diff --properties-only</code>.</li> +</ul> + +<h1>Working Copy Format Upgrade and Compatibility</h1> + +<h2>Summary</h2> + +<ul> +<li>"i525pod" is active when a given WC is in 1.15-compatible format.</li> +<li>To use "i525pod", use <code>svn checkout --compatible-version=1.15</code> +or <code>svn upgrade --compatible-version=1.15</code>.</li> +<li>Subversion 1.15 by default uses 1.8-compatible WC format, with "i525pod" +inactive in those WCs, the same as Subversion 1.8 through 1.14.</li> +<li>Subversion 1.14 and older cannot read or write a 1.15-compatible WC.</li> +<li>Working copies cannot be downgraded</li> +</ul> + +<h2>Details</h2> + +<p>To use "i525pod" in a given working copy (WC), that WC's metadata needs to be in a new 1.15-compatible format (also called WC format 32). You need a Subversion 1.15 client to create or use a WC in this format.</p> + +<p>You can either check out a working copy in this format:</p> + +<pre><code>svn checkout --compatible-version=1.15 +</code></pre> + +<p>or upgrade an existing working copy from a 1.8-compatible or older format to this format:</p> + +<pre><code>svn upgrade --compatible-version=1.15 +</code></pre> + +<p>That working copy:</p> + +<ul> +<li>becomes "bare" right away;</li> +<li>is no longer accessible by Subversion clients below version 1.15;</li> +<li>cannot be converted back to 1.8-compatible format.</li> +</ul> + +<p>[TODO] We might change this so that upgrading to 1.15-compatible format and enabling "i525pod" are separate steps and the latter is optional.</p> + +<p>Subversion 1.15 also supports, and uses by default, the working copy format that has been in use since Subversion 1.8 (format 31). A working copy in that format does not support "i525pod" and is never "bare".</p> + +<pre><code>svn checkout --compatible-version=1.8 +svn checkout # the same, as 1.8 is the default +</code></pre> + +<p>You do not need to use the new format for all your working copies. Subversion 1.15 can work with some working copies in 1.8-compatible format and others in 1.15-compatible format.</p> + +<h2>Upgrade / Downgrade</h2> + +<p>You can upgrade to 1.15-compatible format (WC format 32) with:</p> + +<pre><code>svn upgrade --compatible-version=1.15 +</code></pre> + +<p>from either</p> + +<ul> +<li>1.8-compatible format, which Subversion 1.15 can use; or</li> +<li>1.7-compatible or older format, which Subversion 1.15 cannot use but can upgrade.</li> +</ul> + +<p>By default "svn upgrade" upgrades a working copy to 1.8-compatible format. This is useful for upgrading a WC from the 1.7 and older formats so Subversion 1.8 and newer can use it.</p> + +<pre><code>svn upgrade +svn upgrade --compatible-version=1.8 # the same +</code></pre> + +<p>You CANNOT downgrade any working copy to an older format.</p> + +<p>(If you need a working copy in an older format than your current Subversion client supports, you would have to check out a working copy using an older Subversion client that supports the format you want to use.)</p> + +<h2>[TODO] Enable/disable "i525pod" in a 1.15-compatible WC</h2> + +<p>[TODO] Not yet implemented (2022-03-08).</p> Propchange: subversion/site/staging/docs/i525-user-guide.html ------------------------------------------------------------------------------ svn:eol-style = native Propchange: subversion/site/staging/docs/i525-user-guide.html ------------------------------------------------------------------------------ svn:mime-type = text/html Modified: subversion/site/staging/docs/release-notes/1.15.html URL: http://svn.apache.org/viewvc/subversion/site/staging/docs/release-notes/1.15.html?rev=1899186&r1=1899185&r2=1899186&view=diff ============================================================================== --- subversion/site/staging/docs/release-notes/1.15.html (original) +++ subversion/site/staging/docs/release-notes/1.15.html Thu Mar 24 23:37:13 2022 @@ -199,7 +199,8 @@ users. We'll cover those in this sectio <p class='todo'>Point to the "User Guide" in <a href="https://svn.apache.org/repos/asf/subversion/branches/pristines-on-demand-on-mwf/notes/i525/" -><tt>notes/i525/</tt></a>; ideally, to an HTML-rendered version thereof</p> +><tt>notes/i525/</tt></a>. Regenerate <a href="/docs/i525-user-guide.html" +><tt>/docs/i525-user-guide.html</tt></a> from the source markdown.</p> <p>All Subversion working copies require extra storage space in addition to the size of the checked out files.</p>