Author: stefan2
Date: Tue Mar 25 23:12:02 2014
New Revision: 1581587
URL: http://svn.apache.org/r1581587
Log:
Document a few bits in our 1.9 release notes.
* publish/docs/release-notes/1.9.html
(): Add sections for FSFS format 7, FSX and svnadmin improvements.
Modified:
subversion/site/publish/docs/release-notes/1.9.html
Modified: subversion/site/publish/docs/release-notes/1.9.html
URL:
http://svn.apache.org/viewvc/subversion/site/publish/docs/release-notes/1.9.html?rev=1581587&r1=1581586&r2=1581587&view=diff
==============================================================================
--- subversion/site/publish/docs/release-notes/1.9.html (original)
+++ subversion/site/publish/docs/release-notes/1.9.html Tue Mar 25 23:12:02 2014
@@ -30,8 +30,10 @@
</h2>
<ul>
- <li><a href="#dwim-feature"
- >### Some new feature</a></li>
+ <li><a href="#fsfs-improvements"
+ >FSFS improvements</a></li>
+ <li><a href="#fsx"
+ >FSX - A new experimental repository backend</a></li>
<li><a href="#enhancements"
>Many enhancements and bug fixes</a></li>
<li><a href="#issues"
@@ -98,11 +100,11 @@ and what impact these changes may have.<
<th>Minimum Repository</th>
<th>Notes</th></tr>
<tr>
- <td><a href="#dwim-feature">### some cool feature</a></td>
- <td>1.9</td>
- <td>any</td>
+ <td><a href="#FSX">FSX</a></td>
<td>any</td>
- <td></td></tr>
+ <td>1.9</td>
+ <td>1.9</td>
+ <td>Will not be compatible with 1.10</td></tr>
<tr>
<td colspan="5"><sup>1</sup>Reminder: when using the <tt>file://</tt>
repository access method, the Subversion program is both the client
@@ -226,16 +228,258 @@ for erroneous settings lest they will be
title="Link to this section">¶</a>
</h2>
-<div class="h3" id="dwim-feature">
-<h3>### DWIM option
- <a class="sectionlink" href="#dwim-option"
+<div class="h3" id="fsfs-improvements">
+<h3>FSFS improvements
+ <a class="sectionlink" href="#ffsfs-improvements"
+ title="Link to this section">¶</a>
+</h3>
+
+<div class="h4" id="fsfs-format7">
+<h4>Format bump
+ <a class="sectionlink" href="#fsfs-format7"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>The default repository format number is now 7. In contrast to previous
+releases, this one is a major overhaul with the general goal of I/O reduction.
+Major changes include:
+</p>
+
+<ul>
+ <li>Revision content is no longer addressed by physical location (offset)
+ within the rev file but by logical item numbers. Two index files provide
+ the necessary mapping information. You can no longer manipulate the
+ rev / pack file contents manually.</li>
+ <li><tt>svnadmin pack</tt> reorders the revision data such that related
+ information is put next to each other and everything can often be read
+ with a single I/O.</li>
+ <li>Block reads will now fetch data in larger blocks of configurable size
+ (64kB by default) and cache all their contents. This eliminates much of
+ the OS overhead.</li>
+ <li>Explicit flags for <tt>svn:mergeinfo</tt> changes speed up
+ mergeinfo-related operations like <tt>svn log -g</tt>.</li>
+ <li>Full checksum coverage of all revision data, including meta data and
+ structural information.</li>
+ <li>Additional index files for every revision or pack file. This doubles
+ the number of files in an unpacked repository. Therefore, pack
regularly.
+ </li>
+ <li>Support commits while packing is in progress.</li>
+</ul>
+
+<p>As with earlier releases, you may simply run <tt>svnadmin upgrade</tt> on
+your repository. The new format will then only be used for revision starting
+with the next shard (usually next multiple of 1000). For best performance,
+however, it is strongly recommended to perform a full
+<a
href="http://svnbook.red-bean.com/en/1.9/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate.svnadmin";>dump
/ load</a> cycle.
+</p>
+
+<div class="notice">
+ <p>Make sure to run <tt>svnadmin pack</tt> on your format 7 repositories at
+ regular intervals. Otherwise, they may perform worse than previous formats.
+ </p>
+</div>
+
+</div> <!-- fsfs-format7 -->
+
+<div class="h4" id="format7-options">
+ <h4>Format 7 tuning options
+ <a class="sectionlink" href="#format7-options"
+ title="Link to this section">¶</a>
+ </h4>
+
+<p>New configuration options are available for tuning read block sizes (format
+7 only). They may be changed at any time without causing consistency issues
+with existing revisions. Changing them is rarely necessary, but may result in
+slightly improved performance with your specific storage backend or when
+dealing with multi-TB repositories. See the commentary in <tt>fsfs.conf</tt>
+for more information.
+
+<pre>
+ [io]
+ block-size = 256
+ l2p-page-size = 30000
+ p2l-page-size = 16384
+</pre>
+
+</div> <!-- fsfs-format7 -->
+
+<div class="h4" id="fsfs-compression">
+<h4>Zlib compresssion is now optional
+ <a class="sectionlink" href="#fsfs-compression"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>FSFS uses a combination of two methods to reduce on-disk data size. First,
+we determine the changes (delta) against some previous version of the same
file.
+This process can be controlled by fsfs.conf settings since 1.8. The result
+would then be compressed using ZIP/deflate. That relatively costly operation
+limits commit speeds of large incompressible files. You may now specify the
+compression level in fsfs.conf or disable the ZIP step entirely:
+</p>
+
+<pre>
+ [deltification]
+ compression-level = 0
+</pre>
+
+</div> <!-- fsfs-compression -->
+
+<div class="h4" id="fsfs-defaults">
+<h4>Changed deltification defaults
+ <a class="sectionlink" href="#fsfs-defaults"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>Directory and property deltification are now enabled by default for FSFS
+repositories in 1.6 format or newer. Various optimization were added to delta
+creation code to minimize the read amplification effect and to provide a net
+speedup when using deltification vs. not using it. Don't disable
+<a href="#fsfs-caching">txdelta caches</a> when using directories
+deltification.</p>
+
+<p>If you don't want Subversion 1.9 to deltify properties and directories in
+older format repositories, modify or add the following settings in the
+respository's fsfs.conf file:
+</p>
+
+<pre>
+ [deltification]
+ enable-dir-deltification = false
+ enable-props-deltification = false
+</pre>
+
+</div> <!-- fsfs-defaults -->
+
+<div class="h4" id="fsfs-caching">
+<h4>Changes to the caching
+ <a class="sectionlink" href="#fsfs-caching"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>The caching logic has been enhanced to cope with the extra load caused by
+the access indirection introduced in FSFS format 7. While most of this simply
+means more net speed, some changes may affect your configuration settings.</p>
+
+<ul>
+ <li>Fulltext and txdelta caching are now enabled by default for all servers
+ as well as local repository access. Revision property caching is still
+ opt-in.</li>
+ <li>Directories 4x as large as in 1.8 can now be cached.</li>
+ <li>Caching is quasi-perfect on short time scales (requesting data equal or
+ less than approx. 10% of the cache size). This greately improves
+ performance when e.g. many clients check out the same project.</li>
+ <li>Long-term caching is now priority-based with text deltas having the
lowest
+ priority. This results in slower but more reliable heating up, i.e. it
+ takes multiple similar requests until all frequently used data is kept in
+ cache, but it will reach this point eventually while older release might
+ not. You may disable txdelta caching to reduce cache load and heat caches
+ more quickly.</li>
+</ul>
+
+</div> <!-- fsfs-caching -->
+
+</div> <!-- fsfs-improvements -->
+
+<div class="h3" id="fsx">
+<h3>FSX - A new experimental repository backend
+ <a class="sectionlink" href="#fsx"
title="Link to this section">¶</a>
</h3>
-<p><span style="color: red"><i>### Command line now supports the
-Do What I Mean (TM) flag.</i></span></p>
+<div class="h4" id="fsx-overview">
+<h4>Overview
+ <a class="sectionlink" href="#fsx-overview"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>Since its inception 10 years ago, FSFS has been improved greately and the
+<a href="#fsfs-format7">improvements provided with format 7</a> are another
+major step forward. However, there are limits to what can be done in a
+meaningful and backward compatible way. FSX has been designed to overcome
+these limitations. Here some of the intended improvements:
+</p>
+
+<ul>
+ <li>90% reduction in metadata overhead.</li>
+ <li>Efficient handling of very large files.</li>
+ <li>Higher overall compression rates, in particular for office-style
+ documents.</li>
+ <li>Information required for log and merge operations is more readily
+ available.</li>
+ <li>O(1) handling of large directories.</li>
+ <li>Versioned revision properties.</li>
+ <li>Partitionable storage.</li>
+ <li>Arbitrary meta data storage and indexing facilities.</li>
+</ul>
+
+<p>Development of FSX has been started as a fork of FSFS to guarantee
+functional completeness and testability. The current code still contains
+remnants of FSFS legacy code and not all of the above improvements have been
+implemented, yet. Later releases will close the gap and remove trasitional
+code.
+</p>
+
+<div class="notice">
+ <p><span style="color: red"><b>WARNING:</b></span> FSX in NOT production
+ ready. DO NOT USE IT - unless you have read this section carefully and
+ agreed to the limitations that come with FSX in Subversion 1.9.</p>
+</div>
+
+</div> <!-- fsx-overview -->
+
+<div class="h4" id="fsx-usage-scennarios">
+<h4>Usage scennarios
+ <a class="sectionlink" href="#fsx-usage-scennarios"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>FSX is nowhere near as stable and reliable as FSFS. Furthermore, there will
+be no support for 1.9-style FSX repositories in 1.10
+(see <a href="#fsx-incompatibility">Incompatibility</a>). Think, therefore,
+of all data in a FSX repository as being potentially corrupt. Due to its
+superior performance and storage efficiency, however, there are two use-cases
+where FSX might be a vaiable option:</p>
+
+<ul>
+ <li>Running analysis or reporting tools whose output does not need to be 100%
+ reliable. Use FSX as a high-speed data source in a read-only
mirror.</li>
+ <li>Investigate how FSX may fit into your future infrastructure. How well
+ does it perform with your data set concerning disk, I/O, RAM and CPU
+ usage?</li>
+</ul>
+
+<p></p>
+
+<div class="notice">
+ <p>If you experiment with FSX, please let us know of your findings. This
+ is your chance to get your use-case covered before storage format and
+ principles of operation are set in stone.</p>
+</div>
-</div> <!-- dwim-option -->
+</div> <!-- fsx-usage-scennarios -->
+
+<div class="h4" id="fsx-incompatibility">
+<h4>Incompatibility
+ <a class="sectionlink" href="#fsx-incompatibility"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>The FSX code and storage representation is incomplete with respect to
+the feature set that its developers have in mind. For as long as it keeps
+its experimental status, there will be neither forward nor backward
+compatibility between FSX repositories of different Subversion feature
releases.
+You may use <a
href="http://svnbook.red-bean.com/en/1.9/svn.reposadmin.maint.html#svn.reposadmin.maint.migrate.svnadmin";>dump
and load</a> to upgrade an
+FSX repository from one release to another.</p>
+
+<p>Dump and load is also the only upgrade path between FSFS and FSX.</p>
+
+<p>Finall, there is no guarantee that FSX will eventually be released at all.
+That means even if you find it useful in your specific environment, we might
+decide to remvove FSX entirely from future releases.</p>
+
+</div> <!-- fsx-incompatibility -->
+
+</div> <!-- fsx -->
</div> <!-- new-features -->
@@ -248,15 +492,54 @@ Do What I Mean (TM) flag.</i></span></p>
<!-- Don't need to highlight every bugfix, just major ones which aren't in
any patch release. -->
-<div class="h3" id="new-lorem-ispum">
-<h3>### Introducing blind text
- <a class="sectionlink" href="#new-lorem-ispum"
- title="Link to this section">¶</a>
+<div class="h3" id="svnadmin-improvements">
+<h3><tt>svnadmin</tt> changes and improvements
+ <a class="sectionlink" href="#svnadmin-improvements"
+ title="Link to this section">¶</a>
</h3>
-<p><span style="color: red"><i>### Change this to non-random
text.</i></span></p>
+<div class="h4" id="svnadmin-pack">
+<h4><tt>svnadmin pack</tt>
+ <a class="sectionlink" href="#svnadmin-pack"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>Packing now takes a cache size parameter (<tt>-M</tt>) which is needed for
+efficient operation on <a href="#fsfs-format7">FSFS format 7 repositories</a>.
+<p>
+
+<pre>
+ $ svnadmin pack -M 1000 /path/to/repository
+</pre>
+
+<p></p>
+
+<div class="notice">
+ <p>Packing is paramount to FSFS format 7 performance; do it often. To make
+ this feasible, packing FSFS format 7 repositories will also no longer block
+ commits for the whole time of its execution. Instead, individual commits may
+ now get delayed by much shorter periods of time, usually by less than one
+ second.</p>
+</div>
+
+</div> <!-- svnadmin-pack -->
+
+<div class="h4" id="svnadmin-verify">
+<h4><tt>svnadmin verify</tt>
+ <a class="sectionlink" href="#svnadmin-verify"
+ title="Link to this section">¶</a>
+</h4>
+
+<p>Verification will terminate by default once a problem has been found. The
+new <tt>--keep-going</tt> option instructs svnadmin to continue with the next
+revision such that multiple issues may be found in a single run. If a revision
+has multiple issues and depending on the verification logic, still only the
+first problem may be reported.
+<p>
+
+</div> <!-- svnadmin-verify -->
-</div> <!-- new-lorem-ispum -->
+</div> <!-- svnadmin-improvements -->
</div> <!-- enhancements -->
