Author: stefan2
Date: Thu Aug 8 16:33:13 2013
New Revision: 1511856
URL: http://svn.apache.org/r1511856
Log:
On the log-addressing branch: Enable log->phys address translation.
That means svn_fs_fs__item_offset is now routed through l2p index lookup.
Data read code needs to adapt for the lack of the "revision header"
in logical addressing mode. Instead, root node and changed paths list
have fixed logical addresses (item_index) now.
The main change is to the transaction handling as this has to write
to the proto-index file whenever we add an item to the proto-rev file,
i.e. when it's being assigned to its final offset. For that, we simply
divert the assignment of offsets to a new allocate_item_index() function
and call the new store_l2p_index_entry() in other places.
One critical point is the transition between physical and logical
addressing mode. The switch-over point is known in advance and all
transactions based on the last phys. addressing revision will use
logical addressing. Transactions that based on older revisions
use physical addressing and cannot be committed beyond the switch-
over point (cause SVN_ERR_FS_TXN_ADDRESSING_MODE_OUT_OF_DATE).
Packing will not work until the reverse index (phys->log) is available.
Verififaction and hotcopying need code changes well.
Also, dbg_log_access() is not updated in this commit as the code would
be transient.
* subversion/include/svn_error_codes.h
(SVN_ERR_FS_TXN_ADDRESSING_MODE_OUT_OF_DATE): define new error code
* subversion/libsvn_fs_fs/fs.h
(PATH_TXN_ITEM_INDEX): define yet another file name
* subversion/libsvn_fs_fs/util.h
(svn_fs_fs__item_offset): remove declaration here
(svn_fs_fs__path_txn_item_index): declare yet another file name getter
* subversion/libsvn_fs_fs/util.c
(svn_fs_fs__item_offset): remove code here
(svn_fs_fs__path_txn_item_index): implement
* subversion/libsvn_fs_fs/index.h
(svn_fs_fs__item_offset): declare here instead of utils.h
* subversion/libsvn_fs_fs/index.c
(svn_fs_fs__item_offset): re-implement using index lookup where needed
* subversion/libsvn_fs_fs/id.h
(svn_fs_fs__id_create_root): declare new ID constructor using the
fact that the root has a fixed ID
* subversion/libsvn_fs_fs/id.c
(svn_fs_fs__id_create_root): implement it
* subversion/libsvn_fs_fs/cached_data.c
(SVN_FS_FS__ITEM_TYPE_ANY_REP,
SVN_FS_FS__ITEM_TYPE_NODEREV,
SVN_FS_FS__ITEM_TYPE_CHANGES): remove here as index.h defines these
(svn_fs_fs__rev_get_root,
svn_fs_fs__get_changes): use fixed addresses in in log addressing mode
* subversion/libsvn_fs_fs/fs_fs.c
(write_revision_zero): use different r0 templates in log addressing mode
* subversion/libsvn_fs_fs/transaction.c
(store_l2p_index_entry,
allocate_item_index): new generic item_index handling functions that
work for log and phys addressing alike
(rep_write_contents_close): call the new helpers
(write_hash_rep,
write_hash_delta_rep): ditto; also take the item type as parameter
as a preparation for later changes
(write_final_rev): update caller; write addressing mode-dependent parts
(write_final_changed_path_info): write l2p index entry as well,
requires extra parameter
(commit_body): update caller;
ensure correct addressing mode switch-over;
write addressing mode-dependent parts
Modified:
subversion/branches/log-addressing/subversion/include/svn_error_codes.h
subversion/branches/log-addressing/subversion/libsvn_fs_fs/cached_data.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs.h
subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs_fs.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.h
subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.h
subversion/branches/log-addressing/subversion/libsvn_fs_fs/recovery.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/transaction.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.c
subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.h
Modified:
subversion/branches/log-addressing/subversion/include/svn_error_codes.h
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/include/svn_error_codes.h?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/include/svn_error_codes.h
(original)
+++ subversion/branches/log-addressing/subversion/include/svn_error_codes.h Thu
Aug 8 16:33:13 2013
@@ -836,6 +836,11 @@ SVN_ERROR_START
SVN_ERR_FS_CATEGORY_START + 58,
"Index files are inconsistent.")
+ /** @since New in 1.9. */
+ SVN_ERRDEF(SVN_ERR_FS_TXN_ADDRESSING_MODE_OUT_OF_DATE,
+ SVN_ERR_FS_CATEGORY_START + 59,
+ "Transaction addressing mode is out of date")
+
/* repos errors */
SVN_ERRDEF(SVN_ERR_REPOS_LOCKED,
Modified:
subversion/branches/log-addressing/subversion/libsvn_fs_fs/cached_data.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/cached_data.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/cached_data.c
(original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/cached_data.c
Thu Aug 8 16:33:13 2013
@@ -30,6 +30,7 @@
#include "fs_fs.h"
#include "id.h"
+#include "index.h"
#include "low_level.h"
#include "pack.h"
#include "util.h"
@@ -43,12 +44,6 @@
#define SVN_FS_FS__LOG_ACCESS
*/
-/* Data / item types.
- */
-#define SVN_FS_FS__ITEM_TYPE_ANY_REP 1 /* item is a representation. */
-#define SVN_FS_FS__ITEM_TYPE_NODEREV 2 /* item is a noderev */
-#define SVN_FS_FS__ITEM_TYPE_CHANGES 3 /* item is a changed paths list */
-
/* When SVN_FS_FS__LOG_ACCESS has been defined, write a line to console
* showing where REVISION, ITEM_INDEX is located in FS and use ITEM to
* show details on it's contents if not NULL. To support format 6 and
@@ -470,30 +465,37 @@ svn_fs_fs__rev_get_root(svn_fs_id_t **ro
apr_pool_t *pool)
{
fs_fs_data_t *ffd = fs->fsap_data;
+ SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
- apr_file_t *revision_file;
- apr_off_t root_offset;
- svn_fs_id_t *root_id = NULL;
- svn_boolean_t is_cached;
+ if (svn_fs_fs__use_log_addressing(fs, rev))
+ {
+ *root_id_p = svn_fs_fs__id_create_root(rev, pool);
+ }
+ else
+ {
+ apr_file_t *revision_file;
+ apr_off_t root_offset;
+ svn_fs_id_t *root_id = NULL;
+ svn_boolean_t is_cached;
- SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
- SVN_ERR(svn_cache__get((void **) root_id_p, &is_cached,
- ffd->rev_root_id_cache, &rev, pool));
- if (is_cached)
- return SVN_NO_ERROR;
+ SVN_ERR(svn_cache__get((void **) root_id_p, &is_cached,
+ ffd->rev_root_id_cache, &rev, pool));
+ if (is_cached)
+ return SVN_NO_ERROR;
- SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev, pool));
- SVN_ERR(get_root_changes_offset(&root_offset, NULL, revision_file, fs, rev,
- pool));
+ SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev, pool));
+ SVN_ERR(get_root_changes_offset(&root_offset, NULL, revision_file,
+ fs, rev, pool));
- SVN_ERR(get_fs_id_at_offset(&root_id, revision_file, fs, rev,
- root_offset, pool));
+ SVN_ERR(get_fs_id_at_offset(&root_id, revision_file, fs, rev,
+ root_offset, pool));
- SVN_ERR(svn_io_file_close(revision_file, pool));
+ SVN_ERR(svn_io_file_close(revision_file, pool));
- SVN_ERR(svn_cache__set(ffd->rev_root_id_cache, &rev, root_id, pool));
+ SVN_ERR(svn_cache__set(ffd->rev_root_id_cache, &rev, root_id, pool));
- *root_id_p = root_id;
+ *root_id_p = root_id;
+ }
return SVN_NO_ERROR;
}
@@ -2054,8 +2056,12 @@ svn_fs_fs__get_changes(apr_array_header_
SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev, pool));
- SVN_ERR(get_root_changes_offset(NULL, &changes_offset, revision_file, fs,
- rev, pool));
+ if (svn_fs_fs__use_log_addressing(fs, rev))
+ SVN_ERR(svn_fs_fs__item_offset(&changes_offset, fs, rev, NULL,
+ SVN_FS_FS__ITEM_INDEX_CHANGES, pool));
+ else
+ SVN_ERR(get_root_changes_offset(NULL, &changes_offset, revision_file, fs,
+ rev, pool));
SVN_ERR(aligned_seek(fs, revision_file, changes_offset, pool));
SVN_ERR(svn_fs_fs__read_changes(changes,
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs.h
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs.h?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs.h (original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs.h Thu Aug 8
16:33:13 2013
@@ -86,6 +86,8 @@ extern "C" {
#define PATH_EXT_PROPS ".props" /* Extension for node props */
#define PATH_EXT_REV ".rev" /* Extension of protorev file */
#define PATH_EXT_REV_LOCK ".rev-lock" /* Extension of protorev lock file
*/
+#define PATH_TXN_ITEM_INDEX "itemidx" /* File containing the current item
+ index number */
#define PATH_INDEX "index" /* name of index files w/o ext */
/* Names of files in legacy FS formats */
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs_fs.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs_fs.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs_fs.c
(original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/fs_fs.c Thu Aug
8 16:33:13 2013
@@ -1071,17 +1071,43 @@ write_revision_zero(svn_fs_t *fs)
svn_string_t date;
/* Write out a rev file for revision 0. */
- SVN_ERR(svn_io_file_create(path_revision_zero,
- "PLAIN\nEND\nENDREP\n"
- "id: 0.0.r0/17\n"
- "type: dir\n"
- "count: 0\n"
- "text: 0 0 4 4 "
- "2d2977d1c96f487abe4a1e202dd03b4e\n"
- "cpath: /\n"
- "\n\n17 107\n", fs->pool));
+ if (svn_fs_fs__use_log_addressing(fs, 0))
+ SVN_ERR(svn_io_file_create(path_revision_zero,
+ "PLAIN\nEND\nENDREP\n"
+ "id: 0.0.r0/2\n"
+ "type: dir\n"
+ "count: 0\n"
+ "text: 0 3 4 4 "
+ "2d2977d1c96f487abe4a1e202dd03b4e\n"
+ "cpath: /\n"
+ "\n\n", fs->pool));
+ else
+ SVN_ERR(svn_io_file_create(path_revision_zero,
+ "PLAIN\nEND\nENDREP\n"
+ "id: 0.0.r0/17\n"
+ "type: dir\n"
+ "count: 0\n"
+ "text: 0 0 4 4 "
+ "2d2977d1c96f487abe4a1e202dd03b4e\n"
+ "cpath: /\n"
+ "\n\n17 107\n", fs->pool));
+
SVN_ERR(svn_io_set_file_read_only(path_revision_zero, FALSE, fs->pool));
+ if (svn_fs_fs__use_log_addressing(fs, 0))
+ {
+ const char *path = svn_fs_fs__path_l2p_index(fs, 0, fs->pool);
+ SVN_ERR(svn_io_file_create_binary
+ (path,
+ "\0\x80\x40" /* rev 0, 8k entries per page */
+ "\1\1\1" /* 1 rev, 1 page, 1 page in 1st rev */
+ "\6\4" /* page size: bytes, count */
+ "\0\xd6\1\xb1\1\x21", /* phys offsets + 1 */
+ 14,
+ fs->pool));
+ SVN_ERR(svn_io_set_file_read_only(path, FALSE, fs->pool));
+ }
+
/* Set a date on revision 0. */
date.data = svn_time_to_cstring(apr_time_now(), fs->pool);
date.len = strlen(date.data);
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.c (original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.c Thu Aug 8
16:33:13 2013
@@ -24,6 +24,7 @@
#include <stdlib.h>
#include "id.h"
+#include "index.h"
#include "../libsvn_fs/fs-loader.h"
#include "private/svn_temp_serializer.h"
@@ -363,6 +364,21 @@ svn_fs_fs__id_txn_create_root(const svn_
return (svn_fs_id_t *)id;
}
+svn_fs_id_t *svn_fs_fs__id_create_root(const svn_revnum_t revision,
+ apr_pool_t *pool)
+{
+ fs_fs__id_t *id = apr_pcalloc(pool, sizeof(*id));
+
+ id->private_id.txn_id.revision = SVN_INVALID_REVNUM;
+ id->private_id.rev_item.revision = revision;
+ id->private_id.rev_item.number = SVN_FS_FS__ITEM_INDEX_ROOT_NODE;
+
+ id->generic_id.vtable = &id_vtable;
+ id->generic_id.fsap_data = &id;
+
+ return (svn_fs_id_t *)id;
+}
+
svn_fs_id_t *
svn_fs_fs__id_txn_create(const svn_fs_fs__id_part_t *node_id,
const svn_fs_fs__id_part_t *copy_id,
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.h
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.h?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.h (original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/id.h Thu Aug 8
16:33:13 2013
@@ -126,6 +126,11 @@ int svn_fs_fs__id_part_compare(const svn
svn_fs_id_t *svn_fs_fs__id_txn_create_root(const svn_fs_fs__id_part_t *txn_id,
apr_pool_t *pool);
+/* Create the root ID for REVISION (logical addressing node only).
+ Allocate it in POOL. */
+svn_fs_id_t *svn_fs_fs__id_create_root(const svn_revnum_t revision,
+ apr_pool_t *pool);
+
/* Create an ID within a transaction based on NODE_ID, COPY_ID, and
TXN_ID, allocated in POOL. */
svn_fs_id_t *svn_fs_fs__id_txn_create(const svn_fs_fs__id_part_t *node_id,
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.c
(original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.c Thu Aug
8 16:33:13 2013
@@ -32,6 +32,7 @@
#include "private/svn_temp_serializer.h"
#include "index.h"
+#include "pack.h"
#include "temp_serializer.h"
#include "util.h"
@@ -1328,6 +1329,52 @@ l2p_proto_index_lookup(apr_off_t *offset
return SVN_NO_ERROR;
}
+svn_error_t *
+svn_fs_fs__item_offset(apr_off_t *absolute_position,
+ svn_fs_t *fs,
+ svn_revnum_t revision,
+ const svn_fs_fs__id_part_t *txn_id,
+ apr_uint64_t item_index,
+ apr_pool_t *pool)
+{
+ if (txn_id)
+ {
+ if (svn_fs_fs__use_log_addressing(fs, txn_id->revision + 1))
+ {
+ /* the txn is going to produce a rev with logical addressing.
+ So, we need to get our info from the (proto) index file. */
+ SVN_ERR(l2p_proto_index_lookup(absolute_position, fs, txn_id,
+ item_index, pool));
+ }
+ else
+ {
+ /* for data in txns, item_index *is* the offset */
+ *absolute_position = item_index;
+ }
+ }
+ else if (svn_fs_fs__use_log_addressing(fs, revision))
+ {
+ /* ordinary index lookup */
+ SVN_ERR(l2p_index_lookup(absolute_position, fs, revision,
+ item_index, pool));
+ }
+ else if (svn_fs_fs__is_packed_rev(fs, revision))
+ {
+ /* pack file with physical addressing */
+ apr_off_t rev_offset;
+ SVN_ERR(svn_fs_fs__get_packed_offset(&rev_offset, fs, revision, pool));
+ *absolute_position = rev_offset + item_index;
+ }
+ else
+ {
+ /* for non-packed revs with physical addressing,
+ item_index *is* the offset */
+ *absolute_position = item_index;
+ }
+
+ return SVN_NO_ERROR;
+}
+
/*
* Standard (de-)serialization functions
*/
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.h
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.h?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.h
(original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.h Thu Aug
8 16:33:13 2013
@@ -90,6 +90,18 @@ svn_fs_fs__l2p_index_create(svn_fs_t *fs
svn_revnum_t revision,
apr_pool_t *pool);
+/* For ITEM_INDEX within REV in FS, return the position in the respective
+ rev or pack file in *ABSOLUTE_POSITION. If TXN_ID is not NULL, return
+ the file offset within that transaction and REV should be given as
+ SVN_INVALID_REVNUM in that case. Use POOL for allocations.
+ */
+svn_error_t *
+svn_fs_fs__item_offset(apr_off_t *absolute_position,
+ svn_fs_t *fs,
+ svn_revnum_t revision,
+ const svn_fs_fs__id_part_t *txn_id,
+ apr_uint64_t item_index,
+ apr_pool_t *pool);
/* Serialization and caching interface
*/
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/recovery.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/recovery.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/recovery.c
(original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/recovery.c Thu
Aug 8 16:33:13 2013
@@ -26,6 +26,7 @@
#include "svn_pools.h"
#include "private/svn_string_private.h"
+#include "index.h"
#include "low_level.h"
#include "rep-cache.h"
#include "revprops.h"
Modified:
subversion/branches/log-addressing/subversion/libsvn_fs_fs/transaction.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/transaction.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/transaction.c
(original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/transaction.c
Thu Aug 8 16:33:13 2013
@@ -32,6 +32,7 @@
#include "svn_dirent_uri.h"
#include "fs_fs.h"
+#include "index.h"
#include "tree.h"
#include "util.h"
#include "id.h"
@@ -1536,6 +1537,82 @@ svn_fs_fs__add_change(svn_fs_t *fs,
return svn_io_file_close(file, pool);
}
+/* If the transaction TXN_ID in FS uses logical addressing, store the
+ * (ITEM_INDEX, OFFSET) pair in the txn's log-to-phys proto index file.
+ * Use POOL for allocations.
+ */
+static svn_error_t *
+store_l2p_index_entry(svn_fs_t *fs,
+ const svn_fs_fs__id_part_t *txn_id,
+ apr_off_t offset,
+ apr_uint64_t item_index,
+ apr_pool_t *pool)
+{
+ if (svn_fs_fs__use_log_addressing(fs, txn_id->revision + 1))
+ {
+ const char *path = svn_fs_fs__path_l2p_proto_index(fs, txn_id, pool);
+ apr_file_t *file;
+ SVN_ERR(svn_fs_fs__l2p_proto_index_open(&file, path, pool));
+ SVN_ERR(svn_fs_fs__l2p_proto_index_add_entry(file, offset,
+ item_index, pool));
+ SVN_ERR(svn_io_file_close(file, pool));
+ }
+
+ return SVN_NO_ERROR;
+}
+
+/* Allocate an item index for the given MY_OFFSET in the transaction TXN_ID
+ * of file system FS and return it in *ITEM_INDEX. For old formats, it
+ * will simply return the offset as item index; in new formats, it will
+ * increment the txn's item index counter file and store the mapping in
+ * the proto index file.
+ * Use POOL for allocations.
+ */
+static svn_error_t *
+allocate_item_index(apr_uint64_t *item_index,
+ svn_fs_t *fs,
+ const svn_fs_fs__id_part_t *txn_id,
+ apr_off_t my_offset,
+ apr_pool_t *pool)
+{
+ if (svn_fs_fs__use_log_addressing(fs, txn_id->revision + 1))
+ {
+ apr_file_t *file;
+ char buffer[SVN_INT64_BUFFER_SIZE] = { 0 };
+ svn_boolean_t eof = FALSE;
+ apr_size_t to_write;
+ apr_size_t read;
+ apr_off_t offset = 0;
+
+ /* read number, increment it and write it back to disk */
+ SVN_ERR(svn_io_file_open(&file,
+ svn_fs_fs__path_txn_item_index(fs, txn_id, pool),
+ APR_READ | APR_WRITE | APR_CREATE | APR_BUFFERED,
+ APR_OS_DEFAULT, pool));
+ SVN_ERR(svn_io_file_read_full2(file, buffer, sizeof(buffer)-1,
+ &read, &eof, pool));
+ if (read)
+ SVN_ERR(svn_cstring_atoui64(item_index, buffer));
+ else
+ *item_index = SVN_FS_FS__ITEM_INDEX_FIRST_USER;
+
+ to_write = svn__ui64toa(buffer, *item_index + 1);
+ SVN_ERR(svn_io_file_seek(file, SEEK_SET, &offset, pool));
+ SVN_ERR(svn_io_file_write_full(file, buffer, to_write, NULL, pool));
+ SVN_ERR(svn_io_file_close(file, pool));
+
+ /* write log-to-phys index */
+ SVN_ERR(store_l2p_index_entry(fs, txn_id, my_offset, *item_index,
+ pool));
+ }
+ else
+ {
+ *item_index = (apr_uint64_t)my_offset;
+ }
+
+ return SVN_NO_ERROR;
+}
+
/* This baton is used by the representation writing streams. It keeps
track of the checksum information as well as the total size of the
representation so far. */
@@ -1956,7 +2033,6 @@ rep_write_contents_close(void *baton)
apr_off_t offset;
rep = apr_pcalloc(b->parent_pool, sizeof(*rep));
- rep->item_index = b->rep_offset;
/* Close our delta stream so the last bits of svndiff are written
out. */
@@ -1993,6 +2069,8 @@ rep_write_contents_close(void *baton)
{
/* Write out our cosmetic end marker. */
SVN_ERR(svn_stream_puts(b->rep_stream, "ENDREP\n"));
+ SVN_ERR(allocate_item_index(&rep->item_index, b->fs, &rep->txn_id,
+ b->rep_offset, b->pool));
b->noderev->data_rep = rep;
}
@@ -2189,7 +2267,8 @@ write_hash_handler(void *baton,
/* Write out the hash HASH as a text representation to file FILE. In
the process, record position, the total size of the dump and MD5 as
- well as SHA1 in REP. If rep sharing has been enabled and REPS_HASH
+ well as SHA1 in REP. Add the representation of type ITEM_TYPE to
+ the indexes if necessary. If rep sharing has been enabled and REPS_HASH
is not NULL, it will be used in addition to the on-disk cache to find
earlier reps with the same content. When such existing reps can be
found, we will truncate the one just written from the file and return
@@ -2200,6 +2279,7 @@ write_hash_rep(representation_t *rep,
apr_hash_t *hash,
svn_fs_t *fs,
apr_hash_t *reps_hash,
+ int item_type,
apr_pool_t *pool)
{
svn_stream_t *stream;
@@ -2243,8 +2323,9 @@ write_hash_rep(representation_t *rep,
/* Write out our cosmetic end marker. */
SVN_ERR(svn_stream_puts(whb->stream, "ENDREP\n"));
- rep->item_index = offset;
-
+ SVN_ERR(allocate_item_index(&rep->item_index, fs, &rep->txn_id,
+ offset, pool));
+
/* update the representation */
rep->size = whb->size;
rep->expanded_size = 0;
@@ -2255,12 +2336,14 @@ write_hash_rep(representation_t *rep,
/* Write out the hash HASH pertaining to the NODEREV in FS as a deltified
text representation to file FILE. In the process, record the total size
- and the md5 digest in REP. If rep sharing has been enabled and REPS_HASH
+ and the md5 digest in REP and add the representation of type ITEM_TYPE
+ to the indexes if necessary. If rep sharing has been enabled and REPS_HASH
is not NULL, it will be used in addition to the on-disk cache to find
earlier reps with the same content. When such existing reps can be found,
we will truncate the one just written from the file and return the existing
- rep. If IS_PROPS is set, assume that we want to a props representation as
- the base for our delta. Perform temporary allocations in POOL. */
+ rep. If ITEM_TYPE is IS_PROPS equals SVN_FS_FS__ITEM_TYPE_*_PROPS, assume
+ that we want to a props representation as the base for our delta.
+ Perform temporary allocations in POOL. */
static svn_error_t *
write_hash_delta_rep(representation_t *rep,
apr_file_t *file,
@@ -2268,7 +2351,7 @@ write_hash_delta_rep(representation_t *r
svn_fs_t *fs,
node_revision_t *noderev,
apr_hash_t *reps_hash,
- svn_boolean_t is_props,
+ int item_type,
apr_pool_t *pool)
{
svn_txdelta_window_handler_t diff_wh;
@@ -2288,6 +2371,8 @@ write_hash_delta_rep(representation_t *r
struct write_hash_baton *whb;
fs_fs_data_t *ffd = fs->fsap_data;
int diff_version = ffd->format >= SVN_FS_FS__MIN_SVNDIFF1_FORMAT ? 1 : 0;
+ svn_boolean_t is_props = (item_type == SVN_FS_FS__ITEM_TYPE_FILE_PROPS)
+ || (item_type == SVN_FS_FS__ITEM_TYPE_DIR_PROPS);
/* Get the base for this delta. */
SVN_ERR(choose_delta_base(&base_rep, fs, noderev, is_props, pool));
@@ -2354,7 +2439,8 @@ write_hash_delta_rep(representation_t *r
SVN_ERR(svn_fs_fs__get_file_offset(&rep_end, file, pool));
SVN_ERR(svn_stream_puts(file_stream, "ENDREP\n"));
- rep->item_index = offset;
+ SVN_ERR(allocate_item_index(&rep->item_index, fs, &rep->txn_id,
+ offset, pool));
/* update the representation */
rep->expanded_size = whb->size;
@@ -2495,6 +2581,7 @@ write_final_rev(const svn_fs_id_t **new_
const svn_fs_id_t *new_id;
svn_fs_fs__id_part_t node_id, copy_id, rev_item;
fs_fs_data_t *ffd = fs->fsap_data;
+ const svn_fs_fs__id_part_t *txn_id = svn_fs_fs__id_txn_id(id);
*new_id_p = NULL;
/* Check to see if this is a transaction node. */
@@ -2540,16 +2627,19 @@ write_final_rev(const svn_fs_id_t **new_
/* Write out the contents of this directory as a text rep. */
SVN_ERR(unparse_dir_entries(&str_entries, entries, pool));
- reset_txn_in_rep(noderev->data_rep);
noderev->data_rep->revision = rev;
if (ffd->deltify_directories)
SVN_ERR(write_hash_delta_rep(noderev->data_rep, file,
str_entries, fs, noderev, NULL,
- FALSE, pool));
+ SVN_FS_FS__ITEM_TYPE_DIR_REP,
+ pool));
else
SVN_ERR(write_hash_rep(noderev->data_rep, file, str_entries,
- fs, NULL, pool));
+ fs, NULL, SVN_FS_FS__ITEM_TYPE_DIR_REP,
+ pool));
+
+ reset_txn_in_rep(noderev->data_rep);
}
}
else
@@ -2563,13 +2653,16 @@ write_final_rev(const svn_fs_id_t **new_
reset_txn_in_rep(noderev->data_rep);
noderev->data_rep->revision = rev;
- /* See issue 3845. Some unknown mechanism caused the
- protorev file to get truncated, so check for that
- here. */
- if (noderev->data_rep->item_index + noderev->data_rep->size
- > initial_offset)
- return svn_error_create(SVN_ERR_FS_CORRUPT, NULL,
- _("Truncated protorev file detected"));
+ if (!svn_fs_fs__use_log_addressing(fs, rev))
+ {
+ /* See issue 3845. Some unknown mechanism caused the
+ protorev file to get truncated, so check for that
+ here. */
+ if (noderev->data_rep->item_index + noderev->data_rep->size
+ > initial_offset)
+ return svn_error_create(SVN_ERR_FS_CORRUPT, NULL,
+ _("Truncated protorev file detected"));
+ }
}
}
@@ -2577,18 +2670,22 @@ write_final_rev(const svn_fs_id_t **new_
if (noderev->prop_rep && is_txn_rep(noderev->prop_rep))
{
apr_hash_t *proplist;
+ int item_type = noderev->kind == svn_node_dir
+ ? SVN_FS_FS__ITEM_TYPE_DIR_PROPS
+ : SVN_FS_FS__ITEM_TYPE_FILE_PROPS;
SVN_ERR(svn_fs_fs__get_proplist(&proplist, fs, noderev, pool));
- reset_txn_in_rep(noderev->prop_rep);
noderev->prop_rep->revision = rev;
if (ffd->deltify_properties)
SVN_ERR(write_hash_delta_rep(noderev->prop_rep, file,
proplist, fs, noderev, reps_hash,
- TRUE, pool));
+ item_type, pool));
else
SVN_ERR(write_hash_rep(noderev->prop_rep, file, proplist,
- fs, reps_hash, pool));
+ fs, reps_hash, item_type, pool));
+
+ reset_txn_in_rep(noderev->prop_rep);
}
/* Convert our temporary ID into a permanent revision one. */
@@ -2600,8 +2697,18 @@ write_final_rev(const svn_fs_id_t **new_
if (noderev->copyroot_rev == SVN_INVALID_REVNUM)
noderev->copyroot_rev = rev;
+ /* root nodes have a fixed ID in log addressing mode */
SVN_ERR(svn_fs_fs__get_file_offset(&my_offset, file, pool));
- rev_item.number = my_offset;
+ if (svn_fs_fs__use_log_addressing(fs, rev) && at_root)
+ {
+ /* reference the root noderev from the log-to-phys index */
+ rev_item.number = SVN_FS_FS__ITEM_INDEX_ROOT_NODE;
+ SVN_ERR(store_l2p_index_entry(fs, txn_id, my_offset,
+ rev_item.number, pool));
+ }
+ else
+ SVN_ERR(allocate_item_index(&rev_item.number, fs, txn_id, my_offset,
+ pool));
rev_item.revision = rev;
new_id = svn_fs_fs__id_rev_create(&node_id, ©_id, &rev_item, pool);
@@ -2661,15 +2768,16 @@ write_final_rev(const svn_fs_id_t **new_
return SVN_NO_ERROR;
}
-/* Write the changed path info from transaction TXN_ID in filesystem
- FS to the permanent rev-file FILE. *OFFSET_P is set the to offset
- in the file of the beginning of this information. Perform
- temporary allocations in POOL. */
+/* Write the changed path info from transaction TXN_ID in filesystem FS to
+ the permanent rev-file FILE. *OFFSET_P is set the to offset in the
+ file of the beginning of this information. NEW_REV is the revision
+ currently being committed. Perform temporary allocations in POOL. */
static svn_error_t *
write_final_changed_path_info(apr_off_t *offset_p,
apr_file_t *file,
svn_fs_t *fs,
const svn_fs_fs__id_part_t *txn_id,
+ svn_revnum_t new_rev,
apr_pool_t *pool)
{
apr_hash_t *changed_paths;
@@ -2684,6 +2792,13 @@ write_final_changed_path_info(apr_off_t
*offset_p = offset;
+ /* reference changes from the indexes */
+ if (svn_fs_fs__use_log_addressing(fs, new_rev))
+ {
+ SVN_ERR(store_l2p_index_entry(fs, txn_id, *offset_p,
+ SVN_FS_FS__ITEM_INDEX_CHANGES, pool));
+ }
+
return SVN_NO_ERROR;
}
@@ -2868,7 +2983,6 @@ commit_body(void *baton, apr_pool_t *poo
apr_array_header_t *txnprop_list;
svn_prop_t prop;
const svn_fs_fs__id_part_t *txn_id = svn_fs_fs__txn_get_id(cb->txn);
- svn_stringbuf_t *trailer;
/* Get the current youngest revision. */
SVN_ERR(svn_fs_fs__youngest_rev(&old_rev, cb->fs, pool));
@@ -2893,6 +3007,15 @@ commit_body(void *baton, apr_pool_t *poo
/* We are going to be one better than this puny old revision. */
new_rev = old_rev + 1;
+ /* Make sure that we don't try to commit an old txn that used physical
+ addressing but will be committed into the revision range that requires
+ logical addressing.
+ */
+ if (svn_fs_fs__use_log_addressing(cb->fs, new_rev)
+ && !svn_fs_fs__use_log_addressing(cb->fs, txn_id->revision + 1))
+ return svn_error_create(SVN_ERR_FS_TXN_ADDRESSING_MODE_OUT_OF_DATE, NULL,
+ _("Transaction addressing mode is out of date"));
+
/* Get a write handle on the proto revision file. */
SVN_ERR(get_writable_proto_rev(&proto_file, &proto_file_lockcookie,
cb->fs, txn_id, pool));
@@ -2907,15 +3030,27 @@ commit_body(void *baton, apr_pool_t *poo
/* Write the changed-path information. */
SVN_ERR(write_final_changed_path_info(&changed_path_offset, proto_file,
- cb->fs, txn_id, pool));
+ cb->fs, txn_id, new_rev, pool));
+
+ if (!svn_fs_fs__use_log_addressing(cb->fs, new_rev))
+ {
+ /* Write the final line. */
- /* Write the final line. */
- trailer = svn_fs_fs__unparse_revision_trailer
- (svn_fs_fs__id_item(new_root_id),
- changed_path_offset,
- pool);
- SVN_ERR(svn_io_file_write_full(proto_file, trailer->data, trailer->len,
- NULL, pool));
+ svn_stringbuf_t *trailer;
+ apr_off_t root_offset;
+ SVN_ERR(svn_fs_fs__item_offset(&root_offset,
+ cb->fs,
+ svn_fs_fs__id_rev(new_root_id),
+ NULL,
+ svn_fs_fs__id_item(new_root_id),
+ pool));
+ trailer = svn_fs_fs__unparse_revision_trailer
+ (root_offset,
+ changed_path_offset,
+ pool);
+ SVN_ERR(svn_io_file_write_full(proto_file, trailer->data, trailer->len,
+ NULL, pool));
+ }
SVN_ERR(svn_io_file_flush_to_disk(proto_file, pool));
SVN_ERR(svn_io_file_close(proto_file, pool));
@@ -2981,6 +3116,16 @@ commit_body(void *baton, apr_pool_t *poo
}
}
+ if (svn_fs_fs__use_log_addressing(cb->fs, new_rev))
+ {
+ /* Convert the index files from the proto format into their form
+ in their final location */
+ SVN_ERR(svn_fs_fs__l2p_index_create(cb->fs,
+ svn_fs_fs__path_l2p_index(cb->fs, new_rev, pool),
+ svn_fs_fs__path_l2p_proto_index(cb->fs, txn_id, pool),
+ new_rev, pool));
+ }
+
/* Move the finished rev file into place. */
old_rev_filename = svn_fs_fs__path_rev_absolute(cb->fs, old_rev, pool);
rev_filename = svn_fs_fs__path_rev(cb->fs, new_rev, pool);
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.c
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.c?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.c (original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.c Thu Aug
8 16:33:13 2013
@@ -233,6 +233,15 @@ svn_fs_fs__path_l2p_proto_index(svn_fs_t
}
const char *
+svn_fs_fs__path_txn_item_index(svn_fs_t *fs,
+ const svn_fs_fs__id_part_t *txn_id,
+ apr_pool_t *pool)
+{
+ return svn_dirent_join(svn_fs_fs__path_txn_dir(fs, txn_id, pool),
+ PATH_TXN_ITEM_INDEX, pool);
+}
+
+const char *
svn_fs_fs__path_txn_proto_rev(svn_fs_t *fs,
const svn_fs_fs__id_part_t *txn_id,
apr_pool_t *pool)
@@ -646,28 +655,3 @@ svn_fs_fs__use_log_addressing(svn_fs_t *
return ffd->min_log_addressing_rev != SVN_INVALID_REVNUM
&& ffd->min_log_addressing_rev <= rev;
}
-
-svn_error_t *
-svn_fs_fs__item_offset(apr_off_t *absolute_position,
- svn_fs_t *fs,
- svn_revnum_t rev,
- const svn_fs_fs__id_part_t *txn_id,
- apr_uint64_t item_index,
- apr_pool_t *pool)
-{
- /* for now, we only implement physical addressing */
-
- if (txn_id == NULL && svn_fs_fs__is_packed_rev(fs, rev))
- {
- apr_off_t rev_offset;
- SVN_ERR(svn_fs_fs__get_packed_offset(&rev_offset, fs, rev, pool));
- *absolute_position = rev_offset + item_index;
- }
- else
- {
- /* for TXNs and non-packed revs, item_index *is* the offset */
- *absolute_position = item_index;
- }
-
- return SVN_NO_ERROR;
-}
Modified: subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.h
URL:
http://svn.apache.org/viewvc/subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.h?rev=1511856&r1=1511855&r2=1511856&view=diff
==============================================================================
--- subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.h (original)
+++ subversion/branches/log-addressing/subversion/libsvn_fs_fs/util.h Thu Aug
8 16:33:13 2013
@@ -242,6 +242,15 @@ svn_fs_fs__path_l2p_proto_index(svn_fs_t
const svn_fs_fs__id_part_t *txn_id,
apr_pool_t *pool);
+/* Return the path of the file containing item_index counter for
+ * the transaction identified by TXN_ID in FS.
+ * The result will be allocated in POOL.
+ */
+const char *
+svn_fs_fs__path_txn_item_index(svn_fs_t *fs,
+ const svn_fs_fs__id_part_t *txn_id,
+ apr_pool_t *pool);
+
/* Return the path of the file containing the node origins cachs for
* the given NODE_ID in FS. The result will be allocated in POOL.
*/
@@ -374,16 +383,4 @@ svn_boolean_t
svn_fs_fs__use_log_addressing(svn_fs_t *fs,
svn_revnum_t rev);
-/* For ITEM_INDEX within REV in FS, return the position in the respective
- rev or pack file in *ABSOLUTE_POSITION. If TXN_ID is not NULL, return
- the file offset within that transaction and REV should be given as
- SVN_INVALID_REVNUM in that case. Use POOL for allocations. */
-svn_error_t *
-svn_fs_fs__item_offset(apr_off_t *absolute_position,
- svn_fs_t *fs,
- svn_revnum_t rev,
- const svn_fs_fs__id_part_t *txn_id,
- apr_uint64_t item_index,
- apr_pool_t *pool);
-
-#endif
\ No newline at end of file
+#endif
