D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
pulkit added a comment. In https://phab.mercurial-scm.org/D4099#64270, @durin42 wrote: > In https://phab.mercurial-scm.org/D4099#64268, @pulkit wrote: > > > In https://phab.mercurial-scm.org/D4099#64266, @durin42 wrote: > > > > > There are non-Google users of narrow at this point (recall we've gotten non-Googler patches), so please publish the temporary migration hack someplace for the benefit of the other users. > > > > > > (A phabricator review that we immediately abandon would probably be fine, but we should definitely make a note in the relnotes for 4.8) > > > > > > Yes, we might roll-out narrow with mercurial 4.7 first, so this script will be of great help. (BTW, isn't it just a move of file?) > > > It's "just" a move, but typically the way we handle these migrations at Google is we write an extension that does one of the following: > > 1. Hijacks reads for the new file and reads the old file > 2. Hijacks reads for the new file if it doesn't exist and moves the new file > > we run in mode 1 when we're doing the rollout and might roll back, then switch to mode 2 when we're past the rubicon on the rollout and can finalize the data migration. Once everyone has run the extension in mode 2 for a while, we reap the extension. Oh, I missed that clients need to be updated in real time. It will really be great if this extension is shared. Thanks @durin42 for the explanation. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4099 To: martinvonz, durin42, #hg-reviewers Cc: pulkit, yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
durin42 added a comment. In https://phab.mercurial-scm.org/D4099#64268, @pulkit wrote: > In https://phab.mercurial-scm.org/D4099#64266, @durin42 wrote: > > > There are non-Google users of narrow at this point (recall we've gotten non-Googler patches), so please publish the temporary migration hack someplace for the benefit of the other users. > > > > (A phabricator review that we immediately abandon would probably be fine, but we should definitely make a note in the relnotes for 4.8) > > > Yes, we might roll-out narrow with mercurial 4.7 first, so this script will be of great help. (BTW, isn't it just a move of file?) It's "just" a move, but typically the way we handle these migrations at Google is we write an extension that does one of the following: 1. Hijacks reads for the new file and reads the old file 2. Hijacks reads for the new file if it doesn't exist and moves the new file we run in mode 1 when we're doing the rollout and might roll back, then switch to mode 2 when we're past the rubicon on the rollout and can finalize the data migration. Once everyone has run the extension in mode 2 for a while, we reap the extension. REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4099 To: martinvonz, durin42, #hg-reviewers Cc: pulkit, yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
pulkit added a comment. In https://phab.mercurial-scm.org/D4099#64266, @durin42 wrote: > There are non-Google users of narrow at this point (recall we've gotten non-Googler patches), so please publish the temporary migration hack someplace for the benefit of the other users. > > (A phabricator review that we immediately abandon would probably be fine, but we should definitely make a note in the relnotes for 4.8) Yes, we might roll-out narrow with mercurial 4.7 first, so this script will be of great help. (BTW, isn't it just a move of file?) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4099 To: martinvonz, durin42, #hg-reviewers Cc: pulkit, yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
durin42 added a comment. There are non-Google users of narrow at this point (recall we've gotten non-Googler patches), so please publish the temporary migration hack someplace for the benefit of the other users. (A phabricator review that we immediately abandon would probably be fine, but we should definitely make a note in the relnotes for 4.8) REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4099 To: martinvonz, durin42, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
This revision was automatically updated to reflect the committed changes.
Closed by commit rHG576eef1ab43d: narrow: move .hg/narrowspec to
.hg/store/narrowspec (BC) (authored by martinvonz, committed by ).
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4099?vs=9874=9934
REVISION DETAIL
https://phab.mercurial-scm.org/D4099
AFFECTED FILES
hgext/narrow/__init__.py
hgext/narrow/narrowrepo.py
mercurial/localrepo.py
mercurial/narrowspec.py
mercurial/store.py
tests/test-narrow-debugcommands.t
tests/test-narrow-pull.t
CHANGE DETAILS
diff --git a/tests/test-narrow-pull.t b/tests/test-narrow-pull.t
--- a/tests/test-narrow-pull.t
+++ b/tests/test-narrow-pull.t
@@ -166,7 +166,6 @@
We should also be able to unshare without breaking everything:
$ hg unshare
- devel-warn: write with no wlock: "narrowspec" at:
*/hgext/narrow/narrowrepo.py:* (unsharenarrowspec) (glob)
$ hg verify
checking changesets
checking manifests
diff --git a/tests/test-narrow-debugcommands.t
b/tests/test-narrow-debugcommands.t
--- a/tests/test-narrow-debugcommands.t
+++ b/tests/test-narrow-debugcommands.t
@@ -1,7 +1,7 @@
$ . "$TESTDIR/narrow-library.sh"
$ hg init repo
$ cd repo
- $ cat << EOF > .hg/narrowspec
+ $ cat << EOF > .hg/store/narrowspec
> [include]
> path:foo
> [exclude]
diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -317,8 +317,8 @@
mode = None
return mode
-_data = ('data meta 00manifest.d 00manifest.i 00changelog.d 00changelog.i'
- ' phaseroots obsstore')
+_data = ('narrowspec data meta 00manifest.d 00manifest.i'
+ ' 00changelog.d 00changelog.i phaseroots obsstore')
def isrevlog(f, kind, st):
return kind == stat.S_IFREG and f[-2:] in ('.i', '.d')
@@ -546,7 +546,7 @@
raise
def copylist(self):
-d = ('data meta dh fncache phaseroots obsstore'
+d = ('narrowspec data meta dh fncache phaseroots obsstore'
' 00manifest.d 00manifest.i 00changelog.d 00changelog.i')
return (['requires', '00changelog.i'] +
['store/' + f for f in d.split()])
diff --git a/mercurial/narrowspec.py b/mercurial/narrowspec.py
--- a/mercurial/narrowspec.py
+++ b/mercurial/narrowspec.py
@@ -108,7 +108,7 @@
def load(repo):
try:
-spec = repo.vfs.read(FILENAME)
+spec = repo.svfs.read(FILENAME)
except IOError as e:
# Treat "narrowspec does not exist" the same as "narrowspec file exists
# and is empty".
@@ -125,19 +125,19 @@
def save(repo, includepats, excludepats):
spec = format(includepats, excludepats)
-repo.vfs.write(FILENAME, spec)
+repo.svfs.write(FILENAME, spec)
def savebackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
return
vfs = repo.vfs
vfs.tryunlink(backupname)
-util.copyfile(vfs.join(FILENAME), vfs.join(backupname), hardlink=True)
+util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname),
hardlink=True)
def restorebackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
return
-repo.vfs.rename(backupname, FILENAME)
+util.rename(repo.vfs.join(backupname), repo.svfs.join(FILENAME))
def clearbackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -811,7 +811,7 @@
" working parent %s!\n") % short(node))
return nullid
-@repofilecache(narrowspec.FILENAME)
+@storecache(narrowspec.FILENAME)
def narrowpats(self):
"""matcher patterns for this repository's narrowspec
@@ -823,7 +823,7 @@
source = hg.sharedreposource(self)
return narrowspec.load(source)
-@repofilecache(narrowspec.FILENAME)
+@storecache(narrowspec.FILENAME)
def _narrowmatch(self):
if repository.NARROW_REQUIREMENT not in self.requirements:
return matchmod.always(self.root, '')
diff --git a/hgext/narrow/narrowrepo.py b/hgext/narrow/narrowrepo.py
--- a/hgext/narrow/narrowrepo.py
+++ b/hgext/narrow/narrowrepo.py
@@ -7,34 +7,11 @@
from __future__ import absolute_import
-from mercurial import (
-hg,
-narrowspec,
-repository,
-)
-
from . import (
narrowdirstate,
narrowrevlog,
)
-def wrappostshare(orig, sourcerepo, destrepo, **kwargs):
-orig(sourcerepo, destrepo, **kwargs)
-if repository.NARROW_REQUIREMENT in sourcerepo.requirements:
-with destrepo.wlock():
-with destrepo.vfs('shared', 'a') as fp:
-fp.write(narrowspec.FILENAME + '\n')
-
-def unsharenarrowspec(orig, ui, repo, repopath):
-if (repository.NARROW_REQUIREMENT in repo.requirements
-and repo.path == repopath and repo.shared()):
Re: D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
Queued, thanks. > def savebackup(repo, backupname): > if repository.NARROW_REQUIREMENT not in repo.requirements: > return > vfs = repo.vfs > vfs.tryunlink(backupname) > -util.copyfile(vfs.join(FILENAME), vfs.join(backupname), hardlink=True) > +util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname), > hardlink=True) Maybe move backup file to svfs as well? ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
yuja added a comment. Queued, thanks. > def savebackup(repo, backupname): > if repository.NARROW_REQUIREMENT not in repo.requirements: > return > vfs = repo.vfs > vfs.tryunlink(backupname) > > - util.copyfile(vfs.join(FILENAME), vfs.join(backupname), hardlink=True) + util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname), hardlink=True) Maybe move backup file to svfs as well? REPOSITORY rHG Mercurial REVISION DETAIL https://phab.mercurial-scm.org/D4099 To: martinvonz, durin42, #hg-reviewers Cc: yuja, mercurial-devel ___ Mercurial-devel mailing list Mercurial-devel@mercurial-scm.org https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
martinvonz updated this revision to Diff 9874.
REPOSITORY
rHG Mercurial
CHANGES SINCE LAST UPDATE
https://phab.mercurial-scm.org/D4099?vs=9861=9874
REVISION DETAIL
https://phab.mercurial-scm.org/D4099
AFFECTED FILES
hgext/narrow/__init__.py
hgext/narrow/narrowrepo.py
mercurial/localrepo.py
mercurial/narrowspec.py
mercurial/store.py
tests/test-narrow-debugcommands.t
tests/test-narrow-pull.t
CHANGE DETAILS
diff --git a/tests/test-narrow-pull.t b/tests/test-narrow-pull.t
--- a/tests/test-narrow-pull.t
+++ b/tests/test-narrow-pull.t
@@ -166,7 +166,6 @@
We should also be able to unshare without breaking everything:
$ hg unshare
- devel-warn: write with no wlock: "narrowspec" at:
*/hgext/narrow/narrowrepo.py:* (unsharenarrowspec) (glob)
$ hg verify
checking changesets
checking manifests
diff --git a/tests/test-narrow-debugcommands.t
b/tests/test-narrow-debugcommands.t
--- a/tests/test-narrow-debugcommands.t
+++ b/tests/test-narrow-debugcommands.t
@@ -1,7 +1,7 @@
$ . "$TESTDIR/narrow-library.sh"
$ hg init repo
$ cd repo
- $ cat << EOF > .hg/narrowspec
+ $ cat << EOF > .hg/store/narrowspec
> [include]
> path:foo
> [exclude]
diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -317,8 +317,8 @@
mode = None
return mode
-_data = ('data meta 00manifest.d 00manifest.i 00changelog.d 00changelog.i'
- ' phaseroots obsstore')
+_data = ('narrowspec data meta 00manifest.d 00manifest.i'
+ ' 00changelog.d 00changelog.i phaseroots obsstore')
def isrevlog(f, kind, st):
return kind == stat.S_IFREG and f[-2:] in ('.i', '.d')
@@ -546,7 +546,7 @@
raise
def copylist(self):
-d = ('data meta dh fncache phaseroots obsstore'
+d = ('narrowspec data meta dh fncache phaseroots obsstore'
' 00manifest.d 00manifest.i 00changelog.d 00changelog.i')
return (['requires', '00changelog.i'] +
['store/' + f for f in d.split()])
diff --git a/mercurial/narrowspec.py b/mercurial/narrowspec.py
--- a/mercurial/narrowspec.py
+++ b/mercurial/narrowspec.py
@@ -108,7 +108,7 @@
def load(repo):
try:
-spec = repo.vfs.read(FILENAME)
+spec = repo.svfs.read(FILENAME)
except IOError as e:
# Treat "narrowspec does not exist" the same as "narrowspec file exists
# and is empty".
@@ -125,19 +125,19 @@
def save(repo, includepats, excludepats):
spec = format(includepats, excludepats)
-repo.vfs.write(FILENAME, spec)
+repo.svfs.write(FILENAME, spec)
def savebackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
return
vfs = repo.vfs
vfs.tryunlink(backupname)
-util.copyfile(vfs.join(FILENAME), vfs.join(backupname), hardlink=True)
+util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname),
hardlink=True)
def restorebackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
return
-repo.vfs.rename(backupname, FILENAME)
+util.rename(repo.vfs.join(backupname), repo.svfs.join(FILENAME))
def clearbackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -811,7 +811,7 @@
" working parent %s!\n") % short(node))
return nullid
-@repofilecache(narrowspec.FILENAME)
+@storecache(narrowspec.FILENAME)
def narrowpats(self):
"""matcher patterns for this repository's narrowspec
@@ -823,7 +823,7 @@
source = hg.sharedreposource(self)
return narrowspec.load(source)
-@repofilecache(narrowspec.FILENAME)
+@storecache(narrowspec.FILENAME)
def _narrowmatch(self):
if repository.NARROW_REQUIREMENT not in self.requirements:
return matchmod.always(self.root, '')
diff --git a/hgext/narrow/narrowrepo.py b/hgext/narrow/narrowrepo.py
--- a/hgext/narrow/narrowrepo.py
+++ b/hgext/narrow/narrowrepo.py
@@ -7,34 +7,11 @@
from __future__ import absolute_import
-from mercurial import (
-hg,
-narrowspec,
-repository,
-)
-
from . import (
narrowdirstate,
narrowrevlog,
)
-def wrappostshare(orig, sourcerepo, destrepo, **kwargs):
-orig(sourcerepo, destrepo, **kwargs)
-if repository.NARROW_REQUIREMENT in sourcerepo.requirements:
-with destrepo.wlock():
-with destrepo.vfs('shared', 'a') as fp:
-fp.write(narrowspec.FILENAME + '\n')
-
-def unsharenarrowspec(orig, ui, repo, repopath):
-if (repository.NARROW_REQUIREMENT in repo.requirements
-and repo.path == repopath and repo.shared()):
-srcrepo = hg.sharedreposource(repo)
-with srcrepo.vfs(narrowspec.FILENAME) as f:
-spec = f.read()
-with
D4099: narrow: move .hg/narrowspec to .hg/store/narrowspec (BC)
martinvonz created this revision.
Herald added a reviewer: durin42.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
The narrowspec is more closely related to the store than to the
working copy. For example, if the narrowspec changes, the set of
revlogs also needs to change (the working copy may change, but that
depends on which commit is checked out). Also, when using the share
extension, the narrowspec needs to be shared along with the
store. This patch therefore moves the narrowspec into the store/
directory.
This is clearly a breaking change, but I haven't bothered trying to
fall back to reading the narrowspec from the old location (.hg/),
because there are very few users of narrow out there. (We'll add a
temporary hack to our Google-internal extension to handle the
migration.)
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D4099
AFFECTED FILES
hgext/narrow/__init__.py
hgext/narrow/narrowrepo.py
mercurial/localrepo.py
mercurial/narrowspec.py
mercurial/store.py
tests/test-narrow-debugcommands.t
CHANGE DETAILS
diff --git a/tests/test-narrow-debugcommands.t
b/tests/test-narrow-debugcommands.t
--- a/tests/test-narrow-debugcommands.t
+++ b/tests/test-narrow-debugcommands.t
@@ -1,7 +1,7 @@
$ . "$TESTDIR/narrow-library.sh"
$ hg init repo
$ cd repo
- $ cat << EOF > .hg/narrowspec
+ $ cat << EOF > .hg/store/narrowspec
> [includes]
> path:foo
> [excludes]
diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -317,8 +317,8 @@
mode = None
return mode
-_data = ('data meta 00manifest.d 00manifest.i 00changelog.d 00changelog.i'
- ' phaseroots obsstore')
+_data = ('narrowspec data meta 00manifest.d 00manifest.i'
+ ' 00changelog.d 00changelog.i phaseroots obsstore')
def isrevlog(f, kind, st):
return kind == stat.S_IFREG and f[-2:] in ('.i', '.d')
@@ -546,7 +546,7 @@
raise
def copylist(self):
-d = ('data meta dh fncache phaseroots obsstore'
+d = ('narrowspec data meta dh fncache phaseroots obsstore'
' 00manifest.d 00manifest.i 00changelog.d 00changelog.i')
return (['requires', '00changelog.i'] +
['store/' + f for f in d.split()])
diff --git a/mercurial/narrowspec.py b/mercurial/narrowspec.py
--- a/mercurial/narrowspec.py
+++ b/mercurial/narrowspec.py
@@ -130,7 +130,7 @@
def load(repo):
try:
-spec = repo.vfs.read(FILENAME)
+spec = repo.svfs.read(FILENAME)
except IOError as e:
# Treat "narrowspec does not exist" the same as "narrowspec file exists
# and is empty".
@@ -141,19 +141,19 @@
def save(repo, includepats, excludepats):
spec = format(includepats, excludepats)
-repo.vfs.write(FILENAME, spec)
+repo.svfs.write(FILENAME, spec)
def savebackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
return
vfs = repo.vfs
vfs.tryunlink(backupname)
-util.copyfile(vfs.join(FILENAME), vfs.join(backupname), hardlink=True)
+util.copyfile(repo.svfs.join(FILENAME), vfs.join(backupname),
hardlink=True)
def restorebackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
return
-repo.vfs.rename(backupname, FILENAME)
+util.rename(repo.vfs.join(backupname), repo.svfs.join(FILENAME))
def clearbackup(repo, backupname):
if repository.NARROW_REQUIREMENT not in repo.requirements:
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -811,7 +811,7 @@
" working parent %s!\n") % short(node))
return nullid
-@repofilecache(narrowspec.FILENAME)
+@storecache(narrowspec.FILENAME)
def narrowpats(self):
"""matcher patterns for this repository's narrowspec
@@ -823,7 +823,7 @@
source = hg.sharedreposource(self)
return narrowspec.load(source)
-@repofilecache(narrowspec.FILENAME)
+@storecache(narrowspec.FILENAME)
def _narrowmatch(self):
if repository.NARROW_REQUIREMENT not in self.requirements:
return matchmod.always(self.root, '')
diff --git a/hgext/narrow/narrowrepo.py b/hgext/narrow/narrowrepo.py
--- a/hgext/narrow/narrowrepo.py
+++ b/hgext/narrow/narrowrepo.py
@@ -7,34 +7,11 @@
from __future__ import absolute_import
-from mercurial import (
-hg,
-narrowspec,
-repository,
-)
-
from . import (
narrowdirstate,
narrowrevlog,
)
-def wrappostshare(orig, sourcerepo, destrepo, **kwargs):
-orig(sourcerepo, destrepo, **kwargs)
-if repository.NARROW_REQUIREMENT in sourcerepo.requirements:
-with destrepo.wlock():
-with destrepo.vfs('shared', 'a') as fp:
-fp.write(narrowspec.FILENAME + '\n')
-
-def
