Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
On Tuesday 24 July 2012 14:50:49 Jonathan Nieder wrote: It is unclear how this is different from giving the ceiling by specifying it as the END in -rSTART:END command line. Is this feature really needed? I think the idea is that you put this script (or a symlink to it) on your $PATH with higher precedence than svnrdump and run a command that expected to be able to use svnrdump. Then instead of going to the network, the command you run magically uses your test data instead. If the command you are testing wanted to run svnrdump without the upper endpoint set, we need to handle that request, either by emitting all the revs we have, or by stopping somewhere. The revlimit feature provides the stopping somewhere behavior which is not strictly needed but is presumably very useful when testing incremental fetch. Exactly, the purpose is to transparently replace svnrdump. Callers of svnrdump usually will specify -rSTART:HEAD, because they want to fetch everything they don't yet have. This feature allows to limit HEAD and to simulate incremental fetches using the same dump file. For me it proved very useful. Florian, do you mind if I make the revlimit feature a separate patch when applying this? No problem. Anyway, it looks good and reasonable to me, so will apply. Thanks. Jonathan -- Florian -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
On Mon, Jul 23, 2012 at 2:44 PM, Florian Achleitner florian.achleitner.2.6...@gmail.com wrote: + sys.exit(ret) \ No newline at end of file Nit: add a \n after sys.exit(ret), perhaps? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
Hi, Junio C Hamano wrote: Florian Achleitner florian.achleitner.2.6...@gmail.com writes: To ease testing without depending on a reachable svn server, this compact python script mimics parts of svnrdumps behaviour. It requires the remote url to start with sim://. [...] To allow using the same dump file for simulating multiple incremental imports the highest visible revision can be limited by setting the environment variable SVNRMAX to that value. This effectively limits HEAD to simulate the situation where higher revs don't exist yet. It is unclear how this is different from giving the ceiling by specifying it as the END in -rSTART:END command line. Is this feature really needed? I think the idea is that you put this script (or a symlink to it) on your $PATH with higher precedence than svnrdump and run a command that expected to be able to use svnrdump. Then instead of going to the network, the command you run magically uses your test data instead. If the command you are testing wanted to run svnrdump without the upper endpoint set, we need to handle that request, either by emitting all the revs we have, or by stopping somewhere. The revlimit feature provides the stopping somewhere behavior which is not strictly needed but is presumably very useful when testing incremental fetch. Florian, do you mind if I make the revlimit feature a separate patch when applying this? Anyway, it looks good and reasonable to me, so will apply. Thanks. Jonathan -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Add a svnrdump-simulator replaying a dump file for testing.
To ease testing without depending on a reachable svn server, this
compact python script mimics parts of svnrdumps behaviour.
It requires the remote url to start with sim://.
Start and end revisions are evaluated.
If the requested revision doesn't exist, as it is the case with
incremental imports, if no new commit was added, it returns 1
(like svnrdump).
To allow using the same dump file for simulating multiple
incremental imports the highest revision can be limited by setting
the environment variable SVNRMAX to that value. This simulates the
situation where higher revs don't exist yet.
---
contrib/svn-fe/svnrdump_sim.py | 53
1 file changed, 53 insertions(+)
create mode 100755 contrib/svn-fe/svnrdump_sim.py
diff --git a/contrib/svn-fe/svnrdump_sim.py b/contrib/svn-fe/svnrdump_sim.py
new file mode 100755
index 000..4701d76
--- /dev/null
+++ b/contrib/svn-fe/svnrdump_sim.py
@@ -0,0 +1,53 @@
+#!/usr/bin/python
+
+Simulates svnrdump by replaying an existing dump from a file, taking care
+of the specified revision range.
+To simulate incremental imports the environment variable SVNRMAX can be set
+to the highest revision that should be available.
+
+import sys, os
+
+
+def getrevlimit():
+ var = 'SVNRMAX'
+ if os.environ.has_key(var):
+ return os.environ[var]
+ return None
+
+def writedump(url, lower, upper):
+ if url.startswith('sim://'):
+ filename = url[6:]
+ if filename[-1] == '/': filename = filename[:-1] #remove
terminating slash
+ else:
+ raise ValueError('sim:// url required')
+ f = open(filename, 'r');
+ state = 'header'
+ wroterev = False
+ while(True):
+ l = f.readline()
+ if l == '': break
+ if state == 'header' and l.startswith('Revision-number: '):
+ state = 'prefix'
+ if state == 'prefix' and l == 'Revision-number: %s\n' % lower:
+ state = 'selection'
+ if not upper == 'HEAD' and state == 'selection' and l ==
'Revision-number: %s\n' % upper:
+ break;
+
+ if state == 'header' or state == 'selection':
+ if state == 'selection': wroterev = True
+ sys.stdout.write(l)
+ return wroterev
+
+if __name__ == __main__:
+ if not (len(sys.argv) in (3, 4, 5)):
+ print usage: %s dump URL -rLOWER:UPPER
+ sys.exit(1)
+ if not sys.argv[1] == 'dump': raise NotImplementedError('only dump is
suppported.')
+ url = sys.argv[2]
+ r = ('0', 'HEAD')
+ if len(sys.argv) == 4 and sys.argv[3][0:2] == '-r':
+ r = sys.argv[3][2:].lstrip().split(':')
+ if not getrevlimit() is None: r[1] = getrevlimit()
+ if writedump(url, r[0], r[1]): ret = 0
+ else: ret = 1
+ sys.exit(ret)
\ No newline at end of file
--
1.7.9.5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
Florian Achleitner wrote: To ease testing without depending on a reachable svn server, this compact python script mimics parts of svnrdumps behaviour. Thanks. Mind if I forge your sign-off? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
On Monday 23 July 2012 07:59:21 Jonathan Nieder wrote: Florian Achleitner wrote: To ease testing without depending on a reachable svn server, this compact python script mimics parts of svnrdumps behaviour. Thanks. Mind if I forge your sign-off? Ups. No problem, anyways I've added it locally, so here's the new version .. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
To ease testing without depending on a reachable svn server, this
compact python script mimics parts of svnrdumps behaviour.
It requires the remote url to start with sim://.
Start and end revisions are evaluated.
If the requested revision doesn't exist, as it is the case with
incremental imports, if no new commit was added, it returns 1
(like svnrdump).
To allow using the same dump file for simulating multiple
incremental imports the highest revision can be limited by setting
the environment variable SVNRMAX to that value. This simulates the
situation where higher revs don't exist yet.
Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
I had to fix the missing sign-off anyways..
contrib/svn-fe/svnrdump_sim.py | 53
1 file changed, 53 insertions(+)
create mode 100755 contrib/svn-fe/svnrdump_sim.py
diff --git a/contrib/svn-fe/svnrdump_sim.py b/contrib/svn-fe/svnrdump_sim.py
new file mode 100755
index 000..4701d76
--- /dev/null
+++ b/contrib/svn-fe/svnrdump_sim.py
@@ -0,0 +1,53 @@
+#!/usr/bin/python
+
+Simulates svnrdump by replaying an existing dump from a file, taking care
+of the specified revision range.
+To simulate incremental imports the environment variable SVNRMAX can be set
+to the highest revision that should be available.
+
+import sys, os
+
+
+def getrevlimit():
+ var = 'SVNRMAX'
+ if os.environ.has_key(var):
+ return os.environ[var]
+ return None
+
+def writedump(url, lower, upper):
+ if url.startswith('sim://'):
+ filename = url[6:]
+ if filename[-1] == '/': filename = filename[:-1] #remove
terminating slash
+ else:
+ raise ValueError('sim:// url required')
+ f = open(filename, 'r');
+ state = 'header'
+ wroterev = False
+ while(True):
+ l = f.readline()
+ if l == '': break
+ if state == 'header' and l.startswith('Revision-number: '):
+ state = 'prefix'
+ if state == 'prefix' and l == 'Revision-number: %s\n' % lower:
+ state = 'selection'
+ if not upper == 'HEAD' and state == 'selection' and l ==
'Revision-
number: %s\n' % upper:
+ break;
+
+ if state == 'header' or state == 'selection':
+ if state == 'selection': wroterev = True
+ sys.stdout.write(l)
+ return wroterev
+
+if __name__ == __main__:
+ if not (len(sys.argv) in (3, 4, 5)):
+ print usage: %s dump URL -rLOWER:UPPER
+ sys.exit(1)
+ if not sys.argv[1] == 'dump': raise NotImplementedError('only dump is
suppported.')
+ url = sys.argv[2]
+ r = ('0', 'HEAD')
+ if len(sys.argv) == 4 and sys.argv[3][0:2] == '-r':
+ r = sys.argv[3][2:].lstrip().split(':')
+ if not getrevlimit() is None: r[1] = getrevlimit()
+ if writedump(url, r[0], r[1]): ret = 0
+ else: ret = 1
+ sys.exit(ret)
\ No newline at end of file
--
1.7.9.5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
Florian Achleitner florian.achleitner.2.6...@gmail.com writes: It requires the remote url to start with sim://. Start and end revisions are evaluated. It is a bit unclear where start and end comes from, and if evaluated is the most important aspect of the handling of these two values. Do you mean the tool takes start and end revisions as arguments? If so, describe how. E.g. as two arguments (-rSTART -rEND)? As an argument that shows a range (-rSTART-END? -rSTART,END)? Do not answer with It is in the code (I cheated and peeked to find out it is -rSTART:END, but the reader should not have to peek). If the requested revision doesn't exist, as it is the case with incremental imports, if no new commit was added, it returns 1 (like svnrdump). This sentence does not parse for me. What is it trying to say? Requested revision does not exist _where_? It is unclear how incremental import and revision doesn't exist are related. no new commit was added to _what_ by _whom_? I presume that nobody is adding new commit _to_ an existing dump file, and the only thing this script does is to read and selectively write part of a dump file, so that would not add any new commit either. Puzzled. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
Florian Achleitner florian.achleitner.2.6...@gmail.com writes: I had to fix the missing sign-off anyways.. contrib/svn-fe/svnrdump_sim.py | 53 You also have whitespace damages (i.e. line wrapping introduced by your mailer). Using git-send-email avoids this kind of problem (there are also some advices for some mailers in Documentation/SubmittingPatches). -- Matthieu Moy http://www-verimag.imag.fr/~moy/ -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
On Monday 23 July 2012 18:24:40 Matthieu Moy wrote: You also have whitespace damages (i.e. line wrapping introduced by your mailer). Using git-send-email avoids this kind of problem (there are also some advices for some mailers in Documentation/SubmittingPatches). Damn. That's usually no problem with kmail either, if the config is right. I've already used git-send-email several times. But for replying to threads and adding several Cc: addresses it's a little cumbersome. How do you do that in a nice way? -- Florian -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
Florian Achleitner florian.achleitner.2.6...@gmail.com writes: On Monday 23 July 2012 18:24:40 Matthieu Moy wrote: You also have whitespace damages (i.e. line wrapping introduced by your mailer). Using git-send-email avoids this kind of problem (there are also some advices for some mailers in Documentation/SubmittingPatches). Damn. That's usually no problem with kmail either, if the config is right. I've already used git-send-email several times. But for replying to threads and adding several Cc: addresses it's a little cumbersome. How do you do that in a nice way? For the threading itself, I usually find the message-id, and use git send-email --in-reply-to='cut-and-pasted-id'. The painful part is when you want to reproduce a Cc: list, but I have no magic trick for that ;-). -- Matthieu Moy http://www-verimag.imag.fr/~moy/ -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
On Mon, Jul 23, 2012 at 09:46:49PM +0200, Matthieu Moy wrote:
Damn. That's usually no problem with kmail either, if the config is right.
I've already used git-send-email several times.
But for replying to threads and adding several Cc: addresses it's a little
cumbersome.
How do you do that in a nice way?
For the threading itself, I usually find the message-id, and use
git send-email --in-reply-to='cut-and-pasted-id'. The painful part
is when you want to reproduce a Cc: list, but I have no magic trick for
that ;-).
I save a copy of the message I am replying to (usually my cover letter,
which I generated by just replying in my MUA) into a well-known location
(I use a mutt hot-key to do this), and then run it through this
monstrosity:
get_reply_headers() {
perl -ne '
if (defined $opt /^\s+(.*)/) {
$val .= $1;
next;
}
if (defined $opt) {
print --$opt=, quotemeta($val), ;
$opt = $val = undef;
}
if (/^(cc|to):\s*(.*)/i) {
$opt = lc($1);
$val = $2;
}
elsif (/^message-id:\s*(.*)/i) {
$opt = in-reply-to;
$val = $1;
}
'
}
which can be used on the command line of format-patch or send-email, like:
eval git format-patch $(get_reply_headers your-patch-file)
I put the result in an mbox, then review and send it out in mutt (using
the resend-message command), but you could invoke send-email directly,
or format-patch into a file for review and send it with send-email.
-Peff
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Add a svnrdump-simulator replaying a dump file for testing.
To ease testing without depending on a reachable svn server, this
compact python script mimics parts of svnrdumps behaviour.
It requires the remote url to start with sim://.
Eventual slashes at the end of the url are stripped.
The url specifies the path of the svn dump file (as created by
svnrdump). Selectable parts of it, or the whole file, are written
to stdout. The part is selectable by giving start and end revision
on the command line.
Start and end revisions can be specified on the command line
(-rSTART:END, like for svnrdump).
Only revisions between START and excluding END are replayed from
the dumpfile specified by the url. END can also be HEAD.
If the start revision specified on the command line doesn't exist
in the dump file, it returns 1.
This emulates the behaviour of svnrdump when STARTHEAD, i.e. the
requested start revision doesn't exist on the server.
To allow using the same dump file for simulating multiple
incremental imports the highest visible revision can be limited by
setting the environment variable SVNRMAX to that value. This
effectively limits HEAD to simulate the situation where higher
revs don't exist yet.
Signed-off-by: Florian Achleitner florian.achleitner.2.6...@gmail.com
---
contrib/svn-fe/svnrdump_sim.py | 53
1 file changed, 53 insertions(+)
create mode 100755 contrib/svn-fe/svnrdump_sim.py
diff --git a/contrib/svn-fe/svnrdump_sim.py b/contrib/svn-fe/svnrdump_sim.py
new file mode 100755
index 000..4701d76
--- /dev/null
+++ b/contrib/svn-fe/svnrdump_sim.py
@@ -0,0 +1,53 @@
+#!/usr/bin/python
+
+Simulates svnrdump by replaying an existing dump from a file, taking care
+of the specified revision range.
+To simulate incremental imports the environment variable SVNRMAX can be set
+to the highest revision that should be available.
+
+import sys, os
+
+
+def getrevlimit():
+ var = 'SVNRMAX'
+ if os.environ.has_key(var):
+ return os.environ[var]
+ return None
+
+def writedump(url, lower, upper):
+ if url.startswith('sim://'):
+ filename = url[6:]
+ if filename[-1] == '/': filename = filename[:-1] #remove
terminating slash
+ else:
+ raise ValueError('sim:// url required')
+ f = open(filename, 'r');
+ state = 'header'
+ wroterev = False
+ while(True):
+ l = f.readline()
+ if l == '': break
+ if state == 'header' and l.startswith('Revision-number: '):
+ state = 'prefix'
+ if state == 'prefix' and l == 'Revision-number: %s\n' % lower:
+ state = 'selection'
+ if not upper == 'HEAD' and state == 'selection' and l ==
'Revision-number: %s\n' % upper:
+ break;
+
+ if state == 'header' or state == 'selection':
+ if state == 'selection': wroterev = True
+ sys.stdout.write(l)
+ return wroterev
+
+if __name__ == __main__:
+ if not (len(sys.argv) in (3, 4, 5)):
+ print usage: %s dump URL -rLOWER:UPPER
+ sys.exit(1)
+ if not sys.argv[1] == 'dump': raise NotImplementedError('only dump is
suppported.')
+ url = sys.argv[2]
+ r = ('0', 'HEAD')
+ if len(sys.argv) == 4 and sys.argv[3][0:2] == '-r':
+ r = sys.argv[3][2:].lstrip().split(':')
+ if not getrevlimit() is None: r[1] = getrevlimit()
+ if writedump(url, r[0], r[1]): ret = 0
+ else: ret = 1
+ sys.exit(ret)
\ No newline at end of file
--
1.7.9.5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] Add a svnrdump-simulator replaying a dump file for testing.
Florian Achleitner florian.achleitner.2.6...@gmail.com writes: To ease testing without depending on a reachable svn server, this compact python script mimics parts of svnrdumps behaviour. It requires the remote url to start with sim://. Eventual slashes at the end of the url are stripped. s/ventual/xcess/ perhaps? The url specifies the path of the svn dump file (as created by svnrdump). Selectable parts of it, or the whole file, are written to stdout. The part is selectable by giving start and end revision on the command line. Start and end revisions can be specified on the command line (-rSTART:END, like for svnrdump). Only revisions between START and excluding END are replayed from the dumpfile specified by the url. END can also be HEAD. If the start revision specified on the command line doesn't exist in the dump file, it returns 1. This emulates the behaviour of svnrdump when STARTHEAD, i.e. the requested start revision doesn't exist on the server. Much more understandable than before. To allow using the same dump file for simulating multiple incremental imports the highest visible revision can be limited by setting the environment variable SVNRMAX to that value. This effectively limits HEAD to simulate the situation where higher revs don't exist yet. It is unclear how this is different from giving the ceiling by specifying it as the END in -rSTART:END command line. Is this feature really needed? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
