Re: [Monotone-devel] update multiple projects

2010-04-06 Thread Stephen Leake
Thomas Keller m...@thomaskeller.biz writes:

 Am 02.04.10 10:32, schrieb Stephen Leake:
 Here's a plan for moving forward:
 
 1) Modify update so it doesn't call the external merger if
--non-interactive is set (either by the user or by au stdio).
 
 2) Create a new branch nvm.workspace_conflicts to work on better
methods for resolving workspace conflicts.
 
 I believe 1) will allow the current work_stephe branch to be
 propagated to main, and will allow implementing gds_update_all, which
 is my primary immediate goal.
 
 1) will prevent using 'au update' in my Emacs front end, but that's
 ok; it's motivation for 2.

 That sounds like a good plan!

Ok. work_stephe merged to main.

I also figured out why I'm getting random test failures; it's my backup
utility. Apparently it gets notifications of disk changes, and then
locks the involved files while it figures out that it's not supposed to
back them up. Sigh. Fortunately, it has a stop doing that for 1 hour
feature.

So all tests are back to passing on my system (except Empty_Environment).

-- 
-- Stephe


___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-04-05 Thread Thomas Keller
Am 02.04.10 10:32, schrieb Stephen Leake:
 Here's a plan for moving forward:
 
 1) Modify update so it doesn't call the external merger if
--non-interactive is set (either by the user or by au stdio).
 
 2) Create a new branch nvm.workspace_conflicts to work on better
methods for resolving workspace conflicts.
 
 I believe 1) will allow the current work_stephe branch to be
 propagated to main, and will allow implementing gds_update_all, which
 is my primary immediate goal.
 
 1) will prevent using 'au update' in my Emacs front end, but that's
 ok; it's motivation for 2.

That sounds like a good plan!

Thanks for your work,
Thomas.

-- 
GPG-Key 0x160D1092 | tommyd3...@jabber.ccc.de | http://thomaskeller.biz
Please note that according to the EU law on data retention, information
on every electronic information exchange might be retained for a period
of six months or longer: http://www.vorratsdatenspeicherung.de/?lang=en



signature.asc
Description: OpenPGP digital signature
___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-04-02 Thread Stephen Leake
Thomas Keller m...@thomaskeller.biz writes:

 Am 01.04.10 04:18, schrieb Stephen Leake:
 
 So 'mtn au stdio' 'l6updatee' with the standard hooks does require
 prompting via stdio, if the best available merger is opendiff. So we
 (I) have to do something. Sigh.

 Really, please don't let it call the external merger - this really gets
 messy otherwise. I won't support the inclusion of this code otherwise in
 mainline, but would propose you keep that in your own private branch.

Here's a plan for moving forward:

1) Modify update so it doesn't call the external merger if
   --non-interactive is set (either by the user or by au stdio).

2) Create a new branch nvm.workspace_conflicts to work on better
   methods for resolving workspace conflicts.

I believe 1) will allow the current work_stephe branch to be
propagated to main, and will allow implementing gds_update_all, which
is my primary immediate goal.

1) will prevent using 'au update' in my Emacs front end, but that's
ok; it's motivation for 2.

-- 
-- Stephe


___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-04-01 Thread Thomas Keller
Am 01.04.10 04:18, schrieb Stephen Leake:
 Thomas Keller m...@thomaskeller.biz writes:
 
 
 Currently, whether you do a normal or a
 workspace merge, as soon as a content conflict arises, mtn opens a
 merger process (might it be vim or kdiff3 or emacs or opendiff) and
 waits until this process ends, i.e. returns, to pick up the saved
 temporary file afterwards.
 The problem is now that the shell call which opens the merger
 process has to _block_ until the process actually ends, 
 if it spawns in the background and returns immediately, 
 
 Why would it do this? Sounds like a bug in the merger hook, or the
 merge tool, to me.

Its not a bug, its an OSX feature - simply put it doesn't matter if you
open a file or an application on Mac OS X, for both cases an
asynchronous event is fired which is handled by the system. If the app
which should be started is already running, the app gets a file open
event, otherwise it is first started and then immediately gets a file
open event (there is no standard way to define command line variables,
i.e. arguments to a bundled application, as long as you don't want to go
into its internals, you usually use the `open` call for that).
opendiff now does nothing else than packing the given command line
arguments into an event and sends it to the system when then in turn
opens FileMerge.app (the actual standard merger) and tells it to open
the two files.

 mtn cannot pick up the temp file and therefor not the manual merge
 result. Thats why it waits for a key stroke from the user to act
 upon further. And this is actually the default behaviour for the
 default merger opendiff on Mac OS X...
 
 I see that code in std_hooks.lua mergers.opendiff. I'd call this a bug
 in opendiff. But I'm not volunteering to fix it.
 
 So 'mtn au stdio' 'l6updatee' with the standard hooks does require
 prompting via stdio, if the best available merger is opendiff. So we
 (I) have to do something. Sigh.

Really, please don't let it call the external merger - this really gets
messy otherwise. I won't support the inclusion of this code otherwise in
mainline, but would propose you keep that in your own private branch.

 I think we need to make the value of --non-interactive available in
 Lua hooks, so the user overriding a hook can check it before doing
 something that might prompt.

 This sounds more like a workaround for a problem similar to one which
 you already solved quite smartly. Wouldn't be a better solution to
 expand show_conflicts to work in workspace mode and give mtn automate
 update an option --resolve-conflicts-file to resolve these on the fly?
 
 Yes. But that's a lot of work, and I don't think it's worth it. I'm
 also not sure it's possible; I have looked at the way update finds and
 handles conflicts, and it's more complicated than for an in-database
 merge; there are many separate stages, and you can't always back out
 and leave a clean workspace. I'm certainly not willing to hold up
 'gds_update_all' for it. [...]

The problem I have with all the solutions so far for the workspace
merging issue is that whatever we do it ends up as hack - which in turn
adds another layer of complexity into an already complex setup. I'd
rather see a good solution some time than a bad solution tomorrow,
because the bad solution has to be supported from the remaining
developers by the day it wents into mainline (and there are not so many
of us left as you might know).

This is personally a blocker for me - what do other people think about
it? Stephen, would it be possible for you to use a plain (non-automate)
version of update in your gds_update_all command?

Thomas.

-- 
GPG-Key 0x160D1092 | tommyd3...@jabber.ccc.de | http://thomaskeller.biz
Please note that according to the EU law on data retention, information
on every electronic information exchange might be retained for a period
of six months or longer: http://www.vorratsdatenspeicherung.de/?lang=en



signature.asc
Description: OpenPGP digital signature
___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-03-31 Thread Stephen Leake
Thomas Keller m...@thomaskeller.biz writes:

 1) How does automate update behaves (especially over stdio) when a
 workspace merge has to take place? I don't see any special code handling
 that in a sane way.

 I just tried 'mtn automate update' and 'mtn automate stdio'
 'l6:updatee'; it's the same as the current update behavior (pops up a
 separate Emacs session running ediff3). That doesn't prompt the user
 via the stdio channels; it just spawns a separate process, which can
 prompt or otherwise interact with the user. So I don't see a problem.

 The prompt is the problem, especially on stdio.

I guess the spawned process might use the same stdio channel to
prompt.

 Some platforms, f.e. Mac OS X, even require an explicit key stroke
 because the standard merger call doesn't block itself.

Do you mean this is required even with the standard hooks?

The test I did above was on Windows.

Hm. There was something about merger might prompt in the original
Emacs DVC code. I never did understand the use case for that, so it's
gone now.

 Perhaps automate update should interpret --non-interactive to mean
 don't spawn a merger (fail if a merger is needed). That would be
 simpler than setting MTN_MERGE. Actually, that makes sense for
 non-automate update, anyway (I just checked; update does not currently
 respect --non-interactive in this sense).

 --non-interactive is a new option of 0.46 which was introduced exactly
 for this and other use cases. The original rationale was that key
 decryption should not lead to a passphrase prompt in case all other
 methods (phraseless try, ssh-agent and get_passphrase lua hook) did not
 succeeed.

Right. And I just ran into exactly that problem; someone had their
get_passphrase hook written incorrectly, and the Emacs interface just
froze, with no error message. --non-interactive on commit is the
solution.

 This option is set to true in CMD(stdio) and CMD(remote_stdio), so
 it would just be great if the new update command would honor this
 option as well.

Specifically, do you mean fail if manual merge is required and
--non-interactive is set?

I'd rather not, actually. I can now use 'automate update' from my
Emacs front end, via the automate stdio session I maintain for each
workspace. In that case, I want the current behavior of popping up a
separate Emacs for a workspace conflict.

I think we need to make the value of --non-interactive available in
Lua hooks, so the user overriding a hook can check it before doing
something that might prompt.

 If mtn au update is called however, it shouldn't behave any
 different than mtn update. 

Right.

 And finally, I don't think messing with environment variables is a
 good idea here at all... normally stdio instances are ran within
 clean environments and it would be cool if we could just keep that
 as is.

Messing with environments is only necessary if you want non-standard
behavior; I think that would be acceptable.

I'm not clear exactly what you are proposing.

 2) From what I see change_workspace() might potentially open a can of
 worms / bugs, especially on long-running stdio instances where an
 existing database connection is kept open. One could f.e. think a call
 like `automate lua change_workspace /path/to/new/workspace` is harmless,
 while everything might explode (tm) afterwards because the workspace
 points to an entirely different database...

 change_workspace calls the same function that opens a workspace in the
 first place. After that, any code that wants the database associated
 with the current workspace should parse the workspace options, which
 sets the database cache properly. I guess that is probably not true
 now for most Lua extension functions now; there's a huge assumption
 that the workspace doesn't change. But every top-level function (like
 'update' or 'inventory') does that.

 My main fear is that this introduces some kind of stealth workspace /
 database switch within a running stdio instance - i.e. the database
 which is found initially is opened very early before stdio accepts the
 first command (see db.ensure_open() in cmd_automate.cc) - and I have the
 fuzzy feeling that the other assumptions and hackery we do f.e. to reset
 global options for every single automate command execution, might play
 not nice with that. I haven't tested this thoroughly, I'm just not
 comfortable with that.

It's actually better than what I said above. 

The workspace options are read (by calling workspace::get_options) in
three places: cmd.cc commands::process, cmd_automate.cc
LUAEXT(mtn_automate), and CMD_AUTOMATE_NO_STDIO(stdio). So any command
run via mtn_automate will be ok after change_workspace. 

The other Lua extensions don't refresh the workspace options. So
get_confdir is the only Lua extension that could be wrong after
change_workspace. 

But I guess you are worried about someone executing change_workspace
in some Lua hook, and then returning to the stdio process without
calling mtn_automate. But that is 

Re: [Monotone-devel] update multiple projects

2010-03-31 Thread Thomas Keller
Am 31.03.10 13:10, schrieb Stephen Leake:
 Thomas Keller m...@thomaskeller.biz writes:
 
 1) How does automate update behaves (especially over stdio) when a
 workspace merge has to take place? I don't see any special code handling
 that in a sane way.

 I just tried 'mtn automate update' and 'mtn automate stdio'
 'l6:updatee'; it's the same as the current update behavior (pops up a
 separate Emacs session running ediff3). That doesn't prompt the user
 via the stdio channels; it just spawns a separate process, which can
 prompt or otherwise interact with the user. So I don't see a problem.

 The prompt is the problem, especially on stdio.
 
 I guess the spawned process might use the same stdio channel to
 prompt.

Whatever channel it actually uses, stdio users do not expect that a
command prompts for input and with remote stdio its even more unclear
how this should be handled at all.

 Some platforms, f.e. Mac OS X, even require an explicit key stroke
 because the standard merger call doesn't block itself.
 
 Do you mean this is required even with the standard hooks?

I don't understand the question. Currently, whether you do a normal or a
workspace merge, as soon as a content conflict arises, mtn opens a
merger process (might it be vim or kdiff3 or emacs or opendiff) and
waits until this process ends, i.e. returns, to pick up the saved
temporary file afterwards.
The problem is now that the shell call which opens the merger process
has to _block_ until the process actually ends, if it spawns in the
background and returns immediately, mtn cannot pick up the temp file and
therefor not the manual merge result. Thats why it waits for a key
stroke from the user to act upon further. And this is actually the
default behaviour for the default merger opendiff on Mac OS X...

 [--non-interactive] is set to true in CMD(stdio) and CMD(remote_stdio), so
 it would just be great if the new update command would honor this
 option as well.
 
 Specifically, do you mean fail if manual merge is required and
 --non-interactive is set?
 
 I'd rather not, actually. I can now use 'automate update' from my
 Emacs front end, via the automate stdio session I maintain for each
 workspace. In that case, I want the current behavior of popping up a
 separate Emacs for a workspace conflict.
 
 I think we need to make the value of --non-interactive available in
 Lua hooks, so the user overriding a hook can check it before doing
 something that might prompt.

This sounds more like a workaround for a problem similar to one which
you already solved quite smartly. Wouldn't be a better solution to
expand show_conflicts to work in workspace mode and give mtn automate
update an option --resolve-conflicts-file to resolve these on the fly?
If this would be in place, merge_into_workspace could profit from that
as well...

 And finally, I don't think messing with environment variables is a
 good idea here at all... normally stdio instances are ran within
 clean environments and it would be cool if we could just keep that
 as is.
 
 Messing with environments is only necessary if you want non-standard
 behavior; I think that would be acceptable.
 
 I'm not clear exactly what you are proposing.

No, I'm not proposing anything. I just didn't like the idea to tell the
implementor to set MTN_MERGE=foo just to make the internal merger call
fail... ;)

 My main fear is that this introduces some kind of stealth workspace /
 database switch within a running stdio instance - i.e. the database
 which is found initially is opened very early before stdio accepts the
 first command (see db.ensure_open() in cmd_automate.cc) - and I have the
 fuzzy feeling that the other assumptions and hackery we do f.e. to reset
 global options for every single automate command execution, might play
 not nice with that. I haven't tested this thoroughly, I'm just not
 comfortable with that.
 
 It's actually better than what I said above. 
 
 The workspace options are read (by calling workspace::get_options) in
 three places: cmd.cc commands::process, cmd_automate.cc
 LUAEXT(mtn_automate), and CMD_AUTOMATE_NO_STDIO(stdio). So any command
 run via mtn_automate will be ok after change_workspace. 
 
 The other Lua extensions don't refresh the workspace options. So
 get_confdir is the only Lua extension that could be wrong after
 change_workspace. 
 
 But I guess you are worried about someone executing change_workspace
 in some Lua hook, and then returning to the stdio process without
 calling mtn_automate. But that is safe, because each command in
 automate_stdio refreshes the workspace options. Someone was prescient
 when they wrote automate stdio (I think that was you :).
 
 I've updated the manual to say this.

Heh, well, the code which resets the options before every command
execution is hairy and its not better that its used in similar (but not
quite equal) versions in three places: the mtn_automate lua hook,
CMD(stdio) and CMD(remote_stdio). When I hacked on the remote stuff the

Re: [Monotone-devel] update multiple projects

2010-03-31 Thread Stephen Leake
Thomas Keller m...@thomaskeller.biz writes:


 Currently, whether you do a normal or a
 workspace merge, as soon as a content conflict arises, mtn opens a
 merger process (might it be vim or kdiff3 or emacs or opendiff) and
 waits until this process ends, i.e. returns, to pick up the saved
 temporary file afterwards.

Right.

 The problem is now that the shell call which opens the merger
 process has to _block_ until the process actually ends, 

Right.

 if it spawns in the background and returns immediately, 

Why would it do this? Sounds like a bug in the merger hook, or the
merge tool, to me.

There aught to be a way to spawn a process and wait for it; that
certainly works with Emacs on Linux and Windows. 

But there are Windows apps that I would not be able to do that with;
Microsoft Outlook, for instance. I could not find a way to launch that
from a command line; I have to click on the icon. modern GUI OS
interfaces are broken in many ways!

 mtn cannot pick up the temp file and therefor not the manual merge
 result. Thats why it waits for a key stroke from the user to act
 upon further. And this is actually the default behaviour for the
 default merger opendiff on Mac OS X...

I see that code in std_hooks.lua mergers.opendiff. I'd call this a bug
in opendiff. But I'm not volunteering to fix it.

So 'mtn au stdio' 'l6updatee' with the standard hooks does require
prompting via stdio, if the best available merger is opendiff. So we
(I) have to do something. Sigh.

 [--non-interactive] is set to true in CMD(stdio) and CMD(remote_stdio), so
 it would just be great if the new update command would honor this
 option as well.
 
 ...
 
 I think we need to make the value of --non-interactive available in
 Lua hooks, so the user overriding a hook can check it before doing
 something that might prompt.

 This sounds more like a workaround for a problem similar to one which
 you already solved quite smartly. Wouldn't be a better solution to
 expand show_conflicts to work in workspace mode and give mtn automate
 update an option --resolve-conflicts-file to resolve these on the fly?

Yes. But that's a lot of work, and I don't think it's worth it. I'm
also not sure it's possible; I have looked at the way update finds and
handles conflicts, and it's more complicated than for an in-database
merge; there are many separate stages, and you can't always back out
and leave a clean workspace. I'm certainly not willing to hold up
'gds_update_all' for it.

Another alternative is to let update fail if merge3 is required, and
let the user commit, merge the heads, and then update. When there are
major changes, that is certainly the prefered method. But I don't
always do that now when there are only minor changes, that are easy
enough to handle during update. I don't want changing the internals of
the Emacs mtn front-end to be more efficient to force a change in my
style.

So I'd rather go with this solution/workaround:

Add another argument 'non_interactive' to the merge3 Lua hook. That
requires passing that same flag down from 'update' to the actual call
to 'merge3'.

The standard mergers.emacs function should ignore that flag, since it
doesn't prompt; mergers.opendiff should respect it, and fail if it is
true. The user can then use a better merger with update, or commit, or
revert.

This is upward-compatible with existing user-written merge3 hooks. 

That only addresses this one Lua hook. If there are other hooks that
should care about non-interactive, then we either need to do the same
thing with them, or add a general way to access options from Lua. But
merge3 is the only hook relevant to 'mtn au update' (I think), so this
workaround is good for now.

 ...
 Heh, well, the code which resets the options before every command
 execution is hairy and its not better that its used in similar (but not
 quite equal) versions in three places: the mtn_automate lua hook,
 CMD(stdio) and CMD(remote_stdio). 

True. I was thinking we should try to unify that.

 When I hacked on the remote stuff the last time I tried to unify
 this a bit, but this was not easily possible. In the end I still
 felt not comfortable enough to make major changes in this area
 without opening a big can of worms.

Right. But I am comfortable with 'change_workspace', and putting off
unifying the workspace options code until we encounter a real problem :).

 3) It would be cool if you could test the options for automate update a
 bit more

 ...

 But if you're not up
 for the task because you say its already tested for the normal
 commands then I just have to remember that the next time I review some
 code which touches this area.

Ok. Feel free to add some comments to support this.

-- 
-- Stephe


___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-03-30 Thread Thomas Keller
Am 28.03.10 18:17, schrieb Stephen Leake:
 Stephen Leake stephen_le...@stephe-leake.org writes:
 
 function gds_update_all()
local workspaces = gds_find_workspaces (gds_missions)

for _, ws in pairs(workspaces) do
-- cd(../ .. ws)
-- mtn (update)
mtn_automate (update, --workspace= .. ws)
io.stderr:write(ws .. \n)
end
 end
 
 I've started working on this. 'mtn automate update' is implemented;
 see cmd_merging.cc in 2cc8eafc9f3350379e72c47c24e567ffc52d7fbc in
 branch nvm.stephe. There's a test for it. It was easy, since there is
 no output to format (it's progress, not useful for parsing).
 
 I added a new Lua function 'change_workspace' (cmd_automate.cc in
 f0b5f85a97853d52f94fd3aab2606ba093a40bff), which basically calls
 paths.cc go_to_workspace.
 [...] 
 Any comments on 'automate update' or 'change_workspace'?

Looks good in general, a few open points though:

1) How does automate update behaves (especially over stdio) when a
workspace merge has to take place? I don't see any special code handling
that in a sane way.

2) From what I see change_workspace() might potentially open a can of
worms / bugs, especially on long-running stdio instances where an
existing database connection is kept open. One could f.e. think a call
like `automate lua change_workspace /path/to/new/workspace` is harmless,
while everything might explode (tm) afterwards because the workspace
points to an entirely different database...

3) It would be cool if you could test the options for automate update a
bit more and harden the test when it looks for the right progress
message (progress[9] might break at any time in the future when somebody
decides to add another test file, while it possibly should not do so).

4) What do you mean with

+  // need a variant of P that doesn't require F?
+  P(F(f.what()));

?

5) Minor nitpicks:

+  E(args.empty(), origin::user,
+F(update takes no command arguments));

We usually use one single message - wrong argument count - for this.

+  update (app, args);

There is an additional unneeded space after update.


Thomas.

-- 
GPG-Key 0x160D1092 | tommyd3...@jabber.ccc.de | http://thomaskeller.biz
Please note that according to the EU law on data retention, information
on every electronic information exchange might be retained for a period
of six months or longer: http://www.vorratsdatenspeicherung.de/?lang=en



signature.asc
Description: OpenPGP digital signature
___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-03-30 Thread Thomas Keller
Am 31.03.10 00:12, schrieb Stephen Leake:
 I added a new Lua function 'change_workspace' (cmd_automate.cc in
 f0b5f85a97853d52f94fd3aab2606ba093a40bff), which basically calls
 paths.cc go_to_workspace.
 [...] 
 Any comments on 'automate update' or 'change_workspace'?

 Looks good in general, a few open points though:

 1) How does automate update behaves (especially over stdio) when a
 workspace merge has to take place? I don't see any special code handling
 that in a sane way.
 
 I just tried 'mtn automate update' and 'mtn automate stdio'
 'l6:updatee'; it's the same as the current update behavior (pops up a
 separate Emacs session running ediff3). That doesn't prompt the user
 via the stdio channels; it just spawns a separate process, which can
 prompt or otherwise interact with the user. So I don't see a problem.

The prompt is the problem, especially on stdio. Some platforms, f.e. Mac
OS X, even require an explicit key stroke because the standard merger
call doesn't block itself.

 Perhaps automate update should interpret --non-interactive to mean
 don't spawn a merger (fail if a merger is needed). That would be
 simpler than setting MTN_MERGE. Actually, that makes sense for
 non-automate update, anyway (I just checked; update does not currently
 respect --non-interactive in this sense).

--non-interactive is a new option of 0.46 which was introduced exactly
for this and other use cases. The original rationale was that key
decryption should not lead to a passphrase prompt in case all other
methods (phraseless try, ssh-agent and get_passphrase lua hook) did not
succeeed. This option is set to true in CMD(stdio) and
CMD(remote_stdio), so it would just be great if the new update command
would honor this option as well. If mtn au update is called however, it
shouldn't behave any different than mtn update.
And finally, I don't think messing with environment variables is a good
idea here at all... normally stdio instances are ran within clean
environments and it would be cool if we could just keep that as is.

 By the way, it is your recent enhancements to automate stdio (the 'p'
 channel) that made implementing 'automate update' so easy; thanks
 again for that!

Yes thanks, this is more or less a hack to properly encode out-of-band
chatter in the stdio protocol, given the fact that we would have a very
very hard time to properly implement logging / error tracking in mtn
otherwise.

 2) From what I see change_workspace() might potentially open a can of
 worms / bugs, especially on long-running stdio instances where an
 existing database connection is kept open. One could f.e. think a call
 like `automate lua change_workspace /path/to/new/workspace` is harmless,
 while everything might explode (tm) afterwards because the workspace
 points to an entirely different database...
 
 change_workspace calls the same function that opens a workspace in the
 first place. After that, any code that wants the database associated
 with the current workspace should parse the workspace options, which
 sets the database cache properly. I guess that is probably not true
 now for most Lua extension functions now; there's a huge assumption
 that the workspace doesn't change. But every top-level function (like
 'update' or 'inventory') does that.

My main fear is that this introduces some kind of stealth workspace /
database switch within a running stdio instance - i.e. the database
which is found initially is opened very early before stdio accepts the
first command (see db.ensure_open() in cmd_automate.cc) - and I have the
fuzzy feeling that the other assumptions and hackery we do f.e. to reset
global options for every single automate command execution, might play
not nice with that. I haven't tested this thoroughly, I'm just not
comfortable with that.

I understand your use case though and I tend to say you're right - we
should maybe start thinking about fixing things when these actually come up.

 3) It would be cool if you could test the options for automate update a
 bit more 
 
 Since the normal option processing is the same between 'automate
 update' and 'update', I only need to test the error cases (see
 CMD_AUTOMATE(update...) in cmd_merging.cc). I tried to test
 revision_selectors.size()  1, but couldn't build a stdio command
 string that worked. But I don't have to go thru stdio to test that;
 I've added a test for that.

I know this may sound a bit backwards for now, but imagine the
implementation of update and automate update diverge in the future or
new special features are added to the automate version which do not
exist in the non-automate version or the other way around (which might
even be more problematic, because an apparently harmless change to mtn
update could mean incompatible changes to the stdio version) - for all
these use cases it would just be great if a good test case would tell us
hey, the new implementation no longer spits out what the interface
implementor expects in this interface version, adapt 

Re: [Monotone-devel] update multiple projects

2010-03-29 Thread Stephen Leake
Stephen Leake stephen_le...@stephe-leake.org writes:

 ...

 This basically works, but I'm getting errors about writing the options file:

 mtn: warning: Failed to write options file _MTN/options: error: renaming 
 '_MTN/mtdkyfvx.tmp' to '_MTN/options' failed: Access is denied. (5)

I found the problem; the Lua code I was using to test if a directory
is a workspace opens _MTN/options but doesn't close it.

Here's the final Lua code:

--

gds_missions = {makerules, sal, opentoken, common, lro, elc, gpm, 
mms}

function gds_find_workspaces(missions)
   -- Lua doesn't have directory facilities. So we attempt to open
   -- */_MTN/options, where * is a list of known mission names,
   -- which we have to update once in a while.
   local workspaces = {}
   local file

   for _, ws in pairs(missions) do
  file = io.open (ws .. /_MTN/options)
  if file then
 table.insert (workspaces, ws)
 io.close (file)
  end
   end

   return workspaces
end

function gds_update_all()
   local workspaces = gds_find_workspaces (gds_missions)
   local status, msg

   if 0 == # workspaces then
  error(no GDS workspaces found)
   end

   for _, ws in pairs(workspaces) do
   status, msg = change_workspace(ws)
   if not status then
  error (msg)
   end

   status, msg = mtn_automate(update)

   -- msg is stdout; stderr is output already. stderr contains
   -- lots of progress messages for each workspace; summarize
   -- those that failed here. For some reason, all of these are
   -- output after all of the update outputs, so we don't need to
   -- cache the result for each workspace.
   if not status then
  io.stderr:write(ws ..  failed\n)
   end
   end
end

register_command(gds_update_all, , update all GDS workspaces,
  Update GDS workspaces in child directories, gds_update_all)

--

Any objections to merging work_stephe to main? It does two things:

implements 'automate update'

provides a new Lua function 'change_workspace'

I need to add both of these to monotone.texi and NEWS first.

--
-- Stephe


___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel


Re: [Monotone-devel] update multiple projects

2010-03-28 Thread Stephen Leake
Stephen Leake stephen_le...@stephe-leake.org writes:

 function gds_update_all()
local workspaces = gds_find_workspaces (gds_missions)

for _, ws in pairs(workspaces) do
-- cd(../ .. ws)
-- mtn (update)
mtn_automate (update, --workspace= .. ws)
io.stderr:write(ws .. \n)
end
 end

I've started working on this. 'mtn automate update' is implemented;
see cmd_merging.cc in 2cc8eafc9f3350379e72c47c24e567ffc52d7fbc in
branch nvm.stephe. There's a test for it. It was easy, since there is
no output to format (it's progress, not useful for parsing).

I added a new Lua function 'change_workspace' (cmd_automate.cc in
f0b5f85a97853d52f94fd3aab2606ba093a40bff), which basically calls
paths.cc go_to_workspace.

My Lua function now looks like this (the rest is unchanged from the
previous post):

function gds_update_all()
   local workspaces = gds_find_workspaces (gds_missions)
   local status, output

   if 0 == # workspaces then
  error(no GDS workspaces found)
   end

   for _, ws in pairs(workspaces) do
   change_workspace(ws)
   io.stderr:write(ws)
   status, output = mtn_automate(update)

   -- output is stdout; stderr is output already.
   if status ~= 0 then
  io.stderr:write( failed\n)
   else
  io.stderr:write(n)
   end
   end
end

This basically works, but I'm getting errors about writing the options file:

mtn: warning: Failed to write options file _MTN/options: error: renaming 
'_MTN/mtdkyfvx.tmp' to '_MTN/options' failed: Access is denied. (5)

This is comming from the workspace constructor in update.

The actual update succeeds, but the temp options file is left.

It appears that something has the options file locked, but I can't
figure out what. Any hints?

Any comments on 'automate update' or 'change_workspace'?

-- 
-- Stephe


___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel