Hi David,
I've implemented the 2+3 combination we talked about
last month (had a few weeks of holidays, hence the delay).
The webrev for the new changes is
http://cr.opensolaris.org/~jmcp/gk_tools_update_3/webrev/
On a push, we now see output similar to the following:
============================================================
blinder:minirepo_test $ hg push -v
ssh://o...@localhost//scratch/gate/trees/minirepo
running ssh o...@localhost "hg -R /scratch/gate/trees/minirepo serve --stdio"
pushing to ssh://o...@localhost//scratch/gate/trees/minirepo
searching for changes
calling hook preoutgoing.cdm_pbconfirm: hgext_cdm.pbconfirm
Are you sure you wish to push? [y/N]: y
1 changesets found
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote:
remote:
remote: DO NOT HIT CONTROL C
remote:
remote:
remote: pushing to /scratch/gate/trees/minirepo-clone
remote: searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: 1 files updated, 0 files merged, 0 files removed, 0 files unresolved
remote: Preparing notification email...
remote: ...notification sent
============================================================
The gate-external whining script is, appropriately,
lockfilewhiner.py. It produces output like this:
============================================================
$ python lockfilewhiner.py -R /scratch/gate/trees/minirepo -T 5
Dangling push locks for /scratch/gate/trees/minirepo: investigation required
List of dangling lockfiles in /scratch/gate/trees/minirepo/public/lock
gitdiffs-7e9d87a32fe2b5b7bbacb27dd277063c065e81ea
gitdiffs-318f9248f4fc20dbfac8b5ee38e8e69353db66c9
gitdiffs-a5ecd0ed02b0ef9e6d66a4264267dee318176ec7
gknotify-7e9d87a32fe2b5b7bbacb27dd277063c065e81ea
gknotify-318f9248f4fc20dbfac8b5ee38e8e69353db66c9
gknotify-a5ecd0ed02b0ef9e6d66a4264267dee318176ec7
webrev-7e9d87a32fe2b5b7bbacb27dd277063c065e81ea
webrev-a5ecd0ed02b0ef9e6d66a4264267dee318176ec7
webrev-318f9248f4fc20dbfac8b5ee38e8e69353db66c9
gknotify-894903ef004b58ce935dbba1899e94cf5ea9631a
gitdiffs-894903ef004b58ce935dbba1899e94cf5ea9631a
To obtain the changeset details, run the following
commands:
$ hg log -R /scratch/gate/trees/minirepo -r
7e9d87a32fe2b5b7bbacb27dd277063c065e81ea
$ hg log -R /scratch/gate/trees/minirepo -r
318f9248f4fc20dbfac8b5ee38e8e69353db66c9
$ hg log -R /scratch/gate/trees/minirepo -r
a5ecd0ed02b0ef9e6d66a4264267dee318176ec7
$ hg log -R /scratch/gate/trees/minirepo -r
894903ef004b58ce935dbba1899e94cf5ea9631a
============================================================
[Note that the the version in the webrev is printing to
sys.stderr, and the GateMail m.write() calls are commented
out. This is because I think I've got my test repo config
wrong for GateMail and didn't want to go debugging it late
last night :)]
The incremental build script has locking updates to work
with 1.3.1 (tested on a test buildbox that's setup just like
juarez, but with OSOL and 1.3.1).
Finally, the SUNWonbld-update.py changes get us past the
issues with installing incorrect binaries, and provide a
symlink instead. I'd really like to have a fat SUNWonbld
package instead of this, but there are a few other issues
at play before I can get us to that point, like facets.
Thanks again for your help with all this.
James
On 11/12/09 01:51 AM, David Marker wrote:
Well the nice thing about the hooks is you can re-run
almost all of them from the command line. That was
a major goal I had after dealing with ksh bits and
pieces running only from email :)
So I like the (2)/(3) combo.
You can have a pretxnchangegroup that touches
/some/path/to/pushes/revA-revB
And your last bg-hook can remove it.
Then a cron job just checks for anything that has been
in the pushes dir for> n minutes.
If it emails you, you tell the miscreant gateling
to stop hitting CTRL-C and go re-run the missing
hooks (which conveniently have a rev start and end).
Also I think (3) means you are fine with exactly what
you have now. You can update hg, and implement (3)
later at your convenience. After all, most ON
gatelings *do* use Cadmium.
I have no problem with you ditching bg-hook, but it
does make peeps happier than waiting for 5 minutes
while their webrev is created. The less they wait,
the less likely they are to panic and start hitting
CTRL-C.
If hg ever fixes the delay in sending messages
back to user (ui.write("...") doesn't show up till
everything is done) then IMO bg-hook would not
be worth the trouble. Sending the user status would
be sufficient.
-dvd
On Dec 10, 2009, at 12:15 AM, James C. McPherson wrote:
[...]
So. Having gotten out the drawing board, and gone over
everything again and bounced ideas around with Rich, there
is a window of fail which we are susceptible to if I could
figure out the necessary magic to make bghooks work with the
pending flag.
There is also a window of fail with not using bghooks, if
gatelings aren't using cdm.
The primary problem with bghooks and the pending flag is that
while I am confident I could eventually figure out the right way
to get the 00changelog.i.a used for the localrepo instance,
because bghooks is running as a distinct process from the
parent hg, it's highlylikely that by the time any of the other
background hooks run then the 00changelog.i.a will have disappeared.
This is because the next block of that function calls finalize()
on the changelog object.
I thought about trying to block ^C on the server side, but that
depends on having a tty to futz with - and apart from not seeing
an opportunity for the remote hg to have a tty of any sort, bghooks
explicitly close off stdin, stdout and stderr. I could also try
writing a new hook to go in as a pre... hook, but that runs into the
no tty problem as well.
I've also tried hooking into pretxncommit and pretxnupdate - neither
of these work with bghook, and neither of them fail to stop if we hit
^C when pushing.
So as I see things, I'm left with these options
(1) ditch use of bghook.py, and run the bg-* hooks as other
ordinary in-process hooks in the changegroup section
(2) use bghook.py, and run the bg-* hooks as bg-hooks in the
bg-changegroup section
(3) combine either of 1 or 2 with an external-to-the-gate
cronjob which checks that pushes have resulted in webrevs
or gitdiffs. If this cronjob has work to do, it can run the
hooks by hand.
(4) email on-all to remind gatelings that if they're not using
cdm then they'll get no biscuit, and request that the ON CRT
check this as best they can, too (and update pbconfirm() to
utter a "DO NOT HIT ^C" or something)
or finally,
(5) throw my hands up in disgust, admit defeat and walk away.
I don't particularly like 5.
I prefer 1 and 4, because I'm at the end of my capabilities here,
and I rate the non-use of cdm fail window to be lower risk than
hooks failing because changesets have disappeared from what they
believe to be reality.
The only way is up, I'm sure of it.
In bghook.py:
227 don't think you need this line. List will be created in comprehension on
line 228.
I'll put that down as a "c-ism" :-)
fixed.
231 you raise, but didn't send email off first, I think you want an m.send()
I think you're right, there.
fixed.
thanks again,
James
--
Senior Kernel Software Engineer, Solaris
Sun Microsystems
http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog
--
James C. McPherson
--
Senior Kernel Software Engineer, Solaris
Sun Microsystems
http://blogs.sun.com/jmcp http://www.jmcp.homeunix.com/blog
_______________________________________________
tools-discuss mailing list
tools-discuss@opensolaris.org
