Re: off-box mirror-stream and friends - hacked to work
Matthew Dillon wrote:
:I have what appears to be a 'Catch 22', wherein:
:
:hammer mirror-stream /master user@remote_IP:/new_slave
:
:returns:
:
:PFS slave /new-slave does not exist.
:Do you want to create a new slave PFS? (yes|no) No terminal for response
:Aborting operation
:validate_mrec_header: short read
:
:'No terminal for response'
Has now been hacked to work when ssh requires passwords,
*so long as*
EITHER:
- the user@target user is 'root' or equivalent.
(testing only - NOT recommended for prime-time!)
ELSE
- there is a mount-point on the target that is NOT in an existing PFS
AND the non-root caller UID/EUID has the requisite privs to that mount
to create dirs, PFS, links, files et al on it.
As with a cpdup or rsync operator, a bespoke user:group and carefully
crafted privs and security are the order of the day.
Ugly hack below - needs more work before a formal patch, but it JFW's.
Regards,
Bill Hacker
- find in /usr/src/sbin/hammer/cmd_mirror.c
AS READS =
/*
* Get a yes or no answer from the terminal. The program may be run as
* part of a two-way pipe so we cannot use stdin for this operation.
*/
static int
getyn(void)
{
char buf[256];
FILE *fp;
int result;
fp = fopen(/dev/tty, r);
if (fp == NULL) {
fprintf(stderr, No terminal for response\n);
return(-1);
}
result = -1;
while (fgets(buf, sizeof(buf), fp) != NULL) {
if (buf[0] == 'y' || buf[0] == 'Y') {
result = 1;
break;
}
if (buf[0] == 'n' || buf[0] == 'N') {
result = 0;
break;
}
fprintf(stderr, Response not understood\n);
break;
}
fclose(fp);
return(result);
}
WAS AMENDED TO READ ===
/*
* If we hadn't meant to do it, we'd not have asked, so fake
* a 'Yes' for now. LATER - remove the call to this entirely
* and replace with a count-down timout and instructions to
* use 'Ctrl c' to Abort. Borrow that code from cmd_pseudofs.c
*/
static int
getyn(void)
{
int result;
result = 1;
return(result);
}
==
Built with 'make' from within /usr/src/stdin by temporarily setting
'subdirs = hammer' (and no others), in the /usr/src/stdin Makefile.
Resulting /usr/src/stdin/hammer/hammer binary, and no others, then
manually copied into /sbin on BOTH source and destination machines.
Working well so far..
Re: off-box mirror-stream and friends
Am Sun, 15 Feb 2009 21:38:54 -0800 (PST) schrieb Matthew Dillon dil...@apollo.backplane.com: :I have what appears to be a 'Catch 22', wherein: : :hammer mirror-stream /master user@remote_IP:/new_slave : :returns: : :PFS slave /new-slave does not exist. :Do you want to create a new slave PFS? (yes|no) No terminal for response :Aborting operation :validate_mrec_header: short read : :'No terminal for response' .was ass u me ed to be a byproduct of comign :in off an Xfce4-terminal (Xorg Xfce4 are quite happy on 2.3.0, BTW) : :Dropped back out to the raw tty0 console and tried it from there. : :No joy. Definitely a bug in the hammer utility, I'm not sure there is anything I can do about it though because the remote ssh connection has no channel to accept a Y or N answer... stdin and stdout are used for the protocol stream and I think stderr is output only. In anycase, I think what this means is that this feature currently only works if the slave is local (non-ssh connection). So you would be able to do it with remote_master local_slave. Hm, I remember that we implemented this feature (auto-creation of slaves) so that it can operate over ssh. And IIRC, it once worked for me using ssh (I am not sure if this was a remote machine or not). Does this mean, it is broken? :Command *appear* to succeed if/as/when I *manually* create 'new_slave' :in advance with a matching shared_uuid. A local mirror-copy to it :suceeds, with new_slave showing the files mirrored. : :However, while the -vvv flag gives 5-sec updates, they all show a newer :starting point that pfs-status has for the target, and the contents of :the slave never change. You must access the slave via its softlink to get the latest version synced from the master. If you try to access the slave via a null-mount you will be accessing a snapshot of the slave, not the current state of the slave. The null mount locks in the transaction id of the slave. :By way of contrast, mirror-stream between on-box master and on-box slave : - same command otherwise - works fine. No chdir needed to see the :updates, just a 'View, Reload' in thunar and sputniks. You are probably accessing it via the softlink, yes? The gui is probably using an absolute path. If you were to CD into a sub-directory (even through the softlink), you would be accessing a snapshot as-of when you did the CD, not the latest synced copy. :Query: Can the loop that seeks a 'yes' be changed to a 5-second :countdown-timer with a message such as: : :Creating new_slave Hit Ctrl-c to abort : :.absent which it JFDI. : :Thanks, : :Bill Hacker That won't work, the target over an ssh link has no tty channel. Adding an option to create the slave automatically and passing it to the target hammer utility when it is run via the ssh, so it never has to ask at all, would work. If someone would like to do that and submit a patch, I don't think it would take more then 20 minutes of programming. You mean, something like a -f (force) option? Should be damn easy to implement. I can do that, once I sit in front of a real computer (with DragonFly) again :) Regards, Michael
Re: off-box mirror-stream and friends
Am Mon, 16 Feb 2009 11:45:45 +0100 schrieb Michael Neumann mneum...@ntecs.de: Am Sun, 15 Feb 2009 21:38:54 -0800 (PST) schrieb Matthew Dillon dil...@apollo.backplane.com: :I have what appears to be a 'Catch 22', wherein: : :hammer mirror-stream /master user@remote_IP:/new_slave : :returns: : :PFS slave /new-slave does not exist. :Do you want to create a new slave PFS? (yes|no) No terminal for response :Aborting operation :validate_mrec_header: short read : :'No terminal for response' .was ass u me ed to be a byproduct of comign :in off an Xfce4-terminal (Xorg Xfce4 are quite happy on 2.3.0, BTW) : :Dropped back out to the raw tty0 console and tried it from there. : :No joy. Definitely a bug in the hammer utility, I'm not sure there is anything I can do about it though because the remote ssh connection has no channel to accept a Y or N answer... stdin and stdout are used for the protocol stream and I think stderr is output only. In anycase, I think what this means is that this feature currently only works if the slave is local (non-ssh connection). So you would be able to do it with remote_master local_slave. Hm, I remember that we implemented this feature (auto-creation of slaves) so that it can operate over ssh. And IIRC, it once worked for me using ssh (I am not sure if this was a remote machine or not). Does this mean, it is broken? Looking at function getyn() posted by Bill Hacker I eventually understand what's wrong. Code - the universal language... So, really, the best thing that we can do is to introduce a --force-slave-pfs-creation switch and replace the getyn() call in cmd_mirror.c by a simple if (ForceSlavePfsCreation...). I like this more than the original approach (using ttys), as it is usable from within scripts. While we are working on this, we could replace all interactivity in hammer utilities with optional command line switches. So the strategy could be: first look if there is a switch specified, if not, fall back to /dev/tty, if this fails, assume NO. I do not like a command line switch -f, which means anything (including force). It's too easy to mix it up with -f meaning --file. Think about doing some admin work late after midnight, and your fingers go again faster than your brain; while a rm -rf on a Hammer FS would give you a second chance, a mixed up hammer mirror-copy -f would probably not :). I really would like to see a --force here (or a more specific one). Regards, Michael
Re: off-box mirror-stream and friends
:You mean, something like a -f (force) option? Should be damn easy to :implement. I can do that, once I sit in front of a real computer :(with DragonFly) again :) : :Regards, : : Michael Yah, go for it. -Matt Matthew Dillon dil...@backplane.com
off-box mirror-stream and friends
I have what appears to be a 'Catch 22', wherein: hammer mirror-stream /master user@remote_IP:/new_slave returns: PFS slave /new-slave does not exist. Do you want to create a new slave PFS? (yes|no) No terminal for response Aborting operation validate_mrec_header: short read 'No terminal for response' .was ass u me ed to be a byproduct of comign in off an Xfce4-terminal (Xorg Xfce4 are quite happy on 2.3.0, BTW) Dropped back out to the raw tty0 console and tried it from there. No joy. Not known to me (with only two DFLY boxen so far) if that would succeed is BOTH source and destination were remote from the box making the call. It does not suceed if he IP of the local host is used - only adds the inability to get both passwords into the waiting mouths. Command *appear* to succeed if/as/when I *manually* create 'new_slave' in advance with a matching shared_uuid. A local mirror-copy to it suceeds, with new_slave showing the files mirrored. However, while the -vvv flag gives 5-sec updates, they all show a newer starting point that pfs-status has for the target, and the contents of the slave never change. That holds true even if I add, edit, or delete files on the /master, and if I reblock the master after so doing. By way of contrast, mirror-stream between on-box master and on-box slave - same command otherwise - works fine. No chdir needed to see the updates, just a 'View, Reload' in thunar and sputniks. Query: Can the loop that seeks a 'yes' be changed to a 5-second countdown-timer with a message such as: Creating new_slave Hit Ctrl-c to abort .absent which it JFDI. Thanks, Bill Hacker
Re: off-box mirror-stream and friends
Matthew Dillon wrote: :I have what appears to be a 'Catch 22', wherein: : :hammer mirror-stream /master user@remote_IP:/new_slave : :returns: : :PFS slave /new-slave does not exist. :Do you want to create a new slave PFS? (yes|no) No terminal for response :Aborting operation :validate_mrec_header: short read : :'No terminal for response' .was ass u me ed to be a byproduct of comign :in off an Xfce4-terminal (Xorg Xfce4 are quite happy on 2.3.0, BTW) : :Dropped back out to the raw tty0 console and tried it from there. : :No joy. Definitely a bug in the hammer utility, I'm not sure there is anything I can do about it though because the remote ssh connection has no channel to accept a Y or N answer... stdin and stdout are used for the protocol stream and I think stderr is output only. Found and am working on a fix. in /usr/src/sbin/hammer/cmd_hammer.c the plan is to remove the request for a [yes|no] response and replace it with the countdown-timer that pfs-destroy returns. Not having coded C since 386-16 was brand new doesn't help, but, 'Hacker' has been the family name since some time prior to 1400 AD. ;-) In anycase, I think what this means is that this feature currently only works if the slave is local (non-ssh connection). So you would be able to do it with remote_master local_slave. Had done. Still no joy. likewise form a 'disinterested third party' wherein both ends (appeared to be) remote. No difference if on the console or via Xorg, either. :Command *appear* to succeed if/as/when I *manually* create 'new_slave' :in advance with a matching shared_uuid. A local mirror-copy to it :suceeds, with new_slave showing the files mirrored. : :However, while the -vvv flag gives 5-sec updates, they all show a newer :starting point that pfs-status has for the target, and the contents of :the slave never change. You must access the slave via its softlink to get the latest version synced from the master. If you try to access the slave via a null-mount you will be accessing a snapshot of the slave, not the current state of the slave. The null mount locks in the transaction id of the slave. Thanks - that will help... :By way of contrast, mirror-stream between on-box master and on-box slave : - same command otherwise - works fine. No chdir needed to see the :updates, just a 'View, Reload' in thunar and sputniks. You are probably accessing it via the softlink, yes? The gui is probably using an absolute path. If you were to CD into a sub-directory (even through the softlink), you would be accessing a snapshot as-of when you did the CD, not the latest synced copy. 'Yes but'. Will have to reconfirm, but I can see the softlink, its target, and the mount. Thunar reports the softlin as a 'broken softlink' Thunar rads the nullfs mount in a 5-sec-lag state Thunar sees the @@string25 pfs as frozen at time of creation. AFAIK, this means the mirror-stream is updating what *started as* a snapshot. 'll -F' at the CLI is, if anything, less helpful hammer pfs-status helps. :Query: Can the loop that seeks a 'yes' be changed to a 5-second :countdown-timer with a message such as: : :Creating new_slave Hit Ctrl-c to abort : :.absent which it JFDI. : :Thanks, : :Bill Hacker That won't work, the target over an ssh link has no tty channel. Adding an option to create the slave automatically and passing it to the target hammer utility when it is run via the ssh, so it never has to ask at all, would work. If someone would like to do that and submit a patch, I don't think it would take more then 20 minutes of programming. -Matt Matthew Dillon dil...@backplane.com Beavering away. Perhaps under five minutes of programming for whomever wrote it (Michael? Thomas?). 2 days of researching what and how 'C' does with return variables, and how to sort what was in the mind of the original author when he asked for them where he did.. (why 'tty'' and-not stdin - which accepts the password OK). And should the ssh session have been called with different flags, or least the calling code made to pop-out at internediate points for interactive response, or Not even sure if I should be using 'make', 'bmake' or just 'cc', as my binaries so far have *way* diffent sizes from stock. They all run, oddly enough.. just not yet as wanted. but I'll deal with that... just installed 'heme' as all I *really* need to do is change the one byte in the binary returned by the (failed) tty query to fake success, ELSE a JMP past the query routine altogether. Bill
