On Fri, Apr 3, 2009 at 11:01 PM, Paul Baumgart <[email protected]> wrote: > Yeah, I took the change you pushed and added to that support for more than > one exclude option, because otherwise the exclude parameter isn't nearly as > useful as it could be.
Ok, applied. Thanks! > > Sorry if I wasn't clear. > > Paul > > On Fri, Apr 3, 2009 at 1:41 PM, Christian Vest Hansen <[email protected]> > wrote: >> >> I think you misunderstood me! >> >> I had already pushed a change. Or maybe you're building on top of that? >> >> On Fri, Apr 3, 2009 at 10:34 PM, Paul Baumgart <[email protected]> wrote: >> > Ok, fair enough. :-) >> > >> > How's this then: >> > >> > @operation >> > def rsync_project(remotedir, exclude=[], delete=False, extra_opts='', >> > **kwargs): >> > """ >> > Uploads the current project directory using rsync. >> > By using rsync, only changes since the last upload are actually sent >> > over >> > the wire, rather than the whole directory like when using >> > upload_project. >> > >> > Requires the rsync command-line utility to be available both on the >> > local >> > and the remote machine. >> > >> > Parameters are: >> > remotedir: the directory on the remote machine to which >> > to >> > rsync the current project. The project >> > directory >> > becomes a subdirectory of the remotedir. >> > exclude (optional): values passed to rsync's --exclude option. >> > If the parameter object is iterable, (that >> > is, >> > it >> > defines an __iter__ method), each of its >> > elements is >> > passed to a separate --exclude option. >> > Otherwise, >> > the object is passed as a string to a single >> > --exclude option. >> > See the rsync manpage for details on >> > specifying >> > filter rule arguments for --exclude. >> > delete (optional): True or False, whether to delete remote >> > files >> > that >> > don't exist locally. Defaults to False. >> > extra_opts (optional): Additional command-line options to set >> > for >> > rsync. >> > >> > The rsync command is built from the options as follows: >> > rsync [--delete] [--exclude exclude[0][, --exclude[1][, ...]]] >> > \\ >> > -pthrvz [extra_opts] ../<project dir> >> > <fab_user>@<host>:<remotedir> >> > """ >> > username = ENV.get('fab_user') >> > >> > if not hasattr(exclude, '__iter__'): >> > exclude = [exclude] >> > >> > exclude_opts = ' --exclude "%s"' * len(exclude) >> > exclusions = tuple([str(s).replace('"', '\\\\"') for s in exclude]) >> > >> > options_map = { >> > "delete" : '--delete' if delete else '', >> > "exclude" : exclude_opts % exclusions, >> > "extra" : extra_opts >> > } >> > options = "%(delete)s%(exclude)s -pthrvz %(extra)s" % options_map >> > cwd = '../' + os.getcwd().split(os.sep)[-1] >> > userhost = "$(fab_user)@$(fab_host)" >> > rdir = _lazy_format(remotedir, ENV) >> > >> > cmd = "rsync %s %s %s:%s" % (options, cwd, userhost, rdir) >> > local_per_host(cmd, **kwargs) >> > >> > >> > Besides reducing the length of lines, I also made it more pythonic by >> > using >> > duck typing to determine how to handle the exclude parameter. So now you >> > can >> > pass integers, complex numbers, or anything else with a __str__ method >> > and >> > it will work. >> > >> > I also cleaned up the documentation a bit and made some parts of it more >> > precise. >> > >> > Paul >> > >> > >> > >> > >> > >> > On Fri, Apr 3, 2009 at 10:50 AM, Christian Vest Hansen >> > <[email protected]> wrote: >> >> >> >> Good idea. But as you may have guessed, I'm not fond of long lines :) >> >> >> >> On Fri, Apr 3, 2009 at 5:21 AM, Paul Baumgart <[email protected]> wrote: >> >> > Hi Christian, >> >> > >> >> > Thanks, your version looks a lot cleaner. >> >> > >> >> > I would like to add one thing, because I think it makes the exclude >> >> > parameter much more useful: >> >> > >> >> > @operation >> >> > def rsync_project(remotepath, exclude=False, delete=False, >> >> > extra_opts='', >> >> > **kwargs): >> >> > """ >> >> > Uploads the current project directory using rsync. >> >> > By using rsync, only changes since last upload are actually sent >> >> > over >> >> > the wire, rather than the whole directory like using >> >> > upload_project. >> >> > >> >> > Requires the rsync command-line utility to be available both on >> >> > the >> >> > local >> >> > and the remote machine. >> >> > >> >> > Parameters are: >> >> > remotepath: the path on the remote machine to which to >> >> > rsync >> >> > the >> >> > current project >> >> > exclude (optional): values passed to rsync's --exclude option. >> >> > If the parameter is iterable, each of its >> >> > elements >> >> > is passed to a separate --exclude >> >> > argument. >> >> > If >> >> > the >> >> > parameter is a string, that string is the >> >> > value >> >> > passed to the single --exclude argument. >> >> > See the rsync manpage for details on using >> >> > the >> >> > --exclude option. >> >> > delete (optional): True or False, whether to delete remote >> >> > files >> >> > that >> >> > don't exist locally. >> >> > extra_opts (optional): Additional command-line options to set >> >> > for >> >> > rsync. >> >> > >> >> > The rsync command is built from the options as follows: >> >> > rsync [--delete] [--exclude exclude] -pthrvz [extra_opts] \\ >> >> > <project dir> <fab_user>@<host>:<remotepath> >> >> > """ >> >> > username = ENV.get('fab_user') >> >> > >> >> > if isinstance(exclude, basestring): >> >> > exclude = [exclude] >> >> > >> >> > options_map = { >> >> > "delete" : '--delete' if delete else '', >> >> > "exclude" : exclude and ' '.join(['--exclude "%s"' % >> >> > opt.replace('"', >> >> > '\\\\"') for opt in exclude]) or '', >> >> > "extra" : extra_opts >> >> > } >> >> > options = "%(delete)s %(exclude)s -pthrvz %(extra)s" % options_map >> >> > cwd = '../' + os.getcwd().split(os.sep)[-1] >> >> > userhost = "$(fab_user)@$(fab_host)" >> >> > rpath = _lazy_format(remotepath, ENV) >> >> > >> >> > cmd = "rsync %s %s %s:%s" % (options, cwd, userhost, rpath) >> >> > local_per_host(cmd, **kwargs) >> >> > >> >> > So, instead of taking just a string, it can take either a string or a >> >> > list, >> >> > and multiple --exclude options are generated from a list. A string >> >> > parameter >> >> > is treated just as it is currently. >> >> > >> >> > Paul >> >> > >> >> > On Thu, Apr 2, 2009 at 12:38 PM, Christian Vest Hansen >> >> > <[email protected]> wrote: >> >> >> Ok. I implemented this change, but I modified it a little bit. >> >> >> >> >> >> On Thu, Apr 2, 2009 at 9:17 PM, Christian Vest Hansen >> >> >> <[email protected]> wrote: >> >> >>> On Sun, Mar 22, 2009 at 12:51 AM, Paul Baumgart <[email protected]> >> >> >>> wrote: >> >> >>>> A different topic this time: >> >> >>>> >> >> >>>> The current upload_project() function is a neat feature, but I >> >> >>>> have >> >> >>>> two issues with it: >> >> >>>> >> >> >>>> 1) It doesn't allow me to exclude certain files/directories (like >> >> >>>> .git* for example) >> >> >>>> 2) It uploads the entire project every time, which can be >> >> >>>> irritating >> >> >>>> if the change is small but the project is large. >> >> >>>> >> >> >>>> So, I made this function, which uses rsync to only upload the >> >> >>>> differences between the local and remote copies. >> >> >>>> >> >> >>>> I would like to get feedback on it, primarily as to whether this >> >> >>>> would >> >> >>>> be useful in the mainline Fabric code, and if not, if there is any >> >> >>>> way >> >> >>>> to make it be useful. >> >> >>>> >> >> >>>> @operation >> >> >>>> @connects >> >> >>>> def rsync_project(host, client, env, remotepath, exclude=False, >> >> >>>> delete=False, extra_opts='', **kwargs): >> >> >>>> """ >> >> >>>> Uploads the current project directory using rsync, so only >> >> >>>> changes >> >> >>>> are >> >> >>>> uploaded rather than the whole directory like using >> >> >>>> upload_project. >> >> >>>> >> >> >>>> Requires the rsync command-line utility to be available both on >> >> >>>> the >> >> >>>> local >> >> >>>> and the remote machine. >> >> >>>> >> >> >>>> Parameters are: >> >> >>>> remotepath: the path on the remote machine to which >> >> >>>> to >> >> >>>> rsync the >> >> >>>> current project >> >> >>>> exclude (optional): the string passed to rsync's --exclude >> >> >>>> option. >> >> >>>> See the rsync manpage for details. >> >> >>>> delete (optional): True or False, whether to delete remote >> >> >>>> files >> >> >>>> that >> >> >>>> don't exist locally. >> >> >>>> extra_opts (optional): Additional command-line options to >> >> >>>> set >> >> >>>> for >> >> >>>> rsync. >> >> >>>> >> >> >>>> The rsync command is built from the options as follows: >> >> >>>> rsync [--delete] [--exclude exclude] -pthrvz [extra_opts] >> >> >>>> <project dir> <fab_user>@<host>:<remotepath> >> >> >>>> """ >> >> >>>> >> >> >>>> remotepath = _lazy_format(remotepath, env) >> >> >>>> >> >> >>>> username = ENV.get('fab_user') >> >> >>>> username = username + '@' if username else username >> >> >>>> >> >> >>>> cwd_name = '../' + os.getcwd().split(os.sep)[-1] >> >> >>> >> >> >>> This line right here. Why do you go through all this trouble to >> >> >>> build >> >> >>> a relative CWD? >> >> >> >> >> >> Elementary, dear Watson. To get a prettier output when the assembled >> >> >> command is printed to the console. >> >> >> >> >> >>> >> >> >>> >> >> >>>> >> >> >>>> delete_opt = '--delete' if delete else '' >> >> >>>> >> >> >>>> exclude_opt = '--exclude' if exclude else '' >> >> >>>> exclude = '"' + exclude.strip('"') + '"' if exclude else '' >> >> >>>> >> >> >>>> return local('rsync %(delete_opt)s %(exclude_opt)s %(exclude)s >> >> >>>> -pthrvz %(extra_opts)s %(cwd_name)s >> >> >>>> %(username)s%(host)s:%(remotepath)s' >> >> >>>> % locals(), **kwargs) == 0 >> >> >>>> >> >> >>>> >> >> >>>> Note that it requires adding a line to the bottom of local(): >> >> >>>> >> >> >>>> return retcode >> >> >>>> >> >> >>>> Regards, >> >> >>>> Paul >> >> >>>> >> >> >>>> >> >> >>>> _______________________________________________ >> >> >>>> Fab-user mailing list >> >> >>>> [email protected] >> >> >>>> http://lists.nongnu.org/mailman/listinfo/fab-user >> >> >>>> >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> Venlig hilsen / Kind regards, >> >> >>> Christian Vest Hansen. >> >> >>> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Venlig hilsen / Kind regards, >> >> >> Christian Vest Hansen. >> >> >> >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Venlig hilsen / Kind regards, >> >> Christian Vest Hansen. >> > >> > >> >> >> >> -- >> Venlig hilsen / Kind regards, >> Christian Vest Hansen. > > -- Venlig hilsen / Kind regards, Christian Vest Hansen. _______________________________________________ Fab-user mailing list [email protected] http://lists.nongnu.org/mailman/listinfo/fab-user
