Re: status of sharebox-fs?
On Wed, 23 May 2012 13:42:39 -0400 Joey Hess j...@kitenet.net wrote: I suppose it would not be out of place to mention the Kickstarter project I have just launched today. The first 1/3rd of it will be using inotify to automatically add files, and keep a directory in sync. http://www.kickstarter.com/projects/joeyh/git-annex-assistant-like-dropbox-but-with-your-own very neat. looking fwd to it. wrt. your notes about dvcs-autosync. the most frustrating part about it that I've found is that there's some bad race conditions which are basically not solveable because of some assumptions git makes. I did some research myself and tried to explain it in detail @ http://comments.gmane.org/gmane.comp.version-control.home-dir/665 signature.asc Description: PGP signature ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: dvcs-autosync+git is broken by design
On Sun, 4 Mar 2012 22:39:22 + Adam Spiers vcs-h...@adamspiers.org wrote: Sure - I don't mind changes to existing files being autosynced, but I want to manually add new files to the repo myself. Then you should change startupcmd = git add -A to git add -u, although that's vulnerable to race conditions: if you're updating files while 'git add -u' is running, it may abort. The wrapper script I showed should probably be expanded with '-u' and '-A' like support. Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: dvcs-autosync+git is broken by design
PS: in retrospect, my mail subject may sound worse than it should, sorry about that. ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: dvcs-autosync+git is broken by design
On Sun, 4 Mar 2012 17:19:25 + Adam Spiers vcs-h...@adamspiers.org wrote: I have not yet used dvcs-autosync in anger; what do you mean? addcmd = echo New untracked file %s Am I right in thinking that in this case, there is no synchronisation issue to worry about? synchronisation issue? what do you mean? what do you try to do? also, there are other callbacks which can run a 'git add' (such as a move which by default is git rm, git add, AFAIK) Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: variables for plug-ins
On Thu, 1 Dec 2011 11:47:41 + Adam Spiers vcs-h...@adamspiers.org wrote: Does that sound reasonable? FWIW I'm not too familiar with mr's internals, so I can't answer that question, but you seem to know what you're talking about :) Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: various suggestions for mr
Sorry that I go slightly off-topic but you mentioned Gnu Stow, I looked it up and it seems very nice. i haven't run it yet, but (for those who don't want to read the long description) from the description it seems like a simple and elegant tool, which you give a directory (i want symlinks here) and a bunch of package directories (all files in here must be symlinked) and it will do the right thing on package additions and removals. Some people have actually already written symlink-managers, probably not knowing Stow existed. So yeah, looks neat. Carry on the discussion now :) Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: various suggestions for mr
On Fri, 28 Oct 2011 14:34:16 +0100 Adam Spiers vcs-h...@adamspiers.org wrote: On Fri, Oct 28, 2011 at 12:40 PM, Dieter Plaetinck die...@plaetinck.be wrote: Sorry that I go slightly off-topic but you mentioned Gnu Stow, I looked it up and it seems very nice. i haven't run it yet, but (for those who don't want to read the long description) from the description it seems like a simple and elegant tool, which you give a directory (i want symlinks here) and a bunch of package directories (all files in here must be symlinked) and it will do the right thing on package additions and removals. Yes, it is pretty nice (although the code base is very old-fashioned). One thing it does which I guess other symlink managers wouldn't, is called tree folding - i.e. it makes intelligent decisions about when to symlink to a subdirectory vs. to individual files within that subtree: http://www.gnu.org/software/stow/manual.html#SEC4 IMHO this feature is just common sense and it's among the first things I think of when I'm thinking what would I want a symlink manager to do?, so I would expect that people who implement symlink managers either also do something like this, or at least list it as a todo. For example, imagine I have a stow package called 'foo' which wants to install a file to ~/local/lib/perl/Acme/Foo.pm (by install, I mean set up a symlink so that that path points to the Foo.pm inside the stow package). If ~/local doesn't already exist, and 'foo' is the only stow package which installs anywhere under ~/local, then stow's tree folding feature will ensure that ~/local is a symlink back to $STOW_DIR/foo/local/lib/perl/Acme/Foo.pm. Don't you mean to $STOW_DIR/foo/local ? However I might very well want to manually place other files inside ~/local which have nothing to do with stow, let alone the 'foo' stow package. It might make sense to have ~/local/lib/perl/Acme/ be a symlink to $STOW_DIR/foo/local/lib/perl/Acme/, but symlinks higher up the tree would be undesirable. This is easily overcome by creating a special stow package (which I call 'ANTIFOLD') this seems a bit messy though. Once you go the way of having a tool automagically manage all your symlinks, why not just have the discipline to put all your files in appropriate packages? so that you never _need_ to create antifold packages? what you're doing seems a bit like running into the opposite direction. Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: DVCS-Autosync: syncmethod = none
On Mon, 24 Oct 2011 21:52:47 +0200 Sven Klomp m...@klomp.eu wrote: Hi, I just discovered dvcs-autosync as an good alternative to sparkleshare. Especially for headless operation. However, I ran into some problems: I changed the syncmethod to none, since I dont want to use Jabber. I thought, that autosync will check every xx minutes for changes on the git server, but nothing happens. Is it not implemented? I was used to it with sparkleshare and it is quite handy. How does autosync get the host name? I get following commit message with hostename None: Autocommit of file ... changed on host None Autosync makes correct commits to the repository, however, no push is performed thereafter: DEBUG:root:Substituting cmd part %s with Autocommit of file /srv/http/pim/calendar2.ics changed on host None DEBUG:root:Command 'git commit -m Autocommit of file /srv/http/pim/calendar2.ics changed on host None' in '/srv/http/pim'. Output: [master 6c055fd] Autocommit of file /srv/http/pim/calendar2.ics changed on host None 1 files changed, 0 insertions(+), 9021 deletions(-) delete mode 100644 calendar2.ics DEBUG:root:git reported that there is nothing to commit, not touching commit timer Or is it deactivated if syncmethod = none? Thanks for any help Sven ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home IIRC syncmethod = none really does not do any syncing.. although I can't look up right now to back up my claim, but i remember when the feature was introduced. Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
getting app developers to separate user config and app state info
On Thu, 7 Apr 2011 23:42:26 +0200 Richard Hartmann richih.mailingl...@gmail.com wrote: On Thu, Apr 7, 2011 at 22:40, Dieter Plaetinck die...@plaetinck.be wrote: Me too. But I got kinda demotivated after my previous attempts to get it going failed. Feel free to give it another go, you have my full support. Who else is in favour of this? Do we want to hammer out a full proposal on this list, first? Dieter, can you send any and all suggestions, planning and brainstorming to me/this list so we have it in one place? I think all my ideas are in the `What constitutes user configuration files in the XDG basedir spec?` thread, but I think that thread spans several months, is there a good archive reader to see the entire thread? If so, start reading that :) I have some more notes at home, if anyone knows how I can easily re-read the entire thread I'll check my notes and see if anything should be added. But basically, IIRC I think it comes down to: * separation of data and config can be blurry and some people have different definitions (but this is not necessarily a problem because... * ..there is a clear difference between 1) config as explicitly configured by user (by writing a text file or changing options in preferences settings from inside the app), and 2) config/data/state as automatically saved by the application. What I care mostly about, is that 1) and 2) go into separate files. IIRC we discussed in my fd.o thread at length whether 2) is config or data, but never reached a conclusion, so I would say don't aim to categorize it as config/data, just get app devs to store it in a separate file, either in $XDG_CONFIG_HOME or $XDG_DATA_HOME. We should also think a bit about whether this separation would suffice. We obviously want the full 1) in a VCS (I don't see a problem with 1), but 2) is probably something we don't to version, but we might want to back it up (in host-specific backups). but do we sometimes want to synchronize it to other hosts? that might warrant version controlling it again. But then we'd use something like dvcs-autosync where we care less about single commits, clean history etc, and just use the vcs as syncing method. But what happens when you only want to synchronize specific aspects? To take the claws-mail example, I might want to synchronize which email account had I last open info, but not what was the last window position (both are automatically saved by the app and would go in the same file). OTOH this last point sounds like YAGNI so maybe it would be just fine to not synchronize 2 at all. (just back it up per-host) Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: Announcing: dotfuse a project this list may find useful
On Wed, 6 Apr 2011 10:47:13 +0200 martin f krafft madd...@madduck.net wrote: also sprach Erich Heine sophac...@gmail.com [2011.03.13.1710 +0100]: I'm not sure how to get around this just yet, other than suggest not using this system for configuring programs that modify the rcs internally. Although now that I think about it, a read could trigger a hook, calling a per config file script which would know how to parse the config and possibly decompose it to the appropriate templates. You could write a diff of the file as written by the programme against the file as would be generated from the templates. Existence of a diff would trigger warnings to the user whenever possible to reintegrate the diff into the templates. I use this system for my SSH known_hosts management: new hosts are added to a temporary file, and SSH (wrapped by a zsh function) warns me of its existence, meaning that I have to integrate new hosts. well, in my experience, many of the apps who like to modify the rc/config files themselves, do it quite frequently. Thinks like last window position or last open folder [in the app] can get updated a few to tens of times per day. I wouldn't like to reintegrate all that myself manually. I've been thinking about this problem before (more in the context of manually version controlling everything). I've then pondered things such as selective ignores (a vcs could ignore changes to files on a granular level: i.e. not per-file, but by checking if changes match a pattern or not), if git would support this for example, irrelevant changes could then be made not to show up in `git status` or `git diff`, already greatly reducing any manual interventions. I think Eric's idea of parse the new config and adapt the templates automatically would be too complex to implement/maintain. The scripts would also be very application-specific. Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: [announce] Sharebox, a FUSE filesystem relying on git-annex
On Sun, 3 Apr 2011 11:18:05 -0400 Joey Hess j...@kitenet.net wrote: Dieter Plaetinck wrote: I think having support for this in git-annex would be very useful, even if it's not that efficient: if this can be dealt with in git-annex, individual higherlevel projects like sharebox and dvcs-autosync have less headaches. Not to mention sharebox/dvcs-autosync would need to do really inefficient things to deal with it anyway. (because they can't involve themselves into the actual git/dvcs tricks, they work on a higher level of abstraction), and it might also benefit some users who work with git-annex manually. How do you see this? How hard/cumbersome is it to implement this in git-annex? Why is it inefficient? It's not really clear to me after reading the smudge information on http://www.kernel.org/pub/software/scm/git/docs/gitattributes.html http://git-annex.branchable.com/todo/smudge/ if toobig then git_annex_add file else git_add file git_commit file unfortunately I don't think so: - with dvcs-autosync we often commit early, as in, the file could still be in the process of being written to, or it could be modified again after we added it. From what I understand, we would need to forbid our users from changing the file after it is added to git-annex, and worse: if git-annex does its move file, replace file with symlink trick, while the user is writing to it, this might break things. You're right. However, you would also not want to commit many partial versions of a large file as it was being written. Well, if it ever happens once, that's once too many. Since we're aiming for a dropbox-like near-instant-synchronisation system, the way of working is different then when using git for , say.. version controlling source code. So it _will_ happen that we commit versions of files as they are in the progress of being written. Even if the user decides to store something like a continuously being updated logfile in his dropbox-like system, I want to be able to support that. ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: [announce] Sharebox, a FUSE filesystem relying on git-annex
On Thu, 31 Mar 2011 18:56:54 +0200 Christophe-Marie Duquesne chm.duque...@gmail.com wrote: Hi, I am currently writing a FUSE file system based on git-annex for replicating binary files on several machines. I thought I could share it here in order to get some ideas and contributors. What are your goals? Seamless synchronization à la dropbox. Ability to use with big binary files such as mp3/movies. Entirely decentralized. Don't use unnecessary space Keep it simple: avoid special VCS commands and keep a filesystem interface as much as possible. you also need to do various git/git-annex commands, or am I missing something? Why? Because sparkleshare and dvcs-autosync are bad at versioning binary files I quite like dvcs-autosync, but it indeed lacks space-efficient storage of big files. I would like to try if we can use git-annex to support this in dvcs-autosync, although AFAIK git-annex is not transparent in the way regular git is transparent (i.e. it needs to explicitly copy files between locations), I assume this is the reason you need to go for a FUSE-based approach? or do you just prefer this over regular fs + inotify? Because Unison needs disk space for each couple of hosts it synchronizes and thus does not really scales for more than 2 hosts Because Coda is not completely decentralized and it bothers me you actually tried coda? it's something I'm interested in, on paper it looks like an awesome, maybe-even-perfect open source dropbox-clone but the reality is probably different, I never tried it so I wouldn't know. What do you have? A python implementation. It is about 600 sloc, and you'll find it on https://github.com/chmduquesne/sharebox Be careful, it is very alpha and it still does not have a proper conflict handler. Hey, but copying is slow! On my machine, copying files to a sharebox fs is about 10 times slower than copying it on a normal fs. All the time is spent in python's os.write(): I guess the only way to work around this problem is to rewrite the whole thing in C, but I am keeping this for later. hmm, writing files is i/o-bound, I doubt the language will have much effect here. check with top/vmstat if you get iowait, if so your storage medium is getting saturated and rewriting in C won't help. maybe a network/buffering/.. issue. I am interested in: - suggestions for the functional design (I have my ideas, but I'd love to be challenged). in your REAMDE you suggest to use a crontab for synchronisation; maybe you can reuse/be inspired by the xmpp system dvcs-autosync uses; it works quite well, it's quite robust and it's instant :) Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: Python script for automatic synchronization based on inotify
On Sun, 20 Mar 2011 08:31:39 +0100 René Mayrhofer r...@mayrhofer.eu.org wrote: Am Freitag, 18. März 2011, 22:59:07 schrieb Dieter Plaetinck: Note if you do this, server:test.git won't actually have a master branch yet. So when i touch ~/autosync/test, I get this: No refs in common and none specified; doing nothing. Perhaps you should specify a branch such as 'master'. fatal: The remote end hung up unexpectedly error: failed to push some refs to 'server:autosync.git' workaround is typing `git push origin master` once. after that it works fine. You're right. I've never actually tried it on a completely empty repository. We should either document this in README (the setup process) or make the script realize when there is no master branch and do a git push origin master in this case. I will need to dig into the git remote command a bit more to see what the best option is here (I don't like parsing the output if I can avoid it). best regards, Rene I also thought of some options. What about always explicitly pushing the current branch? unfortunately there is nothing like `git push origin --currentbranch`, so we would need to do ask git what is the current branch, and then `git push origin $branchname`, but that involves an extra process; we could try to cache the result but that gets ugly quickly (you need to know when to invalidate the cache, i.e. when the user creates a new branch). on the other hand, something like this would transparently allow the user to work with multiple branches; as far as i see there is currently no notion of the branch in autosync.py (i.e. master is always assumed), if we always do `git push master` we can fix the problem at hand (but then the user is restricted to the master branch, if he creates a new branch and we do a push of master, his new branch won't be pushed), if we do `git push $currentbranch` and also have a notion of the involved branch in the jabber messages, then the user could transparently use multiple branches. Either way I don't think we should query the remote for what do you have, frankly we shouldn't care. we only care about pushing (explicitly enough) the thing that we need to push. If you only want to support a master branch, hardcoding `git push origin master` is a good choice, but I don't like restricting ourselves like that. We can also say in the readme every time you create a new branch, push it manually once, from then on, we can just `git push origin` and it will do the right thing. it involves a tiny bit of manual work, but maybe this is the most sensible. Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
Re: Python script for automatic synchronization based on inotify
On Mon, 14 Mar 2011 17:02:01 +0100 René Mayrhofer r...@mayrhofer.eu.org wrote: Btw, the project is now online at http://gitorious.org/dvcs-autosync and should even include the full history (already close to a year, although with very irregular commit intervals ;-) ). ugh. why are many code changes committed as autocommit? why do you commit .pyc files? why do you use meaningless commit messages like move another file ? This looks very messy. Dieter ___ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
