Re: [BUG] auto-repack exits prematurely, locking other processing out
Adam Borowski kilob...@angband.pl writes: Hi guys! It looks like the periodic auto-repack backgrounds itself when it shouldn't do so. This causes the command it has triggered as a part of to fail: Yikes. In the meantime, I think you can turn gc.autodetach off as a workaround, e.g. $ git config --global --add gc.autodetach off Duy, 9f673f94 (gc: config option for running --auto in background, 2014-02-08) turns to be not such a hot idea. Sure, if we kick it off background after doing something heavy, immediately before giving control back to the end-user, and expect that the user will stay thinking without making new changes (i.e. read-only stuff like git show would be OK), then daemonize might be a great thing, but we forgot, while doing that commit, that long-running operations trigger the auto gc in the middle *and* they want it finish before they continue, as the purpose of gc is to help the performance during their further operation. == [~/linux](master)$ git pull --rebase remote: Counting objects: 455, done. remote: Compressing objects: 100% (64/64), done. remote: Total 267 (delta 208), reused 262 (delta 203) Receiving objects: 100% (267/267), 44.43 KiB | 0 bytes/s, done. Resolving deltas: 100% (208/208), completed with 80 local objects. From git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux 4b660a7..f02f79d master - linus/master Auto packing the repository in background for optimum performance. See git help gc for manual housekeeping. First, rewinding head to replay your work on top of it... Applying: perf: tools: fix missing casts for printf arguments. Applying: vt: emulate 8- and 24-bit colour codes. fatal: Unable to create '/home/kilobyte/linux/.git/refs/heads/master.lock': File exists. If no other git process is currently running, this probably means a git process crashed in this repository earlier. Make sure no other git process is running and remove the file manually to continue. Could not move back to refs/heads/master [~/linux]((no branch, rebasing (null)))$ == -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [BUG] auto-repack exits prematurely, locking other processing out
On Fri, May 23, 2014 at 02:40:41PM -0700, Junio C Hamano wrote: Adam Borowski kilob...@angband.pl writes: It looks like the periodic auto-repack backgrounds itself when it shouldn't do so. This causes the command it has triggered as a part of to fail: Duy, 9f673f94 (gc: config option for running --auto in background, 2014-02-08) turns to be not such a hot idea. Sure, if we kick it off background after doing something heavy, immediately before giving control back to the end-user, and expect that the user will stay thinking without making new changes (i.e. read-only stuff like git show would be OK), then daemonize might be a great thing, but we forgot, while doing that commit, that long-running operations trigger the auto gc in the middle *and* they want it finish before they continue, as the purpose of gc is to help the performance during their further operation. Just add a lock that's triggered by daemonize, and have things block on this lock. This handles all cases: * --auto in the middle of a command: the block will kick in immediately, effectively reverting to non-daemonized version * --auto at the end, the user does nothing: gc will finish its work in the background, just as you wanted * --auto at the end, the user starts a new write two seconds later: gc works in the foreground with those 2 seconds headstart The only loss is the lack of a progress indicator, and even that can be done. -- Gnome 3, Windows 8, Slashdot Beta, now Firefox Ribbon^WAustralis. WTF is going on with replacing usable interfaces with tabletized ones? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [BUG] auto-repack exits prematurely, locking other processing out
Adam Borowski kilob...@angband.pl writes: On Fri, May 23, 2014 at 02:40:41PM -0700, Junio C Hamano wrote: Adam Borowski kilob...@angband.pl writes: It looks like the periodic auto-repack backgrounds itself when it shouldn't do so. This causes the command it has triggered as a part of to fail: Duy, 9f673f94 (gc: config option for running --auto in background, 2014-02-08) turns to be not such a hot idea. Sure, if we kick it off background after doing something heavy, immediately before giving control back to the end-user, and expect that the user will stay thinking without making new changes (i.e. read-only stuff like git show would be OK), then daemonize might be a great thing, but we forgot, while doing that commit, that long-running operations trigger the auto gc in the middle *and* they want it finish before they continue, as the purpose of gc is to help the performance during their further operation. Just add a lock that's triggered by daemonize, and have things block on this lock. Hmph, it defeats the whole point of running it in the background, doesn't it? How would blocking on the lock be different from launching gc --auto and waiting for it to come back? And it would also require addition of the big-repository-lock and code to take the lock sprinkled all over the place. I am not sure if we want to go there... -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [BUG] auto-repack exits prematurely, locking other processing out
On Sat, May 24, 2014 at 4:40 AM, Junio C Hamano gits...@pobox.com wrote: Duy, 9f673f94 (gc: config option for running --auto in background, 2014-02-08) turns to be not such a hot idea. Sure, if we kick it off background after doing something heavy, immediately before giving control back to the end-user, and expect that the user will stay thinking without making new changes (i.e. read-only stuff like git show would be OK), then daemonize might be a great thing, but we forgot, while doing that commit, that long-running operations trigger the auto gc in the middle *and* they want it finish before they continue, as the purpose of gc is to help the performance during their further operation. If by long-running operations you mean in a single process, it's my first thought too but it looks like autogc is always called when the process is all done and about to exit. The git pull case is different because there's rebase after fetch. I see no easy way to detect this kind of middle of operation. So we have two options: scripts should disable autogc before doing things, a env variable would be more convenient than temporarily updating gc.auto. Or we move pack-refs and reflog expire up, before turning gc into a background task. Any locking will be serialized this way. We could even go further to keep all but repack in the background because it's repack that takes the longest time (maybe prune coming close to second). -- Duy -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
