Re: [msysGit] Consecutive git gc fails on Windows network share
Hi kusma, On Tue, 21 Jan 2014, Erik Faye-Lund wrote: On Tue, Jan 21, 2014 at 12:25 AM, Johannes Schindelin johannes.schinde...@gmx.de wrote: On Mon, 20 Jan 2014, Torsten Bögershausen wrote: b) add +++ at the places where you added the stat() and chmod(), c) and to send the question is this a good implementation ? to upstream git. I think your implementation makes sense. As I said in my other reply, I think that the problem would be addressed more generally in compat/mingw.c. It is to be doubted highly that upstream wants to handle cases such as rename() cannot overwrite read-only files on Windows everywhere they call rename() because the platforms upstream cares about do not have that problem. I'm not so sure. A quick test shows me that this is not the case for NTFS. Since this is over a network-share, the problem is probably server-side, and can affect other systems as well. So a work-around might be appropriate for all systems, no? I do not think that the problem occurs if you run the same commands on Linux, on a mounted Samba share. So I guess that upstream Git can enjoy their luxury of not having to care. In any case, if we would need this also for Linux, doing it for only one user of rename() would probably not be good enough, either... so something similar to mingw_rename() would be needed (interfering with mingw_rename itself, of course). Ciao, Dscho
Re: [msysGit] Consecutive git gc fails on Windows network share
On Tue, Jan 21, 2014 at 5:57 PM, Johannes Schindelin
johannes.schinde...@gmx.de wrote:
Hi kusma,
On Tue, 21 Jan 2014, Erik Faye-Lund wrote:
On Tue, Jan 21, 2014 at 12:25 AM, Johannes Schindelin
johannes.schinde...@gmx.de wrote:
On Mon, 20 Jan 2014, Torsten Bögershausen wrote:
b) add +++ at the places where you added the stat() and chmod(),
c) and to send the question is this a good implementation ? to upstream
git.
I think your implementation makes sense.
As I said in my other reply, I think that the problem would be
addressed more generally in compat/mingw.c. It is to be doubted highly
that upstream wants to handle cases such as rename() cannot overwrite
read-only files on Windows everywhere they call rename() because the
platforms upstream cares about do not have that problem.
I'm not so sure. A quick test shows me that this is not the case for
NTFS. Since this is over a network-share, the problem is probably
server-side, and can affect other systems as well.
So a work-around might be appropriate for all systems, no?
I do not think that the problem occurs if you run the same commands on
Linux, on a mounted Samba share. So I guess that upstream Git can enjoy
their luxury of not having to care.
In any case, if we would need this also for Linux, doing it for only one
user of rename() would probably not be good enough, either... so something
similar to mingw_rename() would be needed (interfering with mingw_rename
itself, of course).
Indeed. I was thinking of something along the lines of adding a
xrename in wrapper.c.
But you're probably right that this doesn't happen under Samba; surely
Samba would have added a work-around for such a filesystem by now. So
yeah, you've convinced me. mingw_rename is probably the place to do
that.
The work-around would probably look something like this:
diff --git a/compat/mingw.c b/compat/mingw.c
index a37bbf3..580b820 100644
--- a/compat/mingw.c
+++ b/compat/mingw.c
@@ -1697,6 +1697,14 @@ int mingw_rename(const char *pold, const char *pnew)
*/
if (!_wrename(wpold, wpnew))
return 0;
+
+if (errno == EPERM) {
+/* read-only files cannot be moved over on network shares */
+_wchmod(wpnew, 0666);
+if (!_wrename(wpold, wpnew))
+return 0;
+}
+
if (errno != EEXIST)
return -1;
repeat:
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [msysGit] Consecutive git gc fails on Windows network share
(No top-posting, please see my comments below) On 2014-01-20 15.02, Jochen wrote: On 01/16/2014 10:55 AM, Jochen Haag wrote: Hello, we want to report the following issue, because it seems to be not an intended behaviour. Maybe someone can have a look at it at some point. It is not urgent for us. After upgrading from Git version 1.8.1.msysgit.1 to 1.8.5.2.msysgit.0 we observed that a consecutive git gc fails, in case the Git repo is located on a Windows network share. Operating system used is Windows 7 64 bit SP1. In case git gc fails temporary packs and index remain in .git/objects/pack/. Consecutive means, that git gc is called on an already packed repository (e.g. git gc is called more than once). This is the output given in case of error: U:\GitEnvgit gc Counting objects: 139, done. Delta compression using up to 8 threads. Compressing objects: 100% (71/71), done. Writing objects: 100% (139/139), done. Total 139 (delta 65), reused 139 (delta 65) fatal: renaming '.git/objects/pack/.tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack' failed: Permission denied error: failed to run repack And here the output with GIT_TRACE = 1: U:\GitEnvgit gc trace: built-in: git 'gc' trace: run_command: 'pack-refs' '--all' '--prune' trace: built-in: git 'pack-refs' '--all' '--prune' trace: run_command: 'reflog' 'expire' '--all' trace: built-in: git 'reflog' 'expire' '--all' trace: run_command: 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: built-in: git 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: run_command: 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' '.git/objects/pack/.tmp-7612-pack' trace: built-in: git 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' '.git/objects/pack/.tmp-7612-pack' Counting objects: 139, done. Delta compression using up to 8 threads. Compressing objects: 100% (71/71), done. Writing objects: 100% (139/139), done. Total 139 (delta 65), reused 139 (delta 65) fatal: renaming '.git/objects/pack/.tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack' failed: Permission denied error: failed to run repack After running git gc twice, this is what is left in .git/objects/pack/: U:\GitEnv\.git\objects\packls -al total 53676 drwxr-xr-x1 hgj2fe Administ0 Jan 16 10:43 . drwxr-xr-x1 hgj2fe Administ0 Jan 14 12:52 .. -r--r--r--1 hgj2fe Administ 4964 Jan 16 10:43 .tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx -r--r--r--1 hgj2fe Administ 18315618 Jan 16 10:43 .tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack -r--r--r--1 hgj2fe Administ 4964 Jan 16 10:40 .tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx -r--r--r--1 hgj2fe Administ 18315618 Jan 16 10:40 .tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack -r--r--r--1 hgj2fe Administ 4964 Jan 14 12:52 pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx -r--r--r--1 hgj2fe Administ 18315618 Jan 14 12:52 pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack In case we remove the read-only flag of the pack and index manually before running git gc again, no problem occurs: U:\GitEnv\.git\objects\packgit gc trace: built-in: git 'gc' trace: run_command: 'pack-refs' '--all' '--prune' trace: built-in: git 'pack-refs' '--all' '--prune' trace: run_command: 'reflog' 'expire' '--all' trace: built-in: git 'reflog' 'expire' '--all' trace: run_command: 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: built-in: git 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: run_command: 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' 'U:/GitEnv/.git/objects/pack/.tmp-3736-pack' trace: built-in: git 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' 'U:/GitEnv/.git/objects/pack/.tmp-3736-pack' Counting objects: 139, done. Delta compression using up to 8 threads. Compressing objects: 100% (71/71), done. Writing objects: 100% (139/139), done. Total 139 (delta 65), reused 139 (delta 65) trace: run_command: 'prune-packed' trace: built-in: git 'prune-packed' trace: run_command: 'update-server-info' trace: built-in: git 'update-server-info' trace: run_command: 'prune' '--expire' '2.weeks.ago' trace: built-in: git 'prune' '--expire' '2.weeks.ago' trace: run_command: 'rerere' 'gc' trace: built-in: git 'rerere' 'gc' The problem might be related to this commit: https://github.com/msysgit/git/commit/a1bbc6c0176f1fa2d4aa571cc0183a1f0ff9b285 Best regards, Jochen
Re: [msysGit] Consecutive git gc fails on Windows network share
On 20.01.2014 18:16, Torsten Bögershausen wrote: (No top-posting, please see my comments below) On 2014-01-20 15.02, Jochen wrote: On 01/16/2014 10:55 AM, Jochen Haag wrote: Hello, we want to report the following issue, because it seems to be not an intended behaviour. Maybe someone can have a look at it at some point. It is not urgent for us. After upgrading from Git version 1.8.1.msysgit.1 to 1.8.5.2.msysgit.0 we observed that a consecutive git gc fails, in case the Git repo is located on a Windows network share. Operating system used is Windows 7 64 bit SP1. In case git gc fails temporary packs and index remain in .git/objects/pack/. Consecutive means, that git gc is called on an already packed repository (e.g. git gc is called more than once). This is the output given in case of error: U:\GitEnvgit gc Counting objects: 139, done. Delta compression using up to 8 threads. Compressing objects: 100% (71/71), done. Writing objects: 100% (139/139), done. Total 139 (delta 65), reused 139 (delta 65) fatal: renaming '.git/objects/pack/.tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack' failed: Permission denied error: failed to run repack And here the output with GIT_TRACE = 1: U:\GitEnvgit gc trace: built-in: git 'gc' trace: run_command: 'pack-refs' '--all' '--prune' trace: built-in: git 'pack-refs' '--all' '--prune' trace: run_command: 'reflog' 'expire' '--all' trace: built-in: git 'reflog' 'expire' '--all' trace: run_command: 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: built-in: git 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: run_command: 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' '.git/objects/pack/.tmp-7612-pack' trace: built-in: git 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' '.git/objects/pack/.tmp-7612-pack' Counting objects: 139, done. Delta compression using up to 8 threads. Compressing objects: 100% (71/71), done. Writing objects: 100% (139/139), done. Total 139 (delta 65), reused 139 (delta 65) fatal: renaming '.git/objects/pack/.tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack' failed: Permission denied error: failed to run repack After running git gc twice, this is what is left in .git/objects/pack/: U:\GitEnv\.git\objects\packls -al total 53676 drwxr-xr-x1 hgj2fe Administ0 Jan 16 10:43 . drwxr-xr-x1 hgj2fe Administ0 Jan 14 12:52 .. -r--r--r--1 hgj2fe Administ 4964 Jan 16 10:43 .tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx -r--r--r--1 hgj2fe Administ 18315618 Jan 16 10:43 .tmp-7612-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack -r--r--r--1 hgj2fe Administ 4964 Jan 16 10:40 .tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx -r--r--r--1 hgj2fe Administ 18315618 Jan 16 10:40 .tmp-7920-pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack -r--r--r--1 hgj2fe Administ 4964 Jan 14 12:52 pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.idx -r--r--r--1 hgj2fe Administ 18315618 Jan 14 12:52 pack-ad9d069e7c27ce19cc5b6cde82b4a38bedbf5884.pack In case we remove the read-only flag of the pack and index manually before running git gc again, no problem occurs: U:\GitEnv\.git\objects\packgit gc trace: built-in: git 'gc' trace: run_command: 'pack-refs' '--all' '--prune' trace: built-in: git 'pack-refs' '--all' '--prune' trace: run_command: 'reflog' 'expire' '--all' trace: built-in: git 'reflog' 'expire' '--all' trace: run_command: 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: built-in: git 'repack' '-d' '-l' '-A' '--unpack-unreachable=2.weeks.ago' trace: run_command: 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' 'U:/GitEnv/.git/objects/pack/.tmp-3736-pack' trace: built-in: git 'pack-objects' '--keep-true-parents' '--honor-pack-keep' '--non-empty' '--all' '--reflog' '--unpack-unreachable=2.weeks.ago' '--local' '--delta-base-offset' 'U:/GitEnv/.git/objects/pack/.tmp-3736-pack' Counting objects: 139, done. Delta compression using up to 8 threads. Compressing objects: 100% (71/71), done. Writing objects: 100% (139/139), done. Total 139 (delta 65), reused 139 (delta 65) trace: run_command: 'prune-packed' trace: built-in: git 'prune-packed' trace: run_command: 'update-server-info' trace: built-in: git 'update-server-info' trace: run_command: 'prune' '--expire' '2.weeks.ago' trace: built-in: git 'prune' '--expire' '2.weeks.ago' trace: run_command: 'rerere' 'gc' trace: built-in: git 'rerere' 'gc' The problem might be related to this commit:
Re: [msysGit] Consecutive git gc fails on Windows network share
Hi Torsten, On Mon, 20 Jan 2014, Torsten Bögershausen wrote: (No top-posting, please see my comments below) already very good! For extra brownie points, just edit the quoted part to reflect the abridged version needed to understand the context. On 2014-01-20 15.02, Jochen wrote: On 01/16/2014 10:55 AM, Jochen Haag wrote: The rename command replaces a mv -f command of the original shell script. Obviously the -f option can also override a read-only file but rename can not on a network share. I allowed me to a) reconstruct the original mail, Please note that together with an exceedingly flakey internet connection, not only having to scroll through the mail (most of which was actually not relevant to your reply) and having to delete most parts again ate up my complete Git time budget for today. Just something you might want to keep in mind. b) add +++ at the places where you added the stat() and chmod(), c) and to send the question is this a good implementation ? to upstream git. I think your implementation makes sense. As I said in my other reply, I think that the problem would be addressed more generally in compat/mingw.c. It is to be doubted highly that upstream wants to handle cases such as rename() cannot overwrite read-only files on Windows everywhere they call rename() because the platforms upstream cares about do not have that problem. So we probably need just the same _wchmod we have in mingw_unlink also in mingw_rename. Ciao, Johannes
Re: [msysGit] Consecutive git gc fails on Windows network share
On Tue, Jan 21, 2014 at 12:25 AM, Johannes Schindelin johannes.schinde...@gmx.de wrote: On Mon, 20 Jan 2014, Torsten Bögershausen wrote: b) add +++ at the places where you added the stat() and chmod(), c) and to send the question is this a good implementation ? to upstream git. I think your implementation makes sense. As I said in my other reply, I think that the problem would be addressed more generally in compat/mingw.c. It is to be doubted highly that upstream wants to handle cases such as rename() cannot overwrite read-only files on Windows everywhere they call rename() because the platforms upstream cares about do not have that problem. I'm not so sure. A quick test shows me that this is not the case for NTFS. Since this is over a network-share, the problem is probably server-side, and can affect other systems as well. So a work-around might be appropriate for all systems, no? -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
