Re: Why is git submodule slow under windows
Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1): Hello, We split a monolithic repository into ~50 submodules. The stored data has the same size. In the 1:1 comparison to the monolithic repository, the submodule handling is very slow. Under Linux everything remains fast, but windows is slow. So, why is git getting slow when it has to deal with a lot of submodules? I read something about the lack of the underlying cygwin to handle NTFS in a efficient way. Is this the root cause, or are there other causes also? Hi, I assume you are using the latetst git from https://msysgit.github.io on windows. I would guess that submodules on windows are slow because git-submodules.sh is a shell script, and bash on windows is not really that fast. There has been some (albeit older) discussion on the msysgit mailinglist, see [1]. You can play around with core.fscache [2] maybe that helps. Thomas [1]: https://groups.google.com/d/msg/msysgit/AuPA4EbwchU/N42tsb6GousJ [2]: https://github.com/msysgit/msysgit/releases/tag/Git-1.8.5.2-preview20131230 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is git submodule slow under windows
On Fri, Jul 18, 2014 at 02:08:36PM +0200, Thomas Braun wrote: Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1): Hello, We split a monolithic repository into ~50 submodules. The stored data has the same size. In the 1:1 comparison to the monolithic repository, the submodule handling is very slow. Under Linux everything remains fast, but windows is slow. So, why is git getting slow when it has to deal with a lot of submodules? I read something about the lack of the underlying cygwin to handle NTFS in a efficient way. Is this the root cause, or are there other causes also? Hi, I assume you are using the latetst git from https://msysgit.github.io on windows. I would guess that submodules on windows are slow because git-submodules.sh is a shell script, and bash on windows is not really that fast. My guess is that because the shell script uses fork() heavily and fork() is an expensive operation on Windows, that alone causes the slowddown. I did a quick test a while back when I rewrote part of git-submodule.sh in lua and runned it on my repo with ~45 submodules. The speedup was significant and should be even bigger on windows. -- Med vänlig hälsning Fredrik Gustafsson tel: 0733-608274 e-post: iv...@iveqy.com -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Why is git submodule slow under windows
Am 18.07.2014 15:02, schrieb Fredrik Gustafsson: On Fri, Jul 18, 2014 at 02:08:36PM +0200, Thomas Braun wrote: Am 18.07.2014 12:14, schrieb Armbruster Joachim (BEG/EMS1): Hello, We split a monolithic repository into ~50 submodules. The stored data has the same size. In the 1:1 comparison to the monolithic repository, the submodule handling is very slow. Under Linux everything remains fast, but windows is slow. So, why is git getting slow when it has to deal with a lot of submodules? I read something about the lack of the underlying cygwin to handle NTFS in a efficient way. Is this the root cause, or are there other causes also? Hi, I assume you are using the latetst git from https://msysgit.github.io on windows. I would guess that submodules on windows are slow because git-submodules.sh is a shell script, and bash on windows is not really that fast. My guess is that because the shell script uses fork() heavily and fork() is an expensive operation on Windows, that alone causes the slowddown. I did a quick test a while back when I rewrote part of git-submodule.sh in lua and runned it on my repo with ~45 submodules. The speedup was significant and should be even bigger on windows. Without having looked at your Lua rewrite I suspect my recursive submodule checkout series could speed that up even more, as it only needs to fork a git status followed by a git checkout or git read-tree for each submodule. Now that my submodule test harness and Heiko's submodule config lookup API did hit next, this is the thing I'm currently working on. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
