First, sorry for the long post, maybe this will get me some help discovering what I can do with git , or help somebody else by describing my pain.
As a user of EDKII, we are currently trying to determine how we can transition to git for our projects. I am also still relatively new to git so I still have a lot of learning to do. Also, our primary development environment for x86 UEFI images is Windows. Here are my struggles with converting to git. Subversion allows us to pull any subfolder of a repository as an external. Therefore, we are able to pull each package folder of EDKII into the root directory of our working copy and keep the expected EDKII directory structure of all packages in the root (That only leaves special handling of the relatively few EDKII root folder files). The other huge advantage is that I can have a development branch which automatically tracks the HEAD of each external. Or, I can just as easily specify a revision of an external for a production branch. We thought about submodules and after much reading figured out it really wasn't a good idea, and not nearly as useful as svn externals. The main reason, is that you can't easily track the HEAD of submodules. While it looks like the commands have become somewhat easier to use to update to the HEAD revision, as far as I can tell, you are still required to commit to the parent project to recognize what to pull from the submodule. Submodules also require extra commands/parameters to clone the submodules. Svn updates externals by default, and you have to specify that you don’t want them updated. The fact that git only works on a complete repository makes it impossible to pull a complete EDKII tree, as some sort of external, such that the package folders go into the root directory as expected by the EDKII build infrastructure. This is why I assume the splitting up of packages into separate repos was suggested. In Linux, it would be easy to pull the EDKII tree into a subdirectory and make links from folders of EDKII into the root directory. Git even seems to be happy following these links and showing status of the complete subproject, etc. Then there's Google repo. Which, under Linux at least, seems to fit the bill relatively nicely. It allows you to specify a sub repository branch at its HEAD, or at a revision. It gives you some nice functionality to see what has changed in subprojects, but you have to use the git tools to commit to each individual subproject. The downside for Windows, Google repo heavily depends on the OS file management capabilities of Linux and therefore is a non-portable python tool that must be run in a Linux like command-line environment. Cygwin provides a relatively Linux capable command prompt environment in Windows and can run google repo, but with some less than desirable side effects. If you've played with google repo, it does a lot of things with symbolic links. Cygwin can be configured to create windows symbolic links when the target file/directory exists. Google repo creates symbolic links even when the target doesn't exist so, you can't force strict native symbolic links. Since the .git subdirectory in subprojects is full of symbolic links back to the .repo directory, you are then forced to do every bit of the git interaction within a Cygwin prompt. That's OK with me because I know enough about Windows command-line and Linux command-line to be pretty dangerous to myself. Our other developers are not as Linux knowledgeable, so now they will have to learn about Linux command-line and git command-line to do the same thing they had a nice pretty GUI in Windows and subversion. (I know about esrlabs/git-repo, but it doesn't look like it gets much love.) So even with those problems and Cygwin, Google repo seems to have less of a downside than submodules. So we will probably use that. Now I'm down to the problem of pulling the EDKII tree into my project. We can't get Cygwin/repo to consistently create Windows native symbolic links. It looks like we are going to pull EDKII as a subdirectory and then in a batch file and from a "cmd.exe" prompt create Windows symbolic links to the proper subdirectories. We use a batch file under "cmd.exe" to run our platform build anyway, so no big deal. This will allow us to at least build in a native windows command environment. One more interesting thing about Cygwin/Linux is that it can do particularly nasty things to file access permissions that windows users don't usually comprehend. It can make an executable file "*.exe" not execute because it got stored in the git repo without the executable bit set. For example when Linux was used to convert from an svn repo to a git repo. One last thought, I assume there would be some strategy where we could start with the EDKII tree as our base project and then put our platform code in. Then use merging to pick up EDKII modifications, but as Jordan said, that has its own concerns. Thanks to anyone that took the time to read this whole thing. Any thoughts would be greatly appreciated. GARRETT KIRKENDALL SMTS Firmware Engineer | CTE 7171 Southwest Parkway, Austin, TX 78735 USA facebook | amd.com -----Original Message----- From: Jordan Justen [mailto:jordan.l.jus...@intel.com] Sent: Wednesday, June 03, 2015 3:01 PM To: Andrew Fish; edk2-devel@lists.sourceforge.net Subject: Re: [edk2] Proposal of Git Repo Layout for EDKII project On 2015-06-03 12:01:58, Andrew Fish wrote: > On Jun 3, 2015, at 7:50 AM, Brian J. Johnson <bjohn...@sgi.com> wrote: > > I fully agree with others' reluctance to use git submodules, and the > > reasons they have expressed: git submodules are a major pain for > > developers, and the concerns Liming listed above can be addressed in > > other ways. > > > > When my internal team first transitioned to git, we set up a complex > > submodule-like system to (theoretically) allow easily updating > > common code among different projects. That only lasted a month or > > two: having to manage multiple repositories for day-to-day work, > > and the lack of a single commit history spanning the entire tree doomed > > that scheme. > > > > I collapsed everything together into a single repo using some git > > filter-branch magic, and we've been happy ever since. > > > > Please, no submodules…. > > I agree that submodules add complexity, and make things harder. > Maybe for hardware project they are OK, but the core of edk2 should be > one project. I also would prefer if EDK II upstream could be a single repo, but I understand why there is also a desire to consider submodules. First and foremost, inside Intel, the svn:externals feature is used extensively to compose platform trees together. And, submodules map very closely to that usage model. But, even if you try to consider alternatives to submodules for composing platform trees, things get complicated. One idea, is to fork the EDK II master tree, and add submodules for your platform specific modules. To me this ends up with the worst of both worlds. 1. All git commands are difficult to use tree-wide, as expressed in this thread, and 2. You don't have the power to select only the EDK II modules that you need for your platform. Another idea is to fork the EDK II master tree and add your platform specific modules directly into the fork. In this case, you can still use all the git commands, but you once again can't select only the EDK II modules that you need for your platform. Other difficulties arise, such as, what if you have a chipset package that you want to share for multiple platforms? Unless all the platforms for that chipset live in the same branch, how do you easily share common code for those chipset packages? (Maybe a separate 'upstream' for the chipset code that the platforms merge in as needed?) I think Android might share some of the same concerns, and their solution was to invent a submodules-like alternative called 'repo' that layers on git. So, can we add these concerns into the discussion, and maybe document an alternative way to address these concerns if submodules aren't used? Thanks, -Jordan ------------------------------------------------------------------------------ _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel ------------------------------------------------------------------------------ _______________________________________________ edk2-devel mailing list edk2-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/edk2-devel
