I completely understand Oliver's view on maintaining every individual repo even if they are submodules of a kicad-footprint repo.
Even from an a user perspective (which is clearly easier than the maintainer one), I like knowing what progress is being made, so I'd like to subscribe for updates of a single repo on github, to follow commits, pull requests, etc. There's no way I will subscribe to every repo and what's most difficult, checking to make sure that I subscribe for new repos as they are created too. I have basically no say here, but I'd go for a single repo and no submodules. Thanks, Diego On Thu, Sep 28, 2017 at 6:53 PM, Bernhard Stegmaier <[email protected] > wrote: > Just my 2 cents… I have seen some projects using submodules (open source > and in my company). > I have seen lots of problems and complaints (… doesn’t compile …) just > because normal users or even normal developers (being no git experts) > didn’t sync all submodules correctly. > Thinking of how many problems are already seen now by users which are not > used to version control systems (because they just don’t understand what’s > happening under the hood) I wouldn’t want to force them to use git > submodules… at least not as long as this is not completely hidden by some > users friendly and dead simple KiCad GUI. > > > Regards, > Bernhard > > On 28. Sep 2017, at 13:29, Oliver Walters <[email protected]> > wrote: > > Simon, David, > > Thanks for the synopsis of your idea David, there are certainly some > advantages to using the submodule approach. > > Is it github compatible? > > > Simon, yes git-submodule is compatible with github. I have a test > repository here - https://github.com/kicad/kicad-footprints - this > contains a submodule for each of the current footprint directories. > > Does it simplify the maintenance of the kicad library by the librarians? > > > Compared to having all the .pretty folders as subdirectories of a single > repo, no. Reorganising libraries or adding new ones is tedious, and for > each of the 100+ .pretty repos, we (the librarians) have to manage: > > * Pull requests and isues > * CI tools (e.g. Travis KLC scripts) > * Updating each library when new requirements are enacted (e.g. KLC) > * Ensure that changes in one repo do not conflict with a different repo > * Handle reclassification or moving of footprints between repos > > It is also quite possible for submodules to become out of sync. The link > that Simon posted has some a good example of how this can occur (detached > head). I have some projects at work that contain about 5 submodules and it > rapidly becomes difficult to deal with even for a competent user of Git. > > It becomes harder for users, too. KiCad users cannot, in the general case, > be classified as competent users of git. Currently, users who wish to > contribute to the footprint libraries have a hard time. For each library > they want to contribute to, they have to fork and manage another > repository. Personally I have done this for 50 separate repositories. > > Further, once changes are made to each submodule, the master repository > then needs to be updated otherwise it's not up to date with the latest > changes. Perhaps we can use git hooks here, but this is just further > complication! > > Let's also consider consistency. The footprint libraries are (currently) > split into separate repos. However the symbol libraries and 3d model > libraries are not. Should we split them too? Now we have 300+ repositories > to manage. > > Let's not make this an academic problem and say "but it's possible to > manage with this complex set of tools"! From the perspective of a > contributor, and as the person who has to manage the libraries, the simpler > the better. > > Oliver > > > > On Thu, Sep 28, 2017 at 7:25 PM, Simon Küppers <[email protected]> > wrote: > >> Thanks for your detailed description. I think it is a nice way to go. >> However two remarks: >> Does it simplify the maintenance of the kicad library by the librarians? >> Is it github compatible? If not, we would need to find another platform to >> host the libraries. >> >> If I look for git submodule, there is a lot of different opinions on why >> you should or should not use it. (e.g. https://www.google.de/amp/s/co >> dingkilledthecat.wordpress.com/2012/04/28/why-your-company- >> shouldnt-use-git-submodules/amp/). Does it apply in our case? What about >> git subtree? >> >> Best regards >> Simon >> >> Am 28. September 2017 02:37:09 MESZ schrieb David Godfrey < >> [email protected]>: >> >>> Hi All >>> >>> I have to agree with some of the other posters, why not use git as it >>> was intended? >>> >>> Specifically I think the following makes sense..... >>> >>> - Keep the multiple repositories, this helps when a company only wants >>> to make certain sets of libs available to it's staff >>> >>> - Have a master repository that includes all other repositories as >>> submodules. >>> >>> - Have branches that are matched to kicad versions. This allows >>> footprint changes in a version safe way. >>> >>> - Use standard "git clone" (initial download) and "git pull" (update) on >>> the main repo which provides the entire set of available libs without >>> actually downloading the content for all libs. >>> >>> - When a specific lib is needed, do similar to what we do now, and use >>> "git submodule update --init --recursive $submoduleName" to just pull >>> that specific submodule >>> >>> - Allow the "main" library repo URI to be altered. This enables a >>> companies fork of the repo to be used. >>> >>> - Allow the "main" library repo URI to be ANY valid git URI. That means >>> a repo on a local fileserver rather than a http server can be used. >>> Along with various security options etc. >>> >>> - Add a fairly simple scripted tool that's run "on release" to retrieve >>> a README.md <http://readme.md/> (an possibly a descriptor/index file) from >>> each actual >>> library repo and update those within the "main" repo. This removes the >>> need to have the current case where manual edits are required to keep >>> the "main" repo in sync with what's available in the individual lib's. >>> >>> To add a new lib, it's then as simple as (from the main repo) doing >>> something like.... >>> - "git submodule add $URI" >>> - "./scripts/update-library-indexes.sh" >>> - "git commit -a -m $'added new module "modulename"\nupdated all module >>> descriptions and index'" >>> - "git push" >>> >>> >>> All of the above allows the Main repo to be forked by a company (or >>> individual) and have their own custom repo's added very easily. >>> It even allows libraries to be easily excluded or replaced, all using >>> extremely well developed management tools. >>> >>> On a side note, git submodules are stored in the main repository >>> basically as URI that includes a commit reference. >>> That means it's easy to specify a specific library version to include in >>> a specific branch/tag of the main repo. >>> >>> >>> As for the person that said "git isn't available for windows", sorry, >>> that's bunkum. >>> There are many many development environments out there that ONLY run on >>> windows that have git integration. >>> A quick google search for "git for windows" will show what you need to >>> know there. >>> >>> Finally, wrapping git to duplicate the effective API that's currently in >>> use should be relatively trivial, resulting in virtually no code changes >>> required to KICAD it's self. >>> Assuming the existing functionality is cleanly wrapped by an API, the >>> only real change to KICAD would be the swapout of the module, and >>> addition of an easy way to change the URI >>> >>> >>> I know I haven't covered everything in this email, but it should be a >>> good outline for further discussion >>> >>> >>> On 22/09/17 09:21, Oliver Walters wrote: >>> >>>> Hi all, >>>> >>>> Ok, now that the website integration with the libraries is (pretty >>>> much) done, and the licensing issue seems to be sorted, there is one >>>> final puzzle piece to solve before I'm happy with the state of the >>>> libraries for a v5 release. >>>> >>>> *Goal: *Merge all footprint library repositories into a single repo to >>>> solve the ongoing dramas of maintaining 100+ repos. >>>> >>>> *Problem*: The *only* thing standing in the way of just doing this is >>>> that some users like the GitHub plugin and previous instruction is >>>> that this functionality cannot be removed. >>>> >>>> The GitHub plugin functions by downloading a .zip file of a .pretty >>>> repo. If we merge all footprint libs into a single repo with multiple >>>> subdirectories, this will not work anymore (as GitHub dosen't allow >>>> you to download a .zip of a single subdirectory). >>>> >>>> Merging the repos is the *right thing to do*. But how to proceed? >>>> >>>> *Options:* >>>> * >>>> * >>>> /a) Drop github plugin feature, replace with library-download tool/ >>>> >>>> I don't think it is a good idea to live-load library data from GitHub >>>> (a lot of other users agree too). It's slow, and a waste of bandwidth >>>> to re-download the libs all the time. >>>> >>>> We drop support for loading libraries direct from GitHub. However, we >>>> add a tool for downloading libraries from GitHub and storing to disk. >>>> Users can update as they like. This can be integrated in KiCad and new >>>> users can run this tool when they first install KICad. This means that >>>> no libs need to distributed with the installer and users can update to >>>> latest libs whenever they want. >>>> >>>> /b) Improve github plugin to allow subdirectory traversal/ >>>> >>>> This is difficult and will only result in the plugin being slower. >>>> There are two ways I can see to do this: >>>> >>>> i. Use Git API - tools exist that use this functionality - >>>> https://github.com/KinoLien/gitzip >>>> ii. Use subversion - GitHub actually provides subversion API - >>>> https://www.seanw.org/blog/download-git-repo-subdirectory/ >>>> >>>> >>>> *Subversion* >>>> >>>> In either case, I think that using the subversion tool to partially >>>> download the libraries would be a good approach (I assume quicker than >>>> using wget and the GitHub API). >>>> >>>> 1. Does anyone have any experience using the C API for SVN? >>>> 2. The Python bindings are pretty good - >>>> https://pypi.python.org/pypi/svn - and much easier to use. However, >>>> can we make the library download tools dependent on enabling the >>>> python plugin? >>>> 3. Is there a way to checkout a subversion remote to memory (to >>>> replicate the functionality of the current GitHub plugin)? If not, I'm >>>> not sure how to approach option b) above. >>>> >>>> >>>> Feedback appreciated. I think that it is very important especially for >>>> new users that this is improved. The GitHub plugin is constantly >>>> causing headaches! >>>> >>>> Cheers, >>>> Oliver >>>> >>>> >>>> ------------------------------ >>>> >>>> Mailing list: https://launchpad.net/~kicad-developers >>>> Post to : [email protected] >>>> Unsubscribe : https://launchpad.net/~kicad-developers >>>> More help : https://help.launchpad.net/ListHelp >>>> >>> Regards >>> David Godfrey >>> SB Tech Services >>> mb: +61 437 286 200 <+61%20437%20286%20200> <tel:+61437286200 >>> <+61%20437%20286%20200>> >>> >>> ------------------------------ >>> >>> Mailing list: https://launchpad.net/~kicad-developers >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~kicad-developers >>> More help : https://help.launchpad.net/ListHelp >>> >>> >> _______________________________________________ >> Mailing list: https://launchpad.net/~kicad-developers >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~kicad-developers >> More help : https://help.launchpad.net/ListHelp >> >> > _______________________________________________ > Mailing list: https://launchpad.net/~kicad-developers > Post to : [email protected] > Unsubscribe : https://launchpad.net/~kicad-developers > More help : https://help.launchpad.net/ListHelp > > > > _______________________________________________ > Mailing list: https://launchpad.net/~kicad-developers > Post to : [email protected] > Unsubscribe : https://launchpad.net/~kicad-developers > More help : https://help.launchpad.net/ListHelp > >
_______________________________________________ Mailing list: https://launchpad.net/~kicad-developers Post to : [email protected] Unsubscribe : https://launchpad.net/~kicad-developers More help : https://help.launchpad.net/ListHelp
