Using the raw end point sounds fine to me for now. Easy to change later if needed.
On Thu, 9 Jun 2016 8:14 am Akbar Gumbira <akbargumb...@gmail.com> wrote: > Hi Ale, > > I just tried Dulwich. There are three ways that I see to checkout a single > file: git sparse checkout, git archive, or shallow clone (still cloning all > files though). > > With git sparse checkout, Dulwich does not provide the porcelain-level > API. The author says that it's possible though ( > https://github.com/jelmer/dulwich/issues/405#issuecomment-224579678). I > am not sure how much work to implement it dealing with the git object, > tree, and blob itself at the plumbing level. > > With git archive, unfortunately Github doesn't allow us to do this ( > https://twitter.com/GitHubHelp/status/322818593748303873). WIth shallow > clone left, I feel that I would over-engineer it forcing to use git just to > be able to do this. I think it's more simple if for now we use the raw > endpoint for each hosts that we know (Github, Bitbucket, etc), and we add > more supported hosts as users put the collection in other hosts as long as > we know the URL to the raw metadata file. > > What do you think? > > Cheers > > On Wed, Jun 8, 2016 at 9:51 AM, Akbar Gumbira <akbargumb...@gmail.com> > wrote: > >> Wasn't that a requisite? If we are going to use git as a storage we need >>> a git client (pure python seems to exist, could be used as a fallback in >>> case git is not installed in the system) >>> https://github.com/jelmer/dulwich . >>> >> >> >>> BTW: if you do not want to deal with git in this first task, you could >>> used the zip endpoint, and assume that the metadata will be available at a >>> known location in the remote http repo and just wget it. >> >> >> Ok. I was just trying to find solution that could work out well without >> having git (as from the last discussion with Martin, it's better to avoid >> using git for now). I will try pygit2 and dulwich and I'll report you back >> what I thought using those two. >> >> On Wed, Jun 8, 2016 at 9:41 AM, Alessandro Pasotti <apaso...@gmail.com> >> wrote: >> >>> 2016-06-08 9:21 GMT+02:00 Akbar Gumbira <akbargumb...@gmail.com>: >>> >>>> Yes, I have tried it. We can do it with sparse checkout, but, it >>>> requires git on the client. >>>> >>> >>> >>> Wasn't that a requisite? If we are going to use git as a storage we need >>> a git client (pure python seems to exist, could be used as a fallback in >>> case git is not installed in the system) >>> https://github.com/jelmer/dulwich . >>> >>> BTW: if you do not want to deal with git in this first task, you could >>> used the zip endpoint, and assume that the metadata will be available at a >>> known location in the remote http repo and just wget it. >>> >>> >>> >>>> >>>> On Wed, Jun 8, 2016 at 9:18 AM, Alessandro Pasotti <apaso...@gmail.com> >>>> wrote: >>>> >>>>> >>>>> 2016-06-08 9:10 GMT+02:00 Akbar Gumbira <akbargumb...@gmail.com>: >>>>> >>>>>> Hi Akbar, >>>>>>> The most flexible installation tool that I know is probably python >>>>>>> pip. >>>>>>> pip can install software from a zip file, from git and from other >>>>>>> sources too. >>>>>>> I'd suggest you to have a look to pip implementation of the install >>>>>>> functionality, maybe there is some interesting for you. >>>>>> >>>>>> I just read pip code base. I think I can pick something from there >>>>>> for downloading the resources. But the problem I have right now is to get >>>>>> only the metadata file from the repository. Or are you suggesting that >>>>>> when >>>>>> users add a repository connection, it also downloads the repository >>>>>> directly? >>>>>> >>>>> >>>>> >>>>> If you are working on the git repos you can probably fetch just one >>>>> file without cloning the whole repo, I did not test it but here are >>>>> probably some pointers: >>>>> >>>>> >>>>> http://stackoverflow.com/questions/2466735/how-to-checkout-only-one-file-from-git-repository >>>>> >>>>> http://stackoverflow.com/questions/1125476/retrieve-a-single-file-from-a-repository >>>>> >>>>> >>>>> >>>>> >>>>>> >>>>>> On Mon, Jun 6, 2016 at 8:39 AM, Alessandro Pasotti < >>>>>> apaso...@gmail.com> wrote: >>>>>> >>>>>>> 2016-06-05 10:13 GMT+02:00 Richard Duivenvoorde <rdmaili...@duif.net >>>>>>> >: >>>>>>> >>>>>>>> On 05-06-16 09:02, Akbar Gumbira wrote: >>>>>>>> >>>>>>>> > *Are you blocked on anything?* >>>>>>>> > ... In Github or Bitbucket they provide a direct link to >>>>>>>> > the raw file. But I think I should look at more general approach >>>>>>>> without >>>>>>>> > manipulating the URL depending on the host. If you have some >>>>>>>> input, I >>>>>>>> > would be happy to assess it. >>>>>>>> >>>>>>>> Thanks Akbar, >>>>>>>> >>>>>>>> I did some googling: >>>>>>>> >>>>>>>> http://stackoverflow.com/questions/14405782/git-fetch-single-file-from-remote-repository-programatically >>>>>>>> If you really want to keep it git, it looks like a shallow >>>>>>>> clone/copy is >>>>>>>> the only way? That post also talks about some undocumented feature, >>>>>>>> but >>>>>>>> I would not depend on that? >>>>>>>> >>>>>>>> Personally I would be ok when both Github and Gitlab/Gog would work >>>>>>>> (as >>>>>>>> both a closed source and open source member of the git-web-world)... >>>>>>>> >>>>>>>> Or: a script running somewhere on our server, (shallow) cloning all >>>>>>>> registred repositories periodically, and making just the >>>>>>>> metadata.txt >>>>>>>> files available via http/webserver? (maybe giving us some time to >>>>>>>> check >>>>>>>> the repo's on structure and (malicious?) content? >>>>>>>> >>>>>>>> Or else: a django app for the metadata... >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Richard >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Hi Akbar, >>>>>>> >>>>>>> The most flexible installation tool that I know is probably python >>>>>>> pip. >>>>>>> >>>>>>> pip can install software from a zip file, from git and from other >>>>>>> sources too. >>>>>>> >>>>>>> I'd suggest you to have a look to pip implementation of the install >>>>>>> functionality, maybe there is some interesting for you. >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Alessandro Pasotti >>>>>>> w3: www.itopen.it >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> *-------------------* >>>>>> *Akbar Gumbira * >>>>>> *www.akbargumbira.com <http://www.akbargumbira.com>* >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Alessandro Pasotti >>>>> w3: www.itopen.it >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> *-------------------* >>>> *Akbar Gumbira * >>>> *www.akbargumbira.com <http://www.akbargumbira.com>* >>>> >>> >>> >>> >>> -- >>> Alessandro Pasotti >>> w3: www.itopen.it >>> >> >> >> >> -- >> >> *-------------------* >> *Akbar Gumbira * >> *www.akbargumbira.com <http://www.akbargumbira.com>* >> > > > > -- > > *-------------------* > *Akbar Gumbira * > *www.akbargumbira.com <http://www.akbargumbira.com>* > _______________________________________________ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer