Hi Nils, Thanks for sharing.
S On Mon, Jun 6, 2016 at 6:47 AM Nils Hempelmann <info at nilshempelmann.de> wrote: > Hi Scott et al. > > I implemented the pygbif in the SDM in birdhouse. Since there all required > dependencies are fetched as conda buildouts Carsten (from DKRZ) was > building a conda buildout. > For those who uses anaconda (https://www.continuum.io/why-anaconda) it > can be installed with > > conda install -c birdhouse pygbif > > Best > Nils > > > > On 02/06/2016 16:17, Scott Chamberlain wrote: > > Nils, > > We do have some download API functions in pygbif, and there's a pull > request now to add the methods to request downloads. We'll add some > functionality to read darwin core dumps as well. > > Scott > > On Thu, Jun 2, 2016 at 6:06 AM Tim Robertson <trobertson at gbif.org> wrote: > >> Hi Nils, >> >> It?s documented here: http://www.gbif.org/developer/occurrence#download >> You POST a JSON doc with the query using HTTP basic authentication. >> >> If you need any help, please say and we can provide some examples using >> CURL or python. >> >> I am actually working on a revision to the mapping API, and will see what >> may be possible returning distinct locations. It?s tricky to do in real >> time though. >> What kind of accuracy do you need please? >> >> Thanks, >> Tim >> >> From: Nils Hempelmann <info at nilshempelmann.de> >> Date: Thursday 2 June 2016 at 15:00 >> To: Tim Robertson <trobertson at gbif.org>, "api-users at lists.gbif.org" < >> api-users at lists.gbif.org>, "wps-dev at lists.dkrz.de" <wps-dev at >> lists.dkrz.de >> > >> Subject: Re: [API-users] birdhouse meets GBIF >> >> Hi Tim >> >> Thanks for the quick answer. If not OGC service, how is the download API >> callable from outside of GBIF? >> >> Merci >> Nils >> >> >> On 02/06/2016 14:45, Tim Robertson wrote: >> >> Hi Nils >> >> We don?t have any OGC services, but there is an asynchronous download API >> which can deliver CSVs. >> Off the top of my head, the only way you can automate this at the moment >> would be to do periodically issue a download (e.g. Daily) process as you >> see fit, and cache the result for your app. >> >> In the download API you can get any size from 1 - > 660 million records, >> which is why it is asynchronous. It?s used a lot by various applications >> and communities. >> >> I hope this helps, >> Tim >> >> >> From: API-users <api-users-bounces at lists.gbif.org> on behalf of Nils >> Hempelmann <info at nilshempelmann.de> >> Date: Thursday 2 June 2016 at 14:25 >> To: "api-users at lists.gbif.org" <api-users at lists.gbif.org>, " >> wps-dev at lists.dkrz.de" <wps-dev at lists.dkrz.de> >> Subject: [API-users] birdhouse meets GBIF >> >> Dear all >> >> Here is an current problem to solve :-) for the birdhouse WPS and GBIF >> developers. >> There is a species distribution process in the birdhouse WPS, processing >> Climate data based on occurrence coordinates from GBIF. >> >> Currently the GBIF data are given to the process via an url of the GBIF >> csv (which had to be generated manually before). >> I was implementing a direct occurrence search from birdhouse to GBIF with >> pygbif. Worked fine BUT: there is a limit of 300 records which is far to >> less to train the Species distribution model. >> >> So here the question: Is there a way to request the species occurrence >> coordinates somehow directly (e.g. like a WPS request)? >> >> (The previous conversation is following) >> >> And some link for the background informations: >> >> Birdhouse: >> http://birdhouse.readthedocs.io/en/latest/ >> GBIF: >> http://www.gbif.org/ >> >> Species distribution process: >> http://flyingpigeon.readthedocs.io/en/latest/tutorials/sdm.html >> >> Birdhouse architecture: >> >> https://github.com/bird-house/birdhouse-docs/blob/master/slides/birdhouse-architecture/birdhouse-architecture.pdf >> >> Merci >> Nils >> >> -------- Forwarded Message ------- >> Subject: Re: pygbif for occurrence coordinates >> Date: Thu, 2 Jun 2016 09:00:49 -0300 >> From: Mauro Cavalcanti <maurobio at gmail.com> <maurobio at gmail.com> >> <maurobio at gmail.com> >> To: Nils Hempelmann <info at nilshempelmann.de> <info at nilshempelmann.de> >> <info at nilshempelmann.de> >> CC: wps-dev at lists.dkrz.de, Carsten Ehbrecht <ehbrecht at dkrz.de> >> <ehbrecht at dkrz.de>, Wolfgang.Falk at lwf.bayern.de, Scott Chamberlain >> <myrmecocystus at gmail.com> <myrmecocystus at gmail.com> >> <myrmecocystus at gmail.com> >> >> Nils, >> >> No, there is no other way to get at once a larger number of records using >> the GBIF API (although one could be able to achieve this by sequentially >> querying the server using "batches" of 200 or 300 records each). As I said, >> there are good operational reasons for the GBIF developers to have imposed >> such limit. >> >> As of your other question, it should better be put to the GBIF developers >> themselves (in the discussion list, so that we can all benefit from the >> answers! :-)) >> >> With warmest regards, >> >> -- >> Dr. Mauro J. Cavalcanti >> E-mail: maurobio at gmail.com >> Web: http://sites.google.com/site/maurobio >> Em 02/06/2016 08:49, "Nils Hempelmann" <info at nilshempelmann.de> escreveu: >> >>> Hi Mauro >>> >>> Oh ... >>> No way to set it unlimited? >>> The 'manual via browser' option and parsing the returning csv is the >>> current status. >>> or any alternative to pygbif? >>> >>> If I understood it correctly, GBIF database is organized as Web Server, >>> so I gues there should be a way to connect to the birdhouse WPS, am I >>> right? >>> >>> (put the web-dev list in copy) >>> >>> Merci >>> Nils >>> >>> >>> On 02/06/2016 12:40, Mauro Cavalcanti wrote: >>> >>> Nils, >>> >>> That's a limit imposed (for good operational reasons) by the GBIF API. >>> If you want a larger number of records, you'll have to download them >>> "manually" (that is, via browser) and then parse locally the csv file >>> returned from the GBIF server. >>> >>> Hope this helps. >>> >>> Best regards, >>> >>> -- >>> Dr. Mauro J. Cavalcanti >>> E-mail: maurobio at gmail.com >>> Web: http://sites.google.com/site/maurobio >>> Em 02/06/2016 04:48, "Nils Hempelmann" <info at nilshempelmann.de> >>> escreveu: >>> >>>> Hi Scott >>>> >>>> works fine. Thanks a lot. Just have a question for the search limits: >>>> >>>> The maximal records seems to be limited to 300. Is that on purpose? >>>> And requesting more than 200000 gives a 'bad request' >>>> >>>> Merci >>>> Nils >>>> >>>> >>>> In [68]: len( occurrences.search(taxonKey=key, limit=100)['results']) >>>> Out[68]: 100 >>>> >>>> In [69]: len( occurrences.search(taxonKey=key, limit=300)['results']) >>>> Out[69]: 300 >>>> >>>> In [70]: len( occurrences.search(taxonKey=key, limit=3000)['results']) >>>> Out[70]: 300 >>>> >>>> In [71]: len( occurrences.search(taxonKey=key, limit=200000)['results']) >>>> Out[71]: 300 >>>> >>>> In [72]: len( occurrences.search(taxonKey=key, limit=200001)['results']) >>>> >>>> --------------------------------------------------------------------------- >>>> HTTPError Traceback (most recent call >>>> last) >>>> <ipython-input-72-2f7d7b4ccba0> in <module>() >>>> ----> 1 len( occurrences.search(taxonKey=key, limit=200001)['results']) >>>> >>>> /home/nils/.conda/envs/birdhouse/lib/python2.7/site-packages/pygbif/occurrences/search.pyc >>>> in search(taxonKey, scientificName, country, publishingCountry, >>>> hasCoordinate, typeStatus, recordNumber, lastInterpreted, continent, >>>> geometry, recordedBy, basisOfRecord, datasetKey, eventDate, catalogNumber, >>>> year, month, decimalLatitude, decimalLongitude, elevation, depth, >>>> institutionCode, collectionCode, hasGeospatialIssue, issue, q, mediatype, >>>> limit, offset, **kwargs) >>>> 251 'collectionCode': collectionCode, 'hasGeospatialIssue': >>>> hasGeospatialIssue, >>>> 252 'issue': issue, 'q': q, 'mediatype': mediatype, >>>> 'limit': limit, >>>> --> 253 'offset': offset}, **kwargs) >>>> 254 return out >>>> >>>> /home/nils/.conda/envs/birdhouse/lib/python2.7/site-packages/pygbif/gbifutils.pyc >>>> in gbif_GET(url, args, **kwargs) >>>> 17 def gbif_GET(url, args, **kwargs): >>>> 18 out = requests.get(url, params=args, headers=make_ua(), >>>> **kwargs) >>>> ---> 19 out.raise_for_status() >>>> 20 stopifnot(out.headers['content-type']) >>>> 21 return out.json() >>>> >>>> /home/nils/.conda/envs/birdhouse/lib/python2.7/site-packages/requests/models.pyc >>>> in raise_for_status(self) >>>> 842 >>>> 843 if http_error_msg: >>>> --> 844 raise HTTPError(http_error_msg, response=self) >>>> 845 >>>> 846 def close(self): >>>> >>>> HTTPError: 400 Client Error: Bad Request for url: >>>> <http://api.gbif.org/v1/occurrence/search?taxonKey=2882316&limit=200001&offset=0> >>>> http://api.gbif.org/v1/occurrence/search?taxonKey=2882316&limit=200001&offset=0 >>>> >>>> In [73]: >>>> >>>> >>>> On 02/06/2016 02:22, Scott Chamberlain wrote: >>>> >>>> Fixes to docs and fix to download_get are up on Github now. Will push >>>> new version to pypi soon. Let me know if some thing still don't work for >>>> you after reinstalling from github >>>> >>>> S >>>> >>>> On Wed, Jun 1, 2016 at 4:51 PM Nils Hempelmann <info at nilshempelmann.de> >>>> wrote: >>>> >>>>> Hi Scott and Mauro >>>>> >>>>> Mauro sended me a snipped of code which worked fine: >>>>> >>>>> https://github.com/bird-house/flyingpigeon/blob/develop/scripts/pygbif_occurence.py >>>>> >>>>> I found the example here: >>>>> https://github.com/maurobio/pygbif#occurrence-data >>>>> >>>>> and here: >>>>> https://github.com/sckott/pygbif#occurrences-module >>>>> >>>>> Thanks a lot very great. That enables a lot ;-) >>>>> I ll keep you posted >>>>> >>>>> merci >>>>> Nils >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 02/06/2016 01:45, Scott Chamberlain wrote: >>>>> >>>>> And where are those example from exactly? I don't see those examples >>>>> searching the repo (which includes all docs). >>>>> >>>>> `pygbif.name_suggest` wouldn't work because `name_suggest` is a method >>>>> in the `species` module, so you'd have to do pygbif.species.name_suggest, >>>>> or `from pygbif import species`, then `species.name_suggest` >>>>> >>>>> Looks like occ.get(taxonKey = 252408386) is a documentation bug, >>>>> that should be `key` instead of `taxonKey`, a copy paste error. will fix >>>>> that. >>>>> >>>>> The `occ.download_get` call has a small bug, will fix that >>>>> >>>>> All other calls work for me >>>>> >>>>> S >>>>> >>>>> On Wed, Jun 1, 2016 at 4:34 PM Scott Chamberlain < >>>>> myrmecocystus at gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> What version of pygbif are you on? And what version of Python> >>>>>> >>>>>> Best, Scott >>>>>> >>>>>> On Wed, Jun 1, 2016 at 4:02 PM Nils Hempelmann < >>>>>> info at nilshempelmann.de> wrote: >>>>>> >>>>>>> Hi Mauro and Scott >>>>>>> >>>>>>> Was checking out pygbif. seems to be a very useful tool. >>>>>>> >>>>>>> Can you help me with the syntax (or forward me to the appropriate >>>>>>> person >>>>>>> ;-) ) >>>>>>> The given snippets of code are outdated. >>>>>>> >>>>>>> I am basically just looking for the occurrence coordinates: >>>>>>> here is my first try : >>>>>>> >>>>>>> import pygbif >>>>>>> occ = pygbif.occurrences.search(scientificName='Fagus sylvatica') >>>>>>> occ['count'] >>>>>>> >>>>>>> ... and further? ;-) >>>>>>> >>>>>>> the examples in the docu are throwing errors: >>>>>>> >>>>>>> key= pygbif.name_suggest(q='Helianthus >>>>>>> annuus',rank='species')['key'] >>>>>>> pygbif.search(taxonKey=key[0]['key'],limit=2) >>>>>>> >>>>>>> from pygbif import occurrences as occ occ.search(taxonKey = 3329049) >>>>>>> occ.get(taxonKey = 252408386) occ.count(isGeoreferenced = True) >>>>>>> occ.download_list(user = "sckott", limit = 5) >>>>>>> occ.download_meta(key = "0000099-140929101555934") >>>>>>> occ.download_get("0000099-140929101555934") >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Nils >>>>>>> >>>>>>> >>>>> >>>> >>> >> >> >> _______________________________________________ >> API-users mailing list >> API-users at lists.gbif.org >> http://lists.gbif.org/mailman/listinfo/api-users >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gbif.org/pipermail/api-users/attachments/20160607/a4b797cc/attachment-0001.html>