Those files have now been made public. I will publish a blog post with some details on that in the near future.
On Sat, Jul 11, 2020 at 8:48 PM Tomaz Muraus <to...@apache.org> wrote: > I added some information on this new behavior here - > https://github.com/apache/libcloud/blob/f122600d2adf181a9b100cdd552cd02979c5b1b9/docs/compute/pricing.rst#downloading-latest-pricing-data-from-an-s3-bucket > > Keep in mind that those 3 files are not public yet. I plan to make them > public and read-only in the near future once the rate limits are sorted out. > > On Sat, Jul 11, 2020 at 4:07 PM Tomaz Muraus <to...@apache.org> wrote: > >> Yeah, I would actually prefer a git repository so everything is version >> controlled, etc., but I went with the fastest and simplest approach >> possible. >> >> I'm not exactly sure what the ASF rules are for something like that (I >> would need to ask ASF infra team to create a new repo, create a bot account >> which we could use in our CI, etc.) and that would likely take much longer >> than the approach I went with. >> >> As far as libraries such as pytz (and to some extent also certifi) go - I >> would say it's a slightly different there - time zones tend to change much >> less frequently than provider pricings so publishing a new library package >> every now and then is probably sufficient. >> >> On Sat, Jul 11, 2020 at 2:38 PM Samuel Marks <samuelma...@gmail.com> >> wrote: >> >>> The other solution is to create a new git repository just for frequently >>> updated files like this one… I mean we don't want to end up like pytz do >>> we? >>> >>> PS: A good thing about pytz is other languages literally just parse >>> pytz's >>> list for their own timezone implementation. No Python. Easy! - With this >>> being in JSON, I could imagine using Terraform libraries in Go; instead >>> of >>> Libcloud; to do multicloud, and use this costing system to say where and >>> when. >>> >>> Samuel Marks >>> Charity <https://sydneyscientific.org> | consultancy < >>> https://offscale.io> >>> | open-source <https://github.com/offscale> | LinkedIn >>> <https://linkedin.com/in/samuelmarks> >>> >>> >>> On Thu, Jul 2, 2020 at 9:51 PM Jay Rolette <role...@infinite.io> wrote: >>> >>> > Same here! >>> > >>> > Thanks, >>> > Jay >>> > >>> > On Wed, Jul 1, 2020 at 12:45 PM Francisco Ros <fj...@doalitic.com> >>> wrote: >>> > >>> > > Hey Tomaz, >>> > > >>> > > I'd really love to see this :-) >>> > > >>> > > Thanks, >>> > > Francisco >>> > > >>> > > > El 1 jul 2020, a las 12:00, Tomaz Muraus <to...@apache.org> >>> escribió: >>> > > > >>> > > > Recently one of the Libcloud contributors (Eis-D-Z) published >>> various >>> > > > improvements to our price scraping scripts and added some new ones >>> - >>> > > > https://github.com/apache/libcloud/pulls/Eis-D-Z. >>> > > > >>> > > > I think it would now make sense to run those scraping scripts on a >>> > > > continuous basis as part of our CI (e.g. once a day) and publish >>> the >>> > > > generated file to some well known location (e.g. public read-only >>> S3 >>> > > > bucket). >>> > > > >>> > > > In fact, that was also the plan when we originally >>> > > > added libcloud.pricing.download_pricing_file function and related >>> > > > functionality quite a long time ago. >>> > > > >>> > > > IIRC, the plan was to include an auto-generated pricing file >>> directly >>> > > > inside the git repo, but this is more complicated and I would need >>> to >>> > > > contact the ASF infra team if they even allow something like that >>> > > (updating >>> > > > and committing a change as a bot user on our CI - Travis CI). >>> > > > >>> > > > So for now, I will probably just publish this auto-generated >>> > pricing.json >>> > > > file to a public read-only S3 bucket (I will make sure to set up >>> > correct >>> > > > rate limits and alerts to prevent abuse, even though the pricing >>> file >>> > > > itself is quite small). >>> > > > >>> > > > What do other people think? >>> > > >>> > > >>> > >>> >>
