Yeah, I kinda agree with you that we should get basics working first, before dealing with any kind of "auto update" functionallity.
For the next release we should aim to implement functionality for specifying a custom path for the pricing file and function which updates a local pricing data. Another thing we need to think about is how to check if the pricing file is out of date. One simple way would be to add a new "last_updated" field (unix timestamp) to the pricing.json file and only download the file if remote last_updated > local last_updated. Adding this field to the pricing.json file is not that ideal compared to some other possible approaches, because it means we will need to download and parse the remote pricing file, but this shouldn't be a big problem because the pricing file is relatively small. On Thu, May 26, 2011 at 12:12 AM, Paul Querna <[email protected]> wrote: > 2011/5/25 Tomaž Muraus <[email protected]>: > > Hi all, > > > > Now that all the compute pricing data is contained in a single JSON file > > (data/pricing.json), updating it will in 95% cases only mean updating > this > > file. This makes it a lot easier and more simple compared to the old way > > (updating variables in each driver file separately). > > > > We should figure out the best way to distribute this JSON file and > implement > > some kind of auto-update functionality in the library. > > > > There are some problems / things which we need to think about: > > > > - where to host this file? It should probably be hosted on a SSL enabled > > apache.org server > > We can work with infrastructure to host this, or we can get resources > to host it on non-ASF hardware if its problematic > > > - how should this file be updated? Probably manually at first, but later > one > > we can build some really simple scrapers (ideally, all the providers > would > > expose a special API endpoint for retrieving the pricing data and then we > > wouldn't need to deal with this at all). > > In an ideal world, all providers would have an API showing their > current pricing. > > Most don't. > > Scrapping is a step up, and a last resort of manually updating -- > remember for the last year, we have done it manually, inside each > driver, it hasn't been a terrible cost. > > > - where should the file be saved locally? We should probably default to > some > > temporary folder, but user should be able to specify a custom one > > I don't think we have that knowledge, and we shouldn't do it. > > I think we should provide two APIs initially: > > libcloud.pricing.use_file(path) > libcloud.pricing.update(path) > > The use_file would point pricing to use that file instead of the > bundled pricing.json. > > The update function would download it from the URL, and save it locally. > > > - how often to check for updates? We should default to some sane period, > > like every 12-86 hours, but user should be able to override this setting > > - should "auto-update" functionality be enabled by default? > > I think we are jumping ahead too far with auto-updating, I think we > should get the basics in first to have it dynamic, and then if we are > unhappy with that, consider doing optional, explicitly enabled, > automatic updates. > > Thanks, > > Paul >
