On Mon, May 5, 2014 at 4:57 PM, David Fawcett <[email protected]>wrote:
> On second thought, I see that the .csv file is 2.1 gb! I would suggest > not following the .csv driver approach. > > I would follow Sean's suggestion and just use some simple Python (or > whatever language that you want to use to consume the data) to grab the XY > values and write out geometries. > > Actually I was able to load the whole shebang into qgis using their CSV importer. The problem with that approach is that that file contains observations for the entire planet, and I just want Japan. (Particularly round Fukushima Daiichi.) Fortunately I can clip to the region of interest within qgis, though it's admittedly a little sluggish. ;) I was interested in their API because I thought that would be an easy way to do the clipping on their end, while making getting regular updates viable. Regularly downloading and processing 2Gb data isn't feasible. This is the first time I've dealt with "big data", so I'm still getting my arms round the associated issues. Cheers! Mark -- [email protected]
_______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
