Hi Bhibhash, Short answer: No. There is no metadata. I took the top 1000 lines and visualized it : the shapes were spread all over the country and there wasn't any order in them.
With an intern's help I've started import of them into a portable postgresql DB (dockerized with a persistent volume) on my webserver. Started yesterday and it's 10 million shapes in; from size comparisons it'll probably be around 79 million shapes, so should be imported in a week's time. I've set an auto-increment primary key column in the DB table, so once imported at least the data can be pulled out in batches. There are 2 paths ahead once I've imported it: ------- *1. Serving vector tiles* I want to serve MVT vector tiles from this DB - and then have a map display the same. That way one can directly have this stuff render on a map as they browse it - fetching only one small region at a time and hopefully not being too slow. Finding some references here: https://blog.jawg.io/how-to-make-mvt-with-postgis/ https://www.crunchydata.com/blog/dynamic-vector-tiles-from-postgis https://medium.com/@frederic.rodrigo/web-mapping-comparing-vector-tile-servers-from-postgres-postgis-405055e69084 I'm new to this MVT business, so if anybody wants to collaborate pls feel free to jump in. ------ *2. Tag with region ids* Add columns to the DB table and tag each shape by state, district ids etc using Within operations either in the DB or in python-geopandas. -- Cheers, Nikhil VJ https://nikhilvj.co.in On Tue, May 24, 2022 at 10:14 AM Bibhash Nath <bibhas...@gmail.com> wrote: > Thanks for sharing. > Do we have state-wise datasets? It seems quite large to process on my > computer. > > Best, > Bibhash > > On Wed, May 18, 2022 at 4:58 AM Nikhil VJ <nikhil...@gmail.com> wrote: > >> Thanks Justin for sharing! >> >> I've downloaded and extracted the India.geojsonl on a webserver. >> >> ref about .geojsonl : https://www.interline.io/blog/geojsonl-extracts/ >> -> pretty useful! One can loop through a huge file without having to load >> it all into RAM. >> >> top lines look like: >> {"type": "Feature", "properties": {},"geometry": {"type": >> "Polygon","coordinates": [[[83.06380515611697, 25.34167404697847], >> [83.06380909901775, 25.341635591519122], [83.06386494585949, >> 25.341640268589657], [83.06386100295869, 25.341678724047526], >> [83.06380515611697, 25.34167404697847]]]}} >> {"type": "Feature", "properties": {},"geometry": {"type": >> "Polygon","coordinates": [[[87.87555977691633, 22.397660095199], >> [87.8754256865811, 22.397658576750985], [87.87542690191553, >> 22.397566835386712], [87.87556099225075, 22.39756835383578], >> [87.87555977691633, 22.397660095199]]]}} >> ... >> So we have just basic polygons, one for each building, no properties or >> categorization. >> >> Here are some ideas on what to do with this: >> >> 1. Postgresql DB: >> - Load all of these into a PostGreSQL DB >> - Setup an api that will take a lat/lon and give all shapes within 1km >> radius >> - Next possible api: send a bounding polygon and get all buildings in that >> >> 2. Split up by district or lower level admin boundaries: >> - Load the admin boundaries into a Postgresql DB >> - Loop through each line (ie each building) >> - Find out which place it's in by ST_Within query >> - Dump it into a separate .geojsonl for that place >> - We've now split it into multiple smaller files that are more usable, >> can be loaded up by OSM mappers etc. >> >> -- >> Cheers, >> Nikhil VJ >> https://nikhilvj.co.in >> >> >> On Wed, May 18, 2022 at 3:51 AM Justin <justinelliotmey...@gmail.com> >> wrote: >> >>> [image: lg.png] >>> >>> https://github.com/microsoft/GlobalMLBuildingFootprints >>> >>> >>> -- >>> Datameet is a community of Data Science enthusiasts in India. Know more >>> about us by visiting http://datameet.org >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "datameet" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to datameet+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/datameet/68c0ecc4-5b07-4eb9-a69d-ae6e5ad6832bn%40googlegroups.com >>> <https://groups.google.com/d/msgid/datameet/68c0ecc4-5b07-4eb9-a69d-ae6e5ad6832bn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> Datameet is a community of Data Science enthusiasts in India. Know more >> about us by visiting http://datameet.org >> --- >> You received this message because you are subscribed to the Google Groups >> "datameet" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to datameet+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/datameet/CAH7jeuM_yRYLDRir7vYjT7JVoP-J%2BkWAEkbp_U84QSb82AX3aA%40mail.gmail.com >> <https://groups.google.com/d/msgid/datameet/CAH7jeuM_yRYLDRir7vYjT7JVoP-J%2BkWAEkbp_U84QSb82AX3aA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > Datameet is a community of Data Science enthusiasts in India. Know more > about us by visiting http://datameet.org > --- > You received this message because you are subscribed to the Google Groups > "datameet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to datameet+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/datameet/CAHRkDO5qPr_E68nzFiTh%3DxSPd1myz0_WMbJi83mdQMDZ53pEkQ%40mail.gmail.com > <https://groups.google.com/d/msgid/datameet/CAHRkDO5qPr_E68nzFiTh%3DxSPd1myz0_WMbJi83mdQMDZ53pEkQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups "datameet" group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CAH7jeuOo0vfbxnsv%2B-V7%2Bt7zgkNGruyaT0xzULECGc-e8h3Qcw%40mail.gmail.com.
