Hello Nikhil, I would definitely love to collobrate. The best i would suggest is to make Mbtiles and host them as data is not going to change a lot so most of the time static.
Rendering from hosted Mbtiles will not only fast but easy to handle on small web server. How to proceed. First creation of Mbtiles second Hosting of Mbtiles Third Styling at client side. Thanks On Tue, 24 May, 2022, 1:03 pm Nikhil VJ, <nikhil...@gmail.com> wrote: > Hi Bhibhash, > > Short answer: No. > There is no metadata. I took the top 1000 lines and visualized it : the > shapes were spread all over the country and there wasn't any order in them. > > With an intern's help I've started import of them into a portable > postgresql DB (dockerized with a persistent volume) on my webserver. > Started yesterday and it's 10 million shapes in; from size comparisons > it'll probably be around 79 million shapes, so should be imported in a > week's time. > > I've set an auto-increment primary key column in the DB table, so once > imported at least the data can be pulled out in batches. > > There are 2 paths ahead once I've imported it: > ------- > > *1. Serving vector tiles* > I want to serve MVT vector tiles from this DB - and then have a map > display the same. That way one can directly have this stuff render on a map > as they browse it - fetching only one small region at a time and hopefully > not being too slow. > > Finding some references here: > https://blog.jawg.io/how-to-make-mvt-with-postgis/ > https://www.crunchydata.com/blog/dynamic-vector-tiles-from-postgis > > https://medium.com/@frederic.rodrigo/web-mapping-comparing-vector-tile-servers-from-postgres-postgis-405055e69084 > > I'm new to this MVT business, so if anybody wants to collaborate pls feel > free to jump in. > > ------ > > *2. Tag with region ids* > Add columns to the DB table and tag each shape by state, district ids etc > using Within operations either in the DB or in python-geopandas. > > > -- > Cheers, > Nikhil VJ > https://nikhilvj.co.in > > > On Tue, May 24, 2022 at 10:14 AM Bibhash Nath <bibhas...@gmail.com> wrote: > >> Thanks for sharing. >> Do we have state-wise datasets? It seems quite large to process on my >> computer. >> >> Best, >> Bibhash >> >> On Wed, May 18, 2022 at 4:58 AM Nikhil VJ <nikhil...@gmail.com> wrote: >> >>> Thanks Justin for sharing! >>> >>> I've downloaded and extracted the India.geojsonl on a webserver. >>> >>> ref about .geojsonl : https://www.interline.io/blog/geojsonl-extracts/ >>> -> pretty useful! One can loop through a huge file without having to >>> load it all into RAM. >>> >>> top lines look like: >>> {"type": "Feature", "properties": {},"geometry": {"type": >>> "Polygon","coordinates": [[[83.06380515611697, 25.34167404697847], >>> [83.06380909901775, 25.341635591519122], [83.06386494585949, >>> 25.341640268589657], [83.06386100295869, 25.341678724047526], >>> [83.06380515611697, 25.34167404697847]]]}} >>> {"type": "Feature", "properties": {},"geometry": {"type": >>> "Polygon","coordinates": [[[87.87555977691633, 22.397660095199], >>> [87.8754256865811, 22.397658576750985], [87.87542690191553, >>> 22.397566835386712], [87.87556099225075, 22.39756835383578], >>> [87.87555977691633, 22.397660095199]]]}} >>> ... >>> So we have just basic polygons, one for each building, no properties or >>> categorization. >>> >>> Here are some ideas on what to do with this: >>> >>> 1. Postgresql DB: >>> - Load all of these into a PostGreSQL DB >>> - Setup an api that will take a lat/lon and give all shapes within 1km >>> radius >>> - Next possible api: send a bounding polygon and get all buildings in >>> that >>> >>> 2. Split up by district or lower level admin boundaries: >>> - Load the admin boundaries into a Postgresql DB >>> - Loop through each line (ie each building) >>> - Find out which place it's in by ST_Within query >>> - Dump it into a separate .geojsonl for that place >>> - We've now split it into multiple smaller files that are more usable, >>> can be loaded up by OSM mappers etc. >>> >>> -- >>> Cheers, >>> Nikhil VJ >>> https://nikhilvj.co.in >>> >>> >>> On Wed, May 18, 2022 at 3:51 AM Justin <justinelliotmey...@gmail.com> >>> wrote: >>> >>>> [image: lg.png] >>>> >>>> https://github.com/microsoft/GlobalMLBuildingFootprints >>>> >>>> >>>> -- >>>> Datameet is a community of Data Science enthusiasts in India. Know more >>>> about us by visiting http://datameet.org >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "datameet" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to datameet+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/datameet/68c0ecc4-5b07-4eb9-a69d-ae6e5ad6832bn%40googlegroups.com >>>> <https://groups.google.com/d/msgid/datameet/68c0ecc4-5b07-4eb9-a69d-ae6e5ad6832bn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> Datameet is a community of Data Science enthusiasts in India. Know more >>> about us by visiting http://datameet.org >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "datameet" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to datameet+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/datameet/CAH7jeuM_yRYLDRir7vYjT7JVoP-J%2BkWAEkbp_U84QSb82AX3aA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/datameet/CAH7jeuM_yRYLDRir7vYjT7JVoP-J%2BkWAEkbp_U84QSb82AX3aA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> Datameet is a community of Data Science enthusiasts in India. Know more >> about us by visiting http://datameet.org >> --- >> You received this message because you are subscribed to the Google Groups >> "datameet" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to datameet+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/datameet/CAHRkDO5qPr_E68nzFiTh%3DxSPd1myz0_WMbJi83mdQMDZ53pEkQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/datameet/CAHRkDO5qPr_E68nzFiTh%3DxSPd1myz0_WMbJi83mdQMDZ53pEkQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > Datameet is a community of Data Science enthusiasts in India. Know more > about us by visiting http://datameet.org > --- > You received this message because you are subscribed to the Google Groups > "datameet" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to datameet+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/datameet/CAH7jeuOo0vfbxnsv%2B-V7%2Bt7zgkNGruyaT0xzULECGc-e8h3Qcw%40mail.gmail.com > <https://groups.google.com/d/msgid/datameet/CAH7jeuOo0vfbxnsv%2B-V7%2Bt7zgkNGruyaT0xzULECGc-e8h3Qcw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- Datameet is a community of Data Science enthusiasts in India. Know more about us by visiting http://datameet.org --- You received this message because you are subscribed to the Google Groups "datameet" group. To unsubscribe from this group and stop receiving emails from it, send an email to datameet+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/datameet/CA%2B4SsQCfA5dxiWne%3Dsghbn8OH_FoBgKUDtVeF9hMs%2BxU6he_JQ%40mail.gmail.com.
