Re: [Qgis-user] Search through folders and combine SHP files

[Nyall Dawson via QGIS-User]

On Fri, 10 May 2024 at 09:06, Tony Shepherd (FarmMaps NZ) via QGIS-User <
qgis-user@lists.osgeo.org> wrote:

> Hi
>
>
>
> I have loads, literally thousands, of SHP, GPKG, TAB files located in
> loads of different folders and sub-folders.
>
>
>
> Is there a plugin that will trawl through a set of folders, copy say just
> the polygons from each file into a new file, and populate a field with say
> the filename and folder the polygon(s) came from?
>
>
>
> Essentially combining a heap of files into one with some metadata about
> where the data came from?
>
>
>
> Why?  I have files from clients dating back about 20 years.  Clients sell
> properties.  Properties get subdivided.  New clients come and see us, and
> often we don’t realise we have GIS info on file for that property from many
> years ago, usually under a different name.  It would be handy to see a
> polygon on the screen and easily go back to some old GIS data.
>
>
>
> Maybe I am missing an easy tool to do this, but I can’t see anything
> obvious.
>

You could use the "Merge Vector Layers" tool from the Processing toolbox,
that does everything you want. The trickiest bit would be getting a master
list of ALL your shapefiles/etc you want to combine and passing this to the
tool. It's not going to be easy from the GUI itself, as that only allows
you to add files from a single folder at once.

I'd run the tool on the files from a single folder, then checkout the
processing history dialog and copy the python command which corresponds to
what you just did. You could then adapt this command to pass a complete
list of all the files you want to combine. (It's quite straightforward in
Python to build a list of files which recursively match a file pattern --
see eg https://stackoverflow.com/a/2186565 )

Hope that helps!
Nyall


>
>
>
>
> Cheers
>
> Tony
>
>
>
>
> --
>
> *Tony Shepherd*
>
> *  GeoSpatial Manager @ FarmMaps NZ & Photographer @ Shepherd Photos*
>
>
>
> *Phone* *–* 027 435 6193  | *Website – *shepherdphotos.co.nz
>
> *Email** – Maps* t...@farmmaps.nz  |  *Email – Photos*
> t...@shepherdphotos.co.nz
>
> *Facebook* TonyShepherdPhotos
> 
>
>
>
>
>
> [image: FM_logo1_small (Custom)]
>
>
> ___
> QGIS-User mailing list
> QGIS-User@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Search through folders and combine SHP files

[chris hermansen via QGIS-User]

Tony and list,

On Thu, May 9, 2024, 16:06 Tony Shepherd (FarmMaps NZ) via QGIS-User <
qgis-user@lists.osgeo.org> wrote:

> Hi
>
>
>
> I have loads, literally thousands, of SHP, GPKG, TAB files located in
> loads of different folders and sub-folders.
>
>
>
> Is there a plugin that will trawl through a set of folders, copy say just
> the polygons from each file into a new file, and populate a field with say
> the filename and folder the polygon(s) came from?
>
>
>
> Essentially combining a heap of files into one with some metadata about
> where the data came from?
>
>
>
> Why?  I have files from clients dating back about 20 years.  Clients sell
> properties.  Properties get subdivided.  New clients come and see us, and
> often we don’t realise we have GIS info on file for that property from many
> years ago, usually under a different name.  It would be handy to see a
> polygon on the screen and easily go back to some old GIS data.
>
>
>
> Maybe I am missing an easy tool to do this, but I can’t see anything
> obvious.
>

Have you considered creating a PostGIS database, where each file imported
is in a table named for the source? You could also create a companion
metadata table.

You would script something like this with shp2pgsql

https://postgis.net/docs/using_postgis_dbmanagement.html

To load the spatial data and likely the psql client with CREATE TABLE  and
INSERT statements in an SQL script.

If you're on Linux or OS/X you would use the find command to create the
shell script. Probably something similar in Power Shell if you're on
Windows.

>
___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

[Qgis-user] Search through folders and combine SHP files

[Tony Shepherd (FarmMaps NZ) via QGIS-User]

Hi

 

I have loads, literally thousands, of SHP, GPKG, TAB files located in loads
of different folders and sub-folders.

 

Is there a plugin that will trawl through a set of folders, copy say just
the polygons from each file into a new file, and populate a field with say
the filename and folder the polygon(s) came from?

 

Essentially combining a heap of files into one with some metadata about
where the data came from?

 

Why?  I have files from clients dating back about 20 years.  Clients sell
properties.  Properties get subdivided.  New clients come and see us, and
often we don't realise we have GIS info on file for that property from many
years ago, usually under a different name.  It would be handy to see a
polygon on the screen and easily go back to some old GIS data.

 

Maybe I am missing an easy tool to do this, but I can't see anything
obvious.

 

 

Cheers

Tony

 


--

Tony Shepherd

  GeoSpatial Manager @ FarmMaps NZ & Photographer @ Shepherd Photos

 

Phone - 027 435 6193  | Website -  
shepherdphotos.co.nz

Email - Maps   t...@farmmaps.nz  |  Email - Photos
 t...@shepherdphotos.co.nz

Facebook   TonyShepherdPhotos

 

 



 

___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user