Re: [Qgis-user] Using Processing for nightly ETL jobs

2024-05-22 Thread Germán Carrillo via QGIS-User 
Hi All,

have a look at:

https://github.com/gacarrillor/AppendFeaturesToLayer

It has ready-to-use models to perform transform and upsert (update or
insert) operations.

You can run it via standalone processing (Python) scripts or via QGIS
Process.

Regards,

Germán


El mié, 22 may 2024 a las 12:20, Etienne Trimaille via QGIS-User (<
qgis-user@lists.osgeo.org>) escribió:

> Hi,
>
> https://docs.qgis.org/3.34/en/docs/user_manual/processing/standalone.html
>
> Le mer. 22 mai 2024 à 18:52, NOSPAM via QGIS-User <
> qgis-user@lists.osgeo.org> a écrit :
>
>> Dear List,
>>
>> has anybody already used QGIS' processing framework for performing
>> nightly ETL jobs?
>>
>> I need to have nightly jobs run that
>> 1) download data (e.g. from a WFS server),
>> 2) perform some tasks like renaming fields or the like
>> 3) and finally load them into a PostGIS database.
>>
>> My idea would be to design the process using the modeller. How can I
>> have the thus created model run non interactively?
>>
>> Any ideas are welcome
>>
>> Bernhard
>> ___
>> QGIS-User mailing list
>> QGIS-User@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
> ___
> QGIS-User mailing list
> QGIS-User@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>


-- 
---
   |\__
(:>__)(
   |/
Soluciones Geoinformáticas Libres
http://geotux.tuxfamily.org/
https://twitter.com/GeoTux2 
___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Pyqgis how to append existing gpkg layer

2023-04-26 Thread Germán Carrillo via QGIS-User 
Hi all,

that's right Andrea. It actually depends on whether you want to append
features to an existing layer, or to "append" (one could also say "create")
a layer (in)to an existing GPKG database. The latter case is well covered
by Anne's answer.

See prior messages in this thread:
https://lists.osgeo.org/pipermail/qgis-user/2023-April/052855.html

Regards,

Germán


El mié, 26 abr 2023 a las 9:11, andrea antonello via QGIS-User (<
qgis-user@lists.osgeo.org>) escribió:

> Hi, from what I saw in the docs, you should be able to append to layers
> with the filewriter option AppendToLayerNoNewFields?
> The CreateOrOverwriteLayer overwrites the layer completely.
>
> Cheers,
> Andrea
>
>
> On Wed, Apr 26, 2023 at 2:58 PM Anne B. Nilsen via QGIS-User <
> qgis-user@lists.osgeo.org> wrote:
>
>> I’m not a pyQGIS-expert, but with help from gis.stackechange.com I have
>> managed to create an empty geopackage file and then adding several layers
>> to it with this code:
>>
>> …
>>
>> gpkg_reg = "karplanteregistrering.gpkg"   # the NEW gpkg
>> to be created
>>
>> if os.path.exists(gpkg_reg):  # If gpkg-file
>> already exists, delete it
>>
>> os.remove(gpkg_reg)
>>
>> …
>>
>> #
>> https://gis.stackexchange.com/questions/417916/creating-empty-layers-in-a-geopackage-using-pyqgis?noredirect=1=1
>>
>> def create_blank_gpkg_layer(gpkg_path: str, layer_name: str, geometry:
>> int,
>>
>> crs: str, fields: QgsFields, append: bool =
>> False
>>
>> ) -> bool:
>>
>> # To add a layer to an existing GPKG file, pass 'append' as True
>>
>> options = QgsVectorFileWriter.SaveVectorOptions()
>>
>> options.driverName = "GPKG"
>>
>> options.layerName = layer_name
>>
>> if append:
>>
>> options.actionOnExistingFile =
>> QgsVectorFileWriter.CreateOrOverwriteLayer
>>
>> writer = QgsVectorFileWriter.create(
>>
>> gpkg_path,
>>
>> fields,
>>
>> geometry,
>>
>> QgsCoordinateReferenceSystem(crs),
>>
>> QgsCoordinateTransformContext(),
>>
>> options)
>>
>> del writer
>>
>> return True
>>
>> …
>>
>> # Layer 1 oversikt – creating an empty layer with attributes read from a
>> csv-file
>>
>> f = open("./egenskaper/egenskaper_oversikt.csv",'r') # open file
>>
>> f.readline()  # read the first line of the file (the "metadata"/heading)
>>
>> layer_name = "oversikt"  # set layer name
>>
>> print('creates layer', layer_name)   # print layer name
>>
>> geom = QgsWkbTypes.PointZ# geometrytype
>>
>> crs = 'epsg:25832'   # coordinate system
>>
>> fields = QgsFields()
>>
>> for line in f:   # Read attrubutes from csv-file
>>
>>  r = {}
>>
>>  r = line.split(';')  # 0 navn, 1 QVariant-type, 2 lengde og 3
>> presisjon
>>
>>  fname = r[0]
>>
>>  l = int(r[2])
>>
>>  p = int(r[3])
>>
>>  try:
>>
>>   fields.append(QgsField(fname,set_variant(r[1]),'',l,p))  # add
>> field to layer
>>
>>  except:
>>
>>   print("Noe gikk galt - får ikke laget egenskaper fra " +str(f))
>>
>> f.close()  # close file
>>
>> create_blank_gpkg_layer(gpkg_reg, layer_name, geom, crs, fields)  # Do
>> NOT include True when the FIRST layer is created
>>
>>
>>
>> # Layer 2 arter (to be added to the same new gpkg-file just created above)
>>
>> f = open("./egenskaper/egenskaper_arter.csv",'r') # open file
>>
>> f.readline()
>>
>> layer_name = "arter"
>>
>> print('creates layer', layer_name)   # print layer name
>>
>> geom = QgsWkbTypes.PointZ# geometrytype
>>
>> crs = 'epsg:25832'   # coordinate system
>>
>> fields = QgsFields()
>>
>> for line in f:   # Read attrubutes from csv-file
>>
>>  r = {}
>>
>>  r = line.split(';')  # 0 navn, 1 QVariant-type, 2 lengde og 3
>> presisjon
>>
>>  fname = r[0]
>>
>>  l = int(r[2])
>>
>>  p = int(r[3])
>>
>>  try:
>>
>>   fields.append(QgsField(fname,set_variant(r[1]),'',l,p))  # add
>> field to layer
>>
>>  except:
>>
>>   print("Noe gikk galt - får ikke laget egenskaper fra " +str(f))
>>
>> f.close()  # close file
>>
>> create_blank_gpkg_layer(gpkg_reg, layer_name, geom, crs, fields, True)  #
>> Include True to add a new layer to the SAME gpkg-file
>>
>> …
>>
>>
>>
>> Note the difference in the use of True and not when calling
>> create_blank_gpk_layer().
>>
>> Perhaps something similar may be the solution to your problem.
>>
>>
>>
>> Kind regards
>>
>> Anne
>>
>>
>>
>>
>> ___
>> QGIS-User mailing list
>> QGIS-User@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
> ___
> QGIS-User mailing list
> QGIS-User@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>

Re: [Qgis-user] [QGIS-Developer] New community voting member

2023-04-24 Thread Germán Carrillo via QGIS-User 
Congratulations Adelcides!
Well done!

Regards,

Germán

El lun, 24 abr 2023 a las 8:32, Peter Petrik via QGIS-Developer (<
qgis-develo...@lists.osgeo.org>) escribió:

> Hi Adelcides,
>
> welcome and wish you many good and wise decisions as the voting member :)
>
> Peter
>
>
>
>
>
> On Mon, Apr 24, 2023 at 2:44 PM Adelcides Varela via QGIS-User <
> qgis-user@lists.osgeo.org> wrote:
>
>> It is with a great pleasure and sense of recognition that I receive this
>> special honour, particularly coming from a small african island nation with
>> a population under half a million.
>>
>> I hope with the help of all the community I will be able to do a good job.
>>
>> Best regards,
>> Adelcides Varela.
>>
>> Marco Bernasocchi via QGIS-User  escreveu no
>> dia segunda, 24/04/2023 à(s) 11:22:
>>
>>> Dear community,
>>> I'd like you to join me in welcoming Adelcides Varela as our newest
>>> community voting member.
>>>
>>> Adelcides is one of the admins of the QGIS Community - Official Virtual
>>> Group (which has almost 60K members) and has been key in helping
>>> various African QGIS communities.
>>>
>>> Thanks a lot, Adelcides for all you do and welcome!
>>>
>>> Please note that Adelcides was selected without the need for a ballot
>>> since he was the only candidate proposed by the community.
>>>
>>> Cheers Marco
>>>
>>> --
>>> Marco Bernasocchi
>>>
>>> QGIS.org Chair
>>> OPENGIS.ch CEO
>>> http://berna.io
>>> ___
>>> QGIS-User mailing list
>>> QGIS-User@lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>
>> ___
>> QGIS-User mailing list
>> QGIS-User@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
> ___
> QGIS-Developer mailing list
> qgis-develo...@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>


-- 
---
   |\__
(:>__)(
   |/
Soluciones Geoinformáticas Libres
http://geotux.tuxfamily.org/
https://twitter.com/GeoTux2 


___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] [QGIS-Developer] Pyqgis how to append existing gpkg layer

2023-04-19 Thread Germán Carrillo via QGIS-User 
Bonjour Sylvain,

J'espère que vous allez bien.


If you want to append features from a source layer to another layer that
already exists in a GPKG file, you can do this:

-
options = QgsVectorFileWriter.SaveVectorOptions()
options.driverName = "GPKG"
options.layerName = "my_existing_layer_name"  *# Write here the existing
layer name in the GPKG*
options.actionOnExistingFile = QgsVectorFileWriter.AppendToLayerNoNewFields
options.onlySelectedFeatures = True

path = "/tmp/existing_gpkg_file_path.gpkg"
layer = QgsVectorLayer("/tmp/my_source_layer.shp", "my source layer name",
"ogr")

result = QgsVectorFileWriter.writeAsVectorFormatV3(
layer,
path,  *# Existing GPKG file path*
layer.transformContext(),
options)
-


If on the contrary, you want to append a whole layer (with its own field
structure, crs, etc.) to an existing GPKG database, you can follow this
solution:

https://gis.stackexchange.com/a/417950


As you can see, the key is to define the options.actionOnExistingFile
property, whose possible values are well documented here:

https://api.qgis.org/api/3.28/classQgsVectorFileWriter.html#afda86eff21ac1da7dc42cbdde424acb1


*Note: SInce we're using writeAsVectorFormatV3(), you need at least QGIS
v3.20 for this solution to work.*


Regards,

Germán



El mié, 19 abr 2023 a las 3:55, PIERRE Sylvain via QGIS-Developer (<
qgis-develo...@lists.osgeo.org>) escribió:

> Hi dev and users
>
>
>
> I don’t understand how to append an existing gpkg layer with pyqgis.
>
> I’ve tried many QgsVectorFileWriter options
>
>
>
> self.options.actionOnExistingFile =
> QgsVectorFileWriter.CreateOrOverwriteLayer
>
>
>
> reset final layer
>
>
>
> And
>
> self.options.EditionCapability =
> QgsVectorFileWriter.CanAppendToExistingLayer
>
>
>
> does not change anything
>
>
>
> So what’s the good way to do this ?
>
>
>
> This is my code (I’m iterating over several other data sources)
>
> self.options = QgsVectorFileWriter.SaveVectorOptions()
>
> self.options.driverName = 'GPKG'
>
> self.options.onlySelectedFeatures = True
>
> self.options.layerName = 'parcelles'
>
>
>
> for dir in dirs:
>
> print(dir)
>
>
>
> if os.path.isdir(dir):
>
> db_GPKG = os.path.join(dir, 'MAEC2023.gpkg')
>
> if os.path.exists(db_GPKG):
>
> gpkg_layer = db_GPKG + "|layername=parcelles"
>
> RPG_layer = QgsVectorLayer(gpkg_layer, "RPG", "ogr")
>
>
>
> expr = (' "fk_mesure" is not null ')
>
> *#expr = ('"fk_mesure" = \'{}\'').format('MAEC
> Papillons')*
>
>
>
> RPG_layer.selectByExpression(expr)
>
> selected_feature = RPG_layer.selectedFeatures()
>
> print(len(selected_feature))
>
> if os.path.exists(self.db_GPKG): *# if the ouput file
> already exist*
>
> self.options.actionOnExistingFile =
> QgsVectorFileWriter.CreateOrOverwriteLayer
>
> *#self.options.actionOnExistingFile =
> QgsVectorFileWriter.AppendToLayerNoNewFields*
>
> *#self.options.actionOnExistingFile =
> QgsVectorFileWriter.AppendToLayerAddFields *
>
> self.options.EditionCapability =
> QgsVectorFileWriter.CanAppendToExistingLayer
>
> else:
>
> self.options.actionOnExistingFile =
> QgsVectorFileWriter.CreateOrOverwriteFile
>
>
>
> write_result, error_message =
> QgsVectorFileWriter.writeAsVectorFormatV2(RPG_layer, self.db_GPKG , self.
> context, self.options)
>
>
>
>
>
>
>
> Thanks
>
>
>
>
>
> Sylvain PIERRE
>
> Chef de projet système d’information
>
> Direction des Systèmes d’Information et du Développement Numérique
>
> Service Projets et Ingénierie Numérique
>
> *Collectivité européenne d’Alsace*
>
> Tél : 03 88 76 68 88
>
> sylvain.pie...@alsace.eu
>
> www.alsace.eu
>
> [image: facebook]  [image: twitter]
>  [image: insta]
> 
>
>
> ___
> QGIS-Developer mailing list
> qgis-develo...@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>


-- 
---
   |\__
(:>__)(
   |/
Soluciones Geoinformáticas Libres
http://geotux.tuxfamily.org/
https://twitter.com/GeoTux2 


___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How to use QGIS plus-ins (QNEAT3, QChainage) in the Python custom applications?

2023-03-17 Thread Germán Carrillo via QGIS-User 
Hi Stephen,


if I understand your intention well, yes, I think it's possible, since I've
used QGIS plugins in a custom PyQGIS application back in the days (namely,
2009!).

I created a post to show off the results and to indicate the procedure
(it's in Spanish). You can find it at [1].

Perhaps you intend to use those plugins in a non-GUI application, which
should also be possible.

Accessing the underlying plugin's API, if provided, would require an
analysis per plugin, which is feasible.

For instance, QNEAT3 is a "Processing provider" plugin, which is handy for
multiple reasons. For instance, you can use it via custom PyQGIS standalone
scripts (see "Using QGIS Processing algorithms from PyQGIS standalone
scripts (outside of GUI)" at [2]) or the "qgis_process" utility [3].

On the contrary, QChainage doesn't seem to be a "Processing provider"
plugin, so further exploration would be required.

I hope that helps you get started.


Regards,

Germán
-
[1]
https://geotux.tuxfamily.org/2009/08/31/cargando-plugins-de-qgis-en-el-visor-de-pyqgis/
[2]
https://gis.stackexchange.com/questions/279874/using-qgis-processing-algorithms-from-pyqgis-standalone-scripts-outside-of-gui
[3]
https://docs.qgis.org/3.28/en/docs/user_manual/processing/standalone.html?highlight=qgis_process#using-processing-from-the-command-line


El vie, 17 mar 2023 a las 4:28, LI, Xin Stephen [MIT] via QGIS-User (<
qgis-user@lists.osgeo.org>) escribió:

> Hello all QGIS users,
>
>
>
> Is there some approaches to import the QGIS plus-ins (QNEAT3, QChainage)
> as a library or package in the Python custom applications where PyQGIS
> imported?
>
>
>
> The functions of QNEAT3 to find shortest paths are expended to be used.
> Then, the shortest paths are expected to be marked/divided by points with
> the same distances.
>
>
>
> Do you have some experience about this or any other approaches/packages
> could achieve similar functions?
>
>
>
> Stephen
>
>
> ___
> QGIS-User mailing list
> QGIS-User@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>

-- 
---
   |\__
(:>__)(
   |/
Soluciones Geoinformáticas Libres
http://geotux.tuxfamily.org/
https://twitter.com/GeoTux2 


___
QGIS-User mailing list
QGIS-User@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] Snap existing geometry to other layer

2022-04-08 Thread Germán Carrillo via Qgis-user 
Hi,

also make sure you use QGIS 3.22.x or above, since the algorithm "Snap
Geometries to Layer" has been optimized just before that version.

Details at:
https://github.com/qgis/QGIS/pull/42781
https://github.com/qgis/QGIS/pull/44766

Regards,

Germán


El vie, 8 abr 2022 a las 9:55, Nicolas Cadieux via Qgis-user (<
qgis-user@lists.osgeo.org>) escribió:

> Backup backup backup!
>
> The snap geometries to layer works very well as Alexandre stated.
>
> Nicolas Cadieux
> https://gitlab.com/njacadieux
>
> Le 8 avr. 2022 à 07:12, Alexandre Neto via Qgis-user <
> qgis-user@lists.osgeo.org> a écrit :
>
> 
> Hi Uwe,
>
> The snap geometries to layer tool in processing toolbox can be used in
> in-place feature editing, which does the changes without creating a new
> layer.
>
>
> https://docs.qgis.org/3.22/en/docs/user_manual/processing_algs/qgis/vectorgeometry.html#snap-geometries-to-layer
>
>
> https://docs.qgis.org/3.22/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html?highlight=place#the-processing-in-place-layer-modifier
>
> Best regards
>
> Alexandre Neto
> User support
> www.qcooperative.net
>
> Uwe Fischer via Qgis-user  escreveu no dia
> sexta, 8/04/2022 à(s) 11:34:
>
>> Hello,
>>
>>
>>
>> I need to snap individual features to another layer. It’s not new
>> features but existing ones. And, the process should not create a new
>> shapefile but it should modify existing features that were selected
>> beforehand for snapping.
>>
>>
>>
>> Google told me that there is a Plugin called „Geometry Snapper“ but I
>> could not find in for QGIS 3.22 LRT.
>>
>>
>>
>> Any ideas? Thank you so much!
>>
>>
>>
>> Regards, Uwe
>>
>>
>> ___
>> Qgis-user mailing list
>> Qgis-user@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>


-- 
---
   |\__
(:>__)(
   |/
Soluciones Geoinformáticas Libres
http://geotux.tuxfamily.org/
https://twitter.com/GeoTux2 


___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user