Re: [QGIS-Developer] Status of transaction support in Geopackages

[Régis Haubourg]

Hi Luigi,
I think you summed up things right.

For the sake of the thread readability, I re-post Mark's answer that for
some reason was not considered a response to this thread by the mailman
list.

--

Feb 27, 2018; 1:47pm

>> Do you have a link to the issues? Does it ignore them or worse?

The original report was 3 years ago and is still active:

https://issues.qgis.org/issues/12479

It was also a topic of a QGIS Grant Applications - August 2016
'Correction of QgsOgrProvider implementaion of GDAL 2.0'

A pull request was offered about Oct/Nov 2016 for Qgis 2.18 so that the
corrected code would be used for the start of Qgis 3.0 - but refused
because it was considered an api change.

Around April 2017 I saw that in Qgis 3.0 had still not been corrected,
despite the many changes that had been made.

Up to then I had been using a gdal version adapted to support writable
SpatialViews, that had not been accepted by gdal. But due to the evolvement
of gdal was becoming to difficult to maintain.

Around June I started work on the Spatialite-Provider, with consultations
with Alessandro Furieri (Sandro) of the Spatialite project about the future
needs, of what is now being termed, as Spatialite 5.0.

>> Having you all on the same code base is a neat solution I think.
Not when the 1 Provider (Ogr) will only offer partial results.
Writable SpatialView has existed since Spatialite 2.4 (we now have 4.3) and
there is no sign that Ogr will support this in the near future.

With Spatialite 5.0, where the world of Vector/Rasters, with Se/Sld Styles
are being brought togeather, this will become more difficult to deal with
with the separated gdal/ogr providers for all aspects of what can be done.

>> the amount of work to review the massive PRs was a limiting factor to
have that merged.
>> Do you have plans to ease that review process?
At the moment I am completing the last stage (of originally 4 stages) of
development (Database Maintenance, adding/renaming of columns etc.).

I have also started work on a Pdf which should help as an orientation as to
what the changes are and the reasons why.
One point I wrote this morning is: 'The Spatialite 5.0 Layout is similar in
nature to WMS/WFS 'getCapabilities', combining Vector, Raster and
Se/Sld-Styles.'

One goal is that such a Pdf should assist any reviewer who is willing to
work through this.

A major problem is also, that since Spatialite 5 has not yet been release,
it is difficult to understand why these massive changes are needed. So the
reluctance it is no surprise to me.

But the facts remain:
a) The present Spatialite-Provider is better than the Ogr support for
Spatialite
b) The present Spatialite-Provider will not be able deal will with what
Spatialite 5 offers

Although I myself am not a co-developer of Spatialite (more like a
collaborator), I intend to keep this up to-date with future developments.
It is also likely that Sandro will take a closer look after the present
development phase is over.

Mark Johnson, Berlin Germany




-

2018-02-28 11:30 GMT+01:00 Luigi Pirelli :

> I agree about GDLA/OGR, and are the positions already expressed in the
> Mark's PR
>
> but take into account that:
>
> 1) maintainer could be Mark (as for other core plugins e.g.
> MetaSearch). I agree that having only a maintainer for a complex core
> part can be dangerous for quality of the overall project => agree with
> Regis, could be a PSC decision.
> 2) The code is already there and should overpass some sqlite
> limitations not present in spatialite giving professional opportunity
> right now (but also reducing the hability to rise funds for a OGR
> solution :( )
> 3) It can be compiled or packaged optionally
> 4) Marks seems not focusing on work on GDAL/OGR
>
> btw without reviewers it's hard to have any merge
> Luigi Pirelli
>
> 
> **
> * LinkedIn: https://www.linkedin.com/in/luigipirelli
> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
> * GitHub: https://github.com/luipir
> * Mastering QGIS 2nd Edition:
> * https://www.packtpub.com/big-data-and-business-
> intelligence/mastering-qgis-second-edition
> * Hire me: http://goo.gl/BYRQKg
> 
> **
>
>
> On 27 February 2018 at 21:40, Alessandro Pasotti 
> wrote:
> > On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli 
> wrote:
> >>
> >> >
> >> > On 27/02/2018 11:12, Mark Johnson wrote:
> >>  that we get rid of the current provider and rely on GDAL only.
> >> >>
> >> >> With the 'current provider' I assume you mean the
> Spatialite-Provider.
> >> >>
> >> >> Please remember that the Spatialite-Provider was never designed to
> >> >> support GeoPackage.
> >> >>
> >> >> Please also remember that Gdal/Ogr does not support all aspects of
> >> >> Spatialite
> >> >> - writable 

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Luigi Pirelli]

I agree about GDLA/OGR, and are the positions already expressed in the Mark's PR

but take into account that:

1) maintainer could be Mark (as for other core plugins e.g.
MetaSearch). I agree that having only a maintainer for a complex core
part can be dangerous for quality of the overall project => agree with
Regis, could be a PSC decision.
2) The code is already there and should overpass some sqlite
limitations not present in spatialite giving professional opportunity
right now (but also reducing the hability to rise funds for a OGR
solution :( )
3) It can be compiled or packaged optionally
4) Marks seems not focusing on work on GDAL/OGR

btw without reviewers it's hard to have any merge
Luigi Pirelli

**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
* 
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**


On 27 February 2018 at 21:40, Alessandro Pasotti  wrote:
> On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli  wrote:
>>
>> >
>> > On 27/02/2018 11:12, Mark Johnson wrote:
>>  that we get rid of the current provider and rely on GDAL only.
>> >>
>> >> With the 'current provider' I assume you mean the Spatialite-Provider.
>> >>
>> >> Please remember that the Spatialite-Provider was never designed to
>> >> support GeoPackage.
>> >>
>> >> Please also remember that Gdal/Ogr does not support all aspects of
>> >> Spatialite
>> >> - writable SpatialViews are not supported
>> >>
>> >> The present QgsOgrProvider does not support Spatialite-Tables with more
>> >> than 1 geometry properly.
>> >
>> > Would it be possible to add these to the QgsOgrProvider, or are there
>> > some limitations ?
>>
>> Hi Hugo
>>
>> Some technical opinion are available in related PR done by Mark to
>> propose a new Spatialite provider.
>> The general opinion is to check before if it make sense to remove
>> spatialite limitations in the gdal provider to sqlite.
>> There are also opinon that the PR is actually not so simple to review,
>> for the complexity and extension. Oslandia can do it if apport more to
>> his business.
>>
>> IMHO I can't see any problem to merge it after review and have a new
>> or parallel spatialite provicer.
>>
>
> Well, I do: I think that unless there is an overwhelming technical reason to
> take a different route, QGIS should not create alternative providers where
> OGR/GDAL can do the job.
>
> The reason is both in how open source works: building wonderful applications
> on top of wonderful libraries (GDAL/OGR in this case) and in how we should
> avoid to enlarge the code base without a valid reason.
>
> The right approach in this particular case is IMHO to work with OGR/GDAL to
> add the missing features in the base libraries or to improve the existing
> QGIS providers if the problems is in them, this will prevent duplication and
> lower the maintenance efforts on the shoulders of QGIS developers.
>
>
> --
> Alessandro Pasotti
> w3:   www.itopen.it
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Régis Haubourg]

> Not sure if the PSC can help here?

Hi Andreas,
I think we can because it is not a technical decision, but a strategic one
about not dividing workforce, and how to not have such situations occur.
I think Mark (and maybe others) have made huge efforts for the new
spatialite provider, that is not merged because of some issues we could
adress at the PSC:

- explain and spread the word better of how to bring huge changes in the
codebase. ie how to get involved step by step, when to submit QEP before,
if it's tedious, come to hackfests and make meetings dedicated to that
change
- explain that open source succeeds because we always try to merge efforts
by working upstream when possible

Now we have to handle the fact that a huge serie of PR is here, and the
most active reviewers are reluctant to merge it as is.

So I think we could do something here to have contribution guidelines and
support to the reviewers.

Cheers
Régis

2018-02-28 7:40 GMT+01:00 Andreas Neumann :

> Hi all,
>
> Thanks all for the discussion around Geopackages, SpatiaLite, etc.
>
> Not sure if the PSC can help here?
>
> To me it seems like many people would prefer that we invest into OGR to
> improve the Geopackage support - right? Isn't there consensus on that
> already?
>
> So - it would be a matter of improving the geopackage support in OGR, so
> QGIS can profit from that.
>
> Who would be a good developer to ask for this? Even? Or others? Which dev
> knows about OGR, Geopackage/SQlite and transaction group support?
>
> Or would we need a small group of devs and/or other contributors to better
> specify what we need and then look for a suitable dev to implement our
> requirements?
>
> Thanks,
>
> Andreas
>
> On 2018-02-27 22:29, Régis Haubourg wrote:
>
> Thanks Alessandro and Nyall,
> that said, it means that two reviewers are probably not found of having
> those PR integrated as is. Our community is based on consensus I think,
> Could we raise that topic up to the PSC level?
> Thoughts?
> Régis
>
> 2018-02-27 22:05 GMT+01:00 Nyall Dawson :
>
>> On 28 February 2018 at 06:40, Alessandro Pasotti 
>> wrote:
>> > On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli 
>> wrote:
>> >>
>> >> >
>> >> > On 27/02/2018 11:12, Mark Johnson wrote:
>> >>  that we get rid of the current provider and rely on GDAL only.
>> >> >>
>> >> >> With the 'current provider' I assume you mean the
>> Spatialite-Provider.
>> >> >>
>> >> >> Please remember that the Spatialite-Provider was never designed to
>> >> >> support GeoPackage.
>> >> >>
>> >> >> Please also remember that Gdal/Ogr does not support all aspects of
>> >> >> Spatialite
>> >> >> - writable SpatialViews are not supported
>> >> >>
>> >> >> The present QgsOgrProvider does not support Spatialite-Tables with
>> more
>> >> >> than 1 geometry properly.
>> >> >
>> >> > Would it be possible to add these to the QgsOgrProvider, or are there
>> >> > some limitations ?
>> >>
>> >> Hi Hugo
>> >>
>> >> Some technical opinion are available in related PR done by Mark to
>> >> propose a new Spatialite provider.
>> >> The general opinion is to check before if it make sense to remove
>> >> spatialite limitations in the gdal provider to sqlite.
>> >> There are also opinon that the PR is actually not so simple to review,
>> >> for the complexity and extension. Oslandia can do it if apport more to
>> >> his business.
>> >>
>> >> IMHO I can't see any problem to merge it after review and have a new
>> >> or parallel spatialite provicer.
>> >>
>> >
>> > Well, I do: I think that unless there is an overwhelming technical
>> reason to
>> > take a different route, QGIS should not create alternative providers
>> where
>> > OGR/GDAL can do the job.
>> +1.
>>
>> I strongly believe it would be a dangerous mistake for the QGIS
>> project to invest further development time and maintenance burden by
>> extending the spatialite provider, and I've made that view clear on
>> every discussion related to the spatialite provider over the 3.0
>> development cycle.
>>
>> > The reason is both in how open source works: building wonderful
>> applications
>> > on top of wonderful libraries (GDAL/OGR in this case) and in how we
>> should
>> > avoid to enlarge the code base without a valid reason.
>> >
>> > The right approach in this particular case is IMHO to work with
>> OGR/GDAL to
>> > add the missing features in the base libraries or to improve the
>> existing
>> > QGIS providers if the problems is in them, this will prevent
>> duplication and
>> > lower the maintenance efforts on the shoulders of QGIS developers.
>>
>> Alessandro has summed up my thoughts exactly.
>>
>> Nyall
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>
> 

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Andreas Neumann]

Hi all, 

Thanks all for the discussion around Geopackages, SpatiaLite, etc. 

Not sure if the PSC can help here? 

To me it seems like many people would prefer that we invest into OGR to
improve the Geopackage support - right? Isn't there consensus on that
already? 

So - it would be a matter of improving the geopackage support in OGR, so
QGIS can profit from that. 

Who would be a good developer to ask for this? Even? Or others? Which
dev knows about OGR, Geopackage/SQlite and transaction group support? 

Or would we need a small group of devs and/or other contributors to
better specify what we need and then look for a suitable dev to
implement our requirements? 

Thanks, 

Andreas 

On 2018-02-27 22:29, Régis Haubourg wrote:

> Thanks Alessandro and Nyall, that said, it means that two reviewers are 
> probably not found of having those PR integrated as is. Our community is 
> based on consensus I think, Could we raise that topic up to the PSC level? 
> Thoughts? Régis 
> 
> 2018-02-27 22:05 GMT+01:00 Nyall Dawson :
> 
>> On 28 February 2018 at 06:40, Alessandro Pasotti  wrote:
>>> On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli  wrote:
 
> 
> On 27/02/2018 11:12, Mark Johnson wrote:
 that we get rid of the current provider and rely on GDAL only.
>> 
>> With the 'current provider' I assume you mean the Spatialite-Provider.
>> 
>> Please remember that the Spatialite-Provider was never designed to
>> support GeoPackage.
>> 
>> Please also remember that Gdal/Ogr does not support all aspects of
>> Spatialite
>> - writable SpatialViews are not supported
>> 
>> The present QgsOgrProvider does not support Spatialite-Tables with more
>> than 1 geometry properly.
> 
> Would it be possible to add these to the QgsOgrProvider, or are there
> some limitations ?
 
 Hi Hugo
 
 Some technical opinion are available in related PR done by Mark to
 propose a new Spatialite provider.
 The general opinion is to check before if it make sense to remove
 spatialite limitations in the gdal provider to sqlite.
 There are also opinon that the PR is actually not so simple to review,
 for the complexity and extension. Oslandia can do it if apport more to
 his business.
 
 IMHO I can't see any problem to merge it after review and have a new
 or parallel spatialite provicer.
 
>>> 
>>> Well, I do: I think that unless there is an overwhelming technical reason to
>>> take a different route, QGIS should not create alternative providers where
>>> OGR/GDAL can do the job.
>> +1.
>> 
>> I strongly believe it would be a dangerous mistake for the QGIS
>> project to invest further development time and maintenance burden by
>> extending the spatialite provider, and I've made that view clear on
>> every discussion related to the spatialite provider over the 3.0
>> development cycle.
>> 
>>> The reason is both in how open source works: building wonderful applications
>>> on top of wonderful libraries (GDAL/OGR in this case) and in how we should
>>> avoid to enlarge the code base without a valid reason.
>>> 
>>> The right approach in this particular case is IMHO to work with OGR/GDAL to
>>> add the missing features in the base libraries or to improve the existing
>>> QGIS providers if the problems is in them, this will prevent duplication and
>>> lower the maintenance efforts on the shoulders of QGIS developers.
>> 
>> Alessandro has summed up my thoughts exactly.
>> 
>> Nyall
>> 
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
> 
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

 

Links:
--
[1] https://lists.osgeo.org/mailman/listinfo/qgis-developer___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Tim Sutton]

Hi

I also think as generally policy supporting OGR is a better way to go where 
possible.

Regards

Tim

> On 27 Feb 2018, at 23:29, Régis Haubourg  wrote:
> 
> Thanks Alessandro and Nyall,
> that said, it means that two reviewers are probably not found of having those 
> PR integrated as is. Our community is based on consensus I think, Could we 
> raise that topic up to the PSC level?
> Thoughts?
> Régis
> 
> 2018-02-27 22:05 GMT+01:00 Nyall Dawson  >:
> On 28 February 2018 at 06:40, Alessandro Pasotti  > wrote:
> > On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli  > > wrote:
> >>
> >> >
> >> > On 27/02/2018 11:12, Mark Johnson wrote:
> >>  that we get rid of the current provider and rely on GDAL only.
> >> >>
> >> >> With the 'current provider' I assume you mean the Spatialite-Provider.
> >> >>
> >> >> Please remember that the Spatialite-Provider was never designed to
> >> >> support GeoPackage.
> >> >>
> >> >> Please also remember that Gdal/Ogr does not support all aspects of
> >> >> Spatialite
> >> >> - writable SpatialViews are not supported
> >> >>
> >> >> The present QgsOgrProvider does not support Spatialite-Tables with more
> >> >> than 1 geometry properly.
> >> >
> >> > Would it be possible to add these to the QgsOgrProvider, or are there
> >> > some limitations ?
> >>
> >> Hi Hugo
> >>
> >> Some technical opinion are available in related PR done by Mark to
> >> propose a new Spatialite provider.
> >> The general opinion is to check before if it make sense to remove
> >> spatialite limitations in the gdal provider to sqlite.
> >> There are also opinon that the PR is actually not so simple to review,
> >> for the complexity and extension. Oslandia can do it if apport more to
> >> his business.
> >>
> >> IMHO I can't see any problem to merge it after review and have a new
> >> or parallel spatialite provicer.
> >>
> >
> > Well, I do: I think that unless there is an overwhelming technical reason to
> > take a different route, QGIS should not create alternative providers where
> > OGR/GDAL can do the job.
> 
> +1.
> 
> I strongly believe it would be a dangerous mistake for the QGIS
> project to invest further development time and maintenance burden by
> extending the spatialite provider, and I've made that view clear on
> every discussion related to the spatialite provider over the 3.0
> development cycle.
> 
> > The reason is both in how open source works: building wonderful applications
> > on top of wonderful libraries (GDAL/OGR in this case) and in how we should
> > avoid to enlarge the code base without a valid reason.
> >
> > The right approach in this particular case is IMHO to work with OGR/GDAL to
> > add the missing features in the base libraries or to improve the existing
> > QGIS providers if the problems is in them, this will prevent duplication and
> > lower the maintenance efforts on the shoulders of QGIS developers.
> 
> Alessandro has summed up my thoughts exactly.
> 
> Nyall
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org 
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
> 
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer 
> 
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

—







Tim Sutton

Co-founder: Kartoza
Project chair: QGIS.org

Visit http://kartoza.com  to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux
IRC: timlinux on #qgis at freenode.net



signature.asc
Description: Message signed with OpenPGP
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Régis Haubourg]

Thanks Alessandro and Nyall,
that said, it means that two reviewers are probably not found of having
those PR integrated as is. Our community is based on consensus I think,
Could we raise that topic up to the PSC level?
Thoughts?
Régis

2018-02-27 22:05 GMT+01:00 Nyall Dawson :

> On 28 February 2018 at 06:40, Alessandro Pasotti 
> wrote:
> > On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli 
> wrote:
> >>
> >> >
> >> > On 27/02/2018 11:12, Mark Johnson wrote:
> >>  that we get rid of the current provider and rely on GDAL only.
> >> >>
> >> >> With the 'current provider' I assume you mean the
> Spatialite-Provider.
> >> >>
> >> >> Please remember that the Spatialite-Provider was never designed to
> >> >> support GeoPackage.
> >> >>
> >> >> Please also remember that Gdal/Ogr does not support all aspects of
> >> >> Spatialite
> >> >> - writable SpatialViews are not supported
> >> >>
> >> >> The present QgsOgrProvider does not support Spatialite-Tables with
> more
> >> >> than 1 geometry properly.
> >> >
> >> > Would it be possible to add these to the QgsOgrProvider, or are there
> >> > some limitations ?
> >>
> >> Hi Hugo
> >>
> >> Some technical opinion are available in related PR done by Mark to
> >> propose a new Spatialite provider.
> >> The general opinion is to check before if it make sense to remove
> >> spatialite limitations in the gdal provider to sqlite.
> >> There are also opinon that the PR is actually not so simple to review,
> >> for the complexity and extension. Oslandia can do it if apport more to
> >> his business.
> >>
> >> IMHO I can't see any problem to merge it after review and have a new
> >> or parallel spatialite provicer.
> >>
> >
> > Well, I do: I think that unless there is an overwhelming technical
> reason to
> > take a different route, QGIS should not create alternative providers
> where
> > OGR/GDAL can do the job.
>
> +1.
>
> I strongly believe it would be a dangerous mistake for the QGIS
> project to invest further development time and maintenance burden by
> extending the spatialite provider, and I've made that view clear on
> every discussion related to the spatialite provider over the 3.0
> development cycle.
>
> > The reason is both in how open source works: building wonderful
> applications
> > on top of wonderful libraries (GDAL/OGR in this case) and in how we
> should
> > avoid to enlarge the code base without a valid reason.
> >
> > The right approach in this particular case is IMHO to work with OGR/GDAL
> to
> > add the missing features in the base libraries or to improve the existing
> > QGIS providers if the problems is in them, this will prevent duplication
> and
> > lower the maintenance efforts on the shoulders of QGIS developers.
>
> Alessandro has summed up my thoughts exactly.
>
> Nyall
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Nyall Dawson]

On 28 February 2018 at 06:40, Alessandro Pasotti  wrote:
> On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli  wrote:
>>
>> >
>> > On 27/02/2018 11:12, Mark Johnson wrote:
>>  that we get rid of the current provider and rely on GDAL only.
>> >>
>> >> With the 'current provider' I assume you mean the Spatialite-Provider.
>> >>
>> >> Please remember that the Spatialite-Provider was never designed to
>> >> support GeoPackage.
>> >>
>> >> Please also remember that Gdal/Ogr does not support all aspects of
>> >> Spatialite
>> >> - writable SpatialViews are not supported
>> >>
>> >> The present QgsOgrProvider does not support Spatialite-Tables with more
>> >> than 1 geometry properly.
>> >
>> > Would it be possible to add these to the QgsOgrProvider, or are there
>> > some limitations ?
>>
>> Hi Hugo
>>
>> Some technical opinion are available in related PR done by Mark to
>> propose a new Spatialite provider.
>> The general opinion is to check before if it make sense to remove
>> spatialite limitations in the gdal provider to sqlite.
>> There are also opinon that the PR is actually not so simple to review,
>> for the complexity and extension. Oslandia can do it if apport more to
>> his business.
>>
>> IMHO I can't see any problem to merge it after review and have a new
>> or parallel spatialite provicer.
>>
>
> Well, I do: I think that unless there is an overwhelming technical reason to
> take a different route, QGIS should not create alternative providers where
> OGR/GDAL can do the job.

+1.

I strongly believe it would be a dangerous mistake for the QGIS
project to invest further development time and maintenance burden by
extending the spatialite provider, and I've made that view clear on
every discussion related to the spatialite provider over the 3.0
development cycle.

> The reason is both in how open source works: building wonderful applications
> on top of wonderful libraries (GDAL/OGR in this case) and in how we should
> avoid to enlarge the code base without a valid reason.
>
> The right approach in this particular case is IMHO to work with OGR/GDAL to
> add the missing features in the base libraries or to improve the existing
> QGIS providers if the problems is in them, this will prevent duplication and
> lower the maintenance efforts on the shoulders of QGIS developers.

Alessandro has summed up my thoughts exactly.

Nyall
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Alessandro Pasotti]

On Tue, Feb 27, 2018 at 12:21 PM, Luigi Pirelli  wrote:

> >
> > On 27/02/2018 11:12, Mark Johnson wrote:
>  that we get rid of the current provider and rely on GDAL only.
> >>
> >> With the 'current provider' I assume you mean the Spatialite-Provider.
> >>
> >> Please remember that the Spatialite-Provider was never designed to
> >> support GeoPackage.
> >>
> >> Please also remember that Gdal/Ogr does not support all aspects of
> >> Spatialite
> >> - writable SpatialViews are not supported
> >>
> >> The present QgsOgrProvider does not support Spatialite-Tables with more
> >> than 1 geometry properly.
> >
> > Would it be possible to add these to the QgsOgrProvider, or are there
> > some limitations ?
>
> Hi Hugo
>
> Some technical opinion are available in related PR done by Mark to
> propose a new Spatialite provider.
> The general opinion is to check before if it make sense to remove
> spatialite limitations in the gdal provider to sqlite.
> There are also opinon that the PR is actually not so simple to review,
> for the complexity and extension. Oslandia can do it if apport more to
> his business.
>
> IMHO I can't see any problem to merge it after review and have a new
> or parallel spatialite provicer.
>
>
Well, I do: I think that unless there is an overwhelming technical reason
to take a different route, QGIS should not create alternative providers
where OGR/GDAL can do the job.

The reason is both in how open source works: building wonderful
applications on top of wonderful libraries (GDAL/OGR in this case) and in
how we should avoid to enlarge the code base without a valid reason.

The right approach in this particular case is IMHO to work with OGR/GDAL to
add the missing features in the base libraries or to improve the existing
QGIS providers if the problems is in them, this will prevent duplication
and lower the maintenance efforts on the shoulders of QGIS developers.


-- 
Alessandro Pasotti
w3:   www.itopen.it
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Luigi Pirelli]

>
> On 27/02/2018 11:12, Mark Johnson wrote:
 that we get rid of the current provider and rely on GDAL only.
>>
>> With the 'current provider' I assume you mean the Spatialite-Provider.
>>
>> Please remember that the Spatialite-Provider was never designed to
>> support GeoPackage.
>>
>> Please also remember that Gdal/Ogr does not support all aspects of
>> Spatialite
>> - writable SpatialViews are not supported
>>
>> The present QgsOgrProvider does not support Spatialite-Tables with more
>> than 1 geometry properly.
>
> Would it be possible to add these to the QgsOgrProvider, or are there
> some limitations ?

Hi Hugo

Some technical opinion are available in related PR done by Mark to
propose a new Spatialite provider.
The general opinion is to check before if it make sense to remove
spatialite limitations in the gdal provider to sqlite.
There are also opinon that the PR is actually not so simple to review,
for the complexity and extension. Oslandia can do it if apport more to
his business.

IMHO I can't see any problem to merge it after review and have a new
or parallel spatialite provicer.

In any case, the Oslandia work related with custom osgeo4w packaging
(see madeira hackmeeting presentations in the event wiki) can simplify
the need to create a custom package with a packaged experimental
spatialite provier. This was the intent of the "extra" folder if I
well remember.
I invite Mark to give it a look.

Luigi Pirelli

**
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
* 
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Régis Haubourg]

2018-02-27 11:12 GMT+01:00 Mark Johnson :

> >> that we get rid of the current provider and rely on GDAL only.
>
> With the 'current provider' I assume you mean the Spatialite-Provider.
>
> Please remember that the Spatialite-Provider was never designed to support
> GeoPackage.
>

Hi Mark, I didn't say that. My suggestion was to activate transaction
groups for all DB providers allowing that. And spatialite would be one of
them, aside from geopackage-gdal.


>
> Please also remember that Gdal/Ogr does not support all aspects of
> Spatialite
> - writable SpatialViews are not supported
>
> The present QgsOgrProvider does not support Spatialite-Tables with more
> than 1 geometry properly.
>

Do you have a link to the issues? Does it ignore them or worse?


>
> 'that we get rid of the current provider' would then mean for me (and no
> doubt other) that Qgis would become fairly useless.
>

mmmh, you already know of the previous discussions and the issue of having
overlapping providers, which mean both don't receive as much love as they
deserve and users get in confusion having two ways of accessing the same
datasource. The same happened for shapefile provider, and still happens for
delimited file. The cleaner option is always to merge effort in Open Source
model, when it's possible. Why not fix GDAL limitations, so that all other
clients will benefit from those improvements?


>
> That the present Spatialite-Provider is sadly underdeveloped and in no way
> will be able to deal with the upcoming Spatialite 5.0 changes is well known.
>

My point above. In my knowledge, less than 3-4 dev are involved in that
area, and are working on two different providers. Having you all on the
same code base is a neat solution I think.


>
> A new Spatialite-Provider is being actively worked on and is earmarked for
> Qgis 3.2, that will deal with the present problems as well as supporting
> the extra functionality of Spatialite 5.0.
>

Ok good to know, and let's make that happen if the work is mostly done.
In my recording, the amount of work to review the massive PRs was a
limiting factor to have that merged. And the QGIS reviewing process is
still remaining on unpaied volunteer work. Do you have plans to ease that
review process?

Best regards,
Régis Haubourg


>
> Mark Johnson, Berlin Germany
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Hugo Mercier]

Hi Mark,

On 27/02/2018 11:12, Mark Johnson wrote:
>>> that we get rid of the current provider and rely on GDAL only.
> 
> With the 'current provider' I assume you mean the Spatialite-Provider.
> 
> Please remember that the Spatialite-Provider was never designed to
> support GeoPackage.
> 
> Please also remember that Gdal/Ogr does not support all aspects of
> Spatialite
> - writable SpatialViews are not supported
> 
> The present QgsOgrProvider does not support Spatialite-Tables with more
> than 1 geometry properly. 

Would it be possible to add these to the QgsOgrProvider, or are there
some limitations ?

> 
> A new Spatialite-Provider is being actively worked on and is earmarked
> for Qgis 3.2, that will deal with the present problems as well as
> supporting the extra functionality of Spatialite 5.0.

Nice ! Good to know.
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Mark Johnson]

>> that we get rid of the current provider and rely on GDAL only.

With the 'current provider' I assume you mean the Spatialite-Provider.

Please remember that the Spatialite-Provider was never designed to support
GeoPackage.

Please also remember that Gdal/Ogr does not support all aspects of
Spatialite
- writable SpatialViews are not supported

The present QgsOgrProvider does not support Spatialite-Tables with more
than 1 geometry properly.

'that we get rid of the current provider' would then mean for me (and no
doubt other) that Qgis would become fairly useless.

That the present Spatialite-Provider is sadly underdeveloped and in no way
will be able to deal with the upcoming Spatialite 5.0 changes is well known.

A new Spatialite-Provider is being actively worked on and is earmarked for
Qgis 3.2, that will deal with the present problems as well as supporting
the extra functionality of Spatialite 5.0.

Mark Johnson, Berlin Germany
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Alessandro Pasotti]

On Tue, Feb 27, 2018 at 8:06 AM, Régis Haubourg 
wrote:

> Hi Andreas,
> ok, that's clear. You are right, that feature is only for the postgreSQL
> provider currently.
>
> I didn't see any missing requirement in SQLITE v3 - the requirement for
> GPKG - maybe Matthias knows more about that. We also use GDAL provider
> here, so we might miss some features there and have to implement or fix
> things upstream. We should check that with Even Rouault who implemented it
> in GDAL 2.0 (see  https://trac.osgeo.org/gdal/wiki/rfc54_dataset_
> transactions)
>
> If we go that way, we should add it to spatialite too, but the state of
> the provider could make it terribly hard to do, so I would set as a
> prerequisite that we get rid of the current provider and rely on GDAL
> only.
>

Big +1 from me if we choose that route ^

That would make it easier to maintain the browser tree too and remove a lot
of code duplication.


-- 
Alessandro Pasotti
w3:   www.itopen.it
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Régis Haubourg]

Hi Andreas,
ok, that's clear. You are right, that feature is only for the postgreSQL
provider currently.

I didn't see any missing requirement in SQLITE v3 - the requirement for
GPKG - maybe Matthias knows more about that. We also use GDAL provider
here, so we might miss some features there and have to implement or fix
things upstream. We should check that with Even Rouault who implemented it
in GDAL 2.0 (see
https://trac.osgeo.org/gdal/wiki/rfc54_dataset_transactions)

If we go that way, we should add it to spatialite too, but the state of the
provider could make it terribly hard to do, so I would set as a
prerequisite that we get rid of the current provider and rely on GDAL
only.

We  also have had hard time to consolidate some transactional glitches in
the PG provider with SQL syntax errors and failing transactions, I expect
even more work with SQLITE since it is less used and polished, and also
rely on GDAL in which we probably we have

Last point to note is that at OSLANDIA we have been using a lot SQLITE -
spatiaLITE to push some business logic in it (the thick database practice).
It revealed way more complex to write and maintain than in postgreSQL
because of SQLITE limitations. So complex that it finally revealed faster
to rewrite all the logic to PG, then package an embedded / no  install
postgreSQL application which is launched as a process when opening a QGIS
project. We called that PGLITE and is one of the reasons of getting
involved in OSGEO4W packaging to easily deploy that. It is very
experimental currently however. Just to say to porting the same application
made with PG to a offline / on field identical application is not easy at
all.

So if your use case is to have simple triggers, that would do, but if you
have complex procedures, developping that is not at all a straightforward
port from PG code.

That said, I would love to have that supported in QGIS!


Regards ,
Régis



2018-02-27 8:17 GMT+01:00 Andreas Neumann :

> Hi Régis,
>
> Yes - I mean the "transaction group" support. Meaning that multiple layers
> can be edited together. Most important issue for us: ability to link
> related tables to newly created features without asking the user to
> manually save the parent table first.
>
> This all works for Postgis now, but I think not for Geopackages - right?
>
> Thanks,
>
> Andreas
>
> On 2018-02-26 22:13, Régis Haubourg wrote:
>
> Hi Andreas,
> by transaction support, you mean "transaction group" support  ? The one
> that allows to edit all the layers in one unique transaction and evaluate
> triggers on the fly?
>
> If not, I'm not sure to understand what is lacking currently for your use
> case.
>
> Nothing is planned here however.
>
>
> Regards!
> Régis
>
> 2018-02-26 13:31 GMT+01:00 Andreas Neumann :
>
>> Hi,
>>
>> We would like to see transaction support in QGIS 3x for geopackages. We
>> want to hand out geopackages and QGIS project files, for external parties
>> to edit data for us - without the need that the third-party has PostgreSQL
>> installed. The projects all use relations.
>>
>> Before we test ourselves, I thought it might make sense to ask what the
>> status or plans are for transaction support in Geopackages?
>>
>> Should it already be supported? If not - is someone planning to work on
>> it?
>>
>> Thanks for your replies,
>>
>> Andreas
>>
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Andreas Neumann]

Hi Régis, 

Yes - I mean the "transaction group" support. Meaning that multiple
layers can be edited together. Most important issue for us: ability to
link related tables to newly created features without asking the user to
manually save the parent table first. 

This all works for Postgis now, but I think not for Geopackages - right?


Thanks, 

Andreas 

On 2018-02-26 22:13, Régis Haubourg wrote:

> Hi Andreas, by transaction support, you mean "transaction group" support  ? 
> The one that allows to edit all the layers in one unique transaction and 
> evaluate triggers on the fly? 
> 
> If not, I'm not sure to understand what is lacking currently for your use 
> case. 
> 
> Nothing is planned here however. 
> 
> Regards! Régis 
> 
> 2018-02-26 13:31 GMT+01:00 Andreas Neumann :
> 
>> Hi, 
>> 
>> We would like to see transaction support in QGIS 3x for geopackages. We want 
>> to hand out geopackages and QGIS project files, for external parties to edit 
>> data for us - without the need that the third-party has PostgreSQL 
>> installed. The projects all use relations. 
>> 
>> Before we test ourselves, I thought it might make sense to ask what the 
>> status or plans are for transaction support in Geopackages? 
>> 
>> Should it already be supported? If not - is someone planning to work on it? 
>> 
>> Thanks for your replies, 
>> 
>> Andreas 
>> ___
>> QGIS-Developer mailing list
>> QGIS-Developer@lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer [1]
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer [1]

 

Links:
--
[1] https://lists.osgeo.org/mailman/listinfo/qgis-developer___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] Status of transaction support in Geopackages

[Régis Haubourg]

Hi Andreas,
by transaction support, you mean "transaction group" support  ? The one
that allows to edit all the layers in one unique transaction and evaluate
triggers on the fly?

If not, I'm not sure to understand what is lacking currently for your use
case.

Nothing is planned here however.


Regards!
Régis

2018-02-26 13:31 GMT+01:00 Andreas Neumann :

> Hi,
>
> We would like to see transaction support in QGIS 3x for geopackages. We
> want to hand out geopackages and QGIS project files, for external parties
> to edit data for us - without the need that the third-party has PostgreSQL
> installed. The projects all use relations.
>
> Before we test ourselves, I thought it might make sense to ask what the
> status or plans are for transaction support in Geopackages?
>
> Should it already be supported? If not - is someone planning to work on it?
>
> Thanks for your replies,
>
> Andreas
>
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer