Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi List, Hi Martin,

TLDR; New QgsFeatureRequest filters
- FilterExpression
- FilterFids
Available here [1]

For another project I'm currently working on, I need additional filter
possibilities in QgsFeatureRequest to filter with a QgsExpression.
As I
- did not want to repeat the code inside each iterator
- but need the possibility to override the behavior (e.g.
vectorlayerfeatureiterator only needs to filter changed features bot
does not need to refilter features already filtered by the provider)
I extended QgsAbstractFeatureIterator to handle this but allow derived
classes to override the method if they want. I introduced a small API
change such that, the nextFeature method calls
nextFeatureFilterExpression which can be overridden by subclasses. Main
filtering is done within a new method fetchFeature which gets called by
nextFeature and should not perform any filtering as long as
FilterExpression is not implemented.
Same goes for FilterFids which takes multiple featureids but is now
implemented to iterate over all features, but can easily be
reimplemented inside providers to be executed on the server side.
The change updates the API, but only for the implementation of iterators
and not for any module which uses the iterator (same nextFeature call as
before).

Anyway, before spending more time on this, I'd like to hear your opinion.

Matthias

[1] https://github.com/matthias-kuhn/Quantum-GIS/tree/request-filters

On 28.12.2012 17:03, Martin Dobias wrote:
> Hi everyone,
>
> it seems that now it is a good time to ask for a broader review of the
> work I have been doing during recent months: essentially making QGIS
> vector API more flexible and more ready for introduction of threading
> for rendering. That resulted in a greater refactoring of some parts of
> QgsVectorLayer class and QgsVectorDataProvider implementations.
> Everything is in new_vector_api branch in QGIS repository on GitHub
> [1].
>
> There are few things that are not finished, but should not take too
> much work to sort out:
> - disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated
to new API
> - disabled mapserver - there are few things to update
>
> If no serious problems will be found, I would like to merge the branch
> to master and continue working on that on master branch to avoid the
> possibility to drift (again) too much from the quick development that
> happens on master. In short term I plan to do some polishing and
> fixing bugs, then eventually start looking at the threading again.
>
> For more details about what happened in the branch, see the text below
> (long read!). A great help from early testers would be to compile the
> branch, try to do some "usual" stuff with vectors and see if things
> break apart (crashes? data corruption?).
>
> Looking forward to your comments!
>
> Regards
> Martin
>
> [1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api
>
>
>
> QGIS VECTOR API CHANGES
>
> 1. QgsFeature (changes)
>
> a) access to attributes by name. Vector data providers and vector
> layer assign pointer to the fields in QgsFeature, so it is possible to
> set/get attributes by their name. This is not as efficient as with
> direct access by index, but often this convenience is useful for
> users.
>
> b) attributes are stored in a QVector container rather than a QMap.
> The major advantage is simplification of logic related to handling of
> attributes: it's not possible to have "holes" in the array of
> fields/attributes. Currently it is possible that for example layer
> with three fields returns indexes 0,1,3 - but it is not so common nor
> obvious, so it's a common source of errors. After refactoring there
> must not be any holes, so a layer with three fields always returns
> indexes 0,1,2. When iterating over layer's features, QgsFeature always
> contains a vector of all layer's attributes. In case the client has
> not requested attributes to be fetched, the attributes contain invalid
> values.
>
>
> 2. QgsFields (new class)
>
> Just like attributes within a feature are stored in a vector instead
> of map, also layer's fields are now stored in a vector. QgsFields is a
> new class that mimics QVector API and adds two more pieces of
> functionality:
>
> a) fast lookup of field index from name. When QgsFields is populated
> with fields, it creates a map of fields that facilitates fast lookups
>
> b) detection of field origin. When working with a vector layer, it is
> sometimes useful to find out origin of the field - whether it comes
> from provider, from a join or whether it is a newly added field (not
> committed). In the future we could add also expression-based fields,
> creating a field calculator that calculates the values on the fly.
>
>
> 3. QgsFeatureRequest (new class)
>
> Class that encapsulates requests for features to a vector layer or
> vector data provider. Right now in master branch, the request is
> defined by arguments to select() method. That's not very flexible nor
> simple to use. Feature req

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi Matthias

On Wed, Apr 3, 2013 at 11:06 AM, Matthias Kuhn  wrote:
>
> According to this discussion, most people would appreciate the switch.
>
> I would like to have this change included as fast as possible, to give
> plugin developers as much time as possible to make the necessary
> changes before 2.0 is released.
>
> Martin, if you are still too busy, would it be possible to push your
> branch and create a ToDo list, so others could jump in if they find
> some time?

I will have a look at that during the weekend to make see what else is
missing - if that will be too much work, I will just put the initial
work somewhere so that others can help.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi

On Mit 03 Apr 2013 11:00:15 CEST, Martin Dobias wrote:
> Hi
>
> On Tue, Apr 2, 2013 at 3:10 PM, Matthias Kuhn  wrote:
>> Despite feature freeze, I'd like to ask what happened to that plan. As
>> far as I know, this change has not been made, and if this change is not
>> made before QGIS 2.0 it will have to wait for 3.0 (This is not something
>> to be done for 2.1).
>> It will probably be a small change (to switch in the core) but with a
>> huge impace (concerning plugins).
>
> I have a patch pending on my machine, but I have found out it needs
> more work than just running few "sip.setapi(...)" calls. In PyQt4
> QVariant API v2 there is basically no QVariant anymore. Given that we
> use QVariant.Type for things like defining QgsField types, we need to
> find a workaround (in PyQt4 they had a bit similar situation with
> QSettings).
>
> I still would like to get it into 2.0, but recently I have been too
> busy with real life to do anything with it. In the following weeks I
> should be able to find more time, so hopefully we will make a switch.
>
> Martin

According to this discussion, most people would appreciate the switch.

I would like to have this change included as fast as possible, to give 
plugin developers as much time as possible to make the necessary 
changes before 2.0 is released.

Martin, if you are still too busy, would it be possible to push your 
branch and create a ToDo list, so others could jump in if they find 
some time?

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi

On Tue, Apr 2, 2013 at 3:10 PM, Matthias Kuhn  wrote:
> Despite feature freeze, I'd like to ask what happened to that plan. As
> far as I know, this change has not been made, and if this change is not
> made before QGIS 2.0 it will have to wait for 3.0 (This is not something
> to be done for 2.1).
> It will probably be a small change (to switch in the core) but with a
> huge impace (concerning plugins).

I have a patch pending on my machine, but I have found out it needs
more work than just running few "sip.setapi(...)" calls. In PyQt4
QVariant API v2 there is basically no QVariant anymore. Given that we
use QVariant.Type for things like defining QgsField types, we need to
find a workaround (in PyQt4 they had a bit similar situation with
QSettings).

I still would like to get it into 2.0, but recently I have been too
busy with real life to do anything with it. In the following weeks I
should be able to find more time, so hopefully we will make a switch.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Ivan Mincik]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 04/02/2013 03:10 PM, Matthias Kuhn wrote:
> Hi Martin, hi List
>
> On 04.01.2013 22:19, Martin Dobias wrote:
>> Hi Matthias
>>
>> thanks for your mail.
>>
>> On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn 
wrote:
>>> You've been talking about updating to PyQt4 SIP version 2. I noticed
this
>>> change did not yet happen. What are your plans regarding this update?
>> Yes, the plans are still there. I'm only wondering what would be the
>> best time to switch - right now within the branch or a bit later once
>> the branch is merged and the new code settles down a bit? I can see a
>> slight advantage of the latter approach of having a greater chance of
>> separating problems caused by the merge and problems caused by the SIP
>> API change, but I do not have a strong opinion here.
>>
>>
> Despite feature freeze, I'd like to ask what happened to that plan. As
> far as I know, this change has not been made, and if this change is not
> made before QGIS 2.0 it will have to wait for 3.0 (This is not something
> to be done for 2.1).
> It will probably be a small change (to switch in the core) but with a
> huge impace (concerning plugins).

I would also vote to switch now when API is broken.


- -- 
Ivan Mincik
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQEcBAEBAgAGBQJRWypZAAoJEPfdLsR5UpoeXIMIAKw/IIaDGdRGlF8Q2tKpbvnR
VrjteXlFQB+u2g+X6DCgSnbQKUEdpS+HflFJjVVNcTIUMOarMx0qn8nMe8MUiSVE
C/IpKvysW4FlF/bhSdpER/yj42oVNBfzcc+eX6rfZl3qNoidNGN6U+2hvpMDKLX0
oPr6YqPUNwjj9L+MlaV70jBM3kZSPs9dBx9cvktBZeXWH0n9iJpB4vpsSgb6SwDa
CYL6IJZSwCohVdQ7pWOUnb0eQ/Te2GKx9sv22Tlh1rahr7VBWBrKCXssyrkDNiDL
ZjACCof+DxkUOGASDOpFCVbLQDWZsbC2s6eeZxlXNNswycxGch+p2idKeozPluA=
=spqX
-END PGP SIGNATURE-

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Andreas Neumann]

Hi,

+1 to switch now to the new PyQT API. We already a big break - so lets
break now (even more) and then stay stable for a while. Plugin
developers will have to rework their plugins anyway because of the new
Vector API and the switch to the new PyQT API can be done in the same step.

Andreas

Am 02.04.2013 15:38, schrieb Alexander Bruy:
> Hi,
> 
> I would vote to switch now, because later we will have much more
> plugins to update. Also have two breaks (first — new API, second
> — PyQt API update) is not good IMO.
> 
> On Tue, 02 Apr 2013 15:10:05 +0200
> Matthias Kuhn  wrote:
> 
>> Despite feature freeze, I'd like to ask what happened to that plan. As
>> far as I know, this change has not been made, and if this change is not
>> made before QGIS 2.0 it will have to wait for 3.0 (This is not something
>> to be done for 2.1).
>> It will probably be a small change (to switch in the core) but with a
>> huge impace (concerning plugins).
> 
> 

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Alexander Bruy]

Hi,

I would vote to switch now, because later we will have much more
plugins to update. Also have two breaks (first — new API, second
— PyQt API update) is not good IMO.

On Tue, 02 Apr 2013 15:10:05 +0200
Matthias Kuhn  wrote:

> Despite feature freeze, I'd like to ask what happened to that plan. As
> far as I know, this change has not been made, and if this change is not
> made before QGIS 2.0 it will have to wait for 3.0 (This is not something
> to be done for 2.1).
> It will probably be a small change (to switch in the core) but with a
> huge impace (concerning plugins).


-- 
Alexander Bruy
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Nathan Woodrow]

Yeah if we could flick it on it would make a load of difference to how
clean the code is.

- Nathan


On Tue, Apr 2, 2013 at 11:10 PM, Matthias Kuhn  wrote:

> Hi Martin, hi List
>
> On 04.01.2013 22:19, Martin Dobias wrote:
> > Hi Matthias
> >
> > thanks for your mail.
> >
> > On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn 
> wrote:
> >> You've been talking about updating to PyQt4 SIP version 2. I noticed
> this
> >> change did not yet happen. What are your plans regarding this update?
> > Yes, the plans are still there. I'm only wondering what would be the
> > best time to switch - right now within the branch or a bit later once
> > the branch is merged and the new code settles down a bit? I can see a
> > slight advantage of the latter approach of having a greater chance of
> > separating problems caused by the merge and problems caused by the SIP
> > API change, but I do not have a strong opinion here.
> >
> >
> Despite feature freeze, I'd like to ask what happened to that plan. As
> far as I know, this change has not been made, and if this change is not
> made before QGIS 2.0 it will have to wait for 3.0 (This is not something
> to be done for 2.1).
> It will probably be a small change (to switch in the core) but with a
> huge impace (concerning plugins).
>
> Matthias
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin, hi List

On 04.01.2013 22:19, Martin Dobias wrote:
> Hi Matthias
>
> thanks for your mail.
>
> On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn  wrote:
>> You've been talking about updating to PyQt4 SIP version 2. I noticed this
>> change did not yet happen. What are your plans regarding this update?
> Yes, the plans are still there. I'm only wondering what would be the
> best time to switch - right now within the branch or a bit later once
> the branch is merged and the new code settles down a bit? I can see a
> slight advantage of the latter approach of having a greater chance of
> separating problems caused by the merge and problems caused by the SIP
> API change, but I do not have a strong opinion here.
>
>
Despite feature freeze, I'd like to ask what happened to that plan. As
far as I know, this change has not been made, and if this change is not
made before QGIS 2.0 it will have to wait for 3.0 (This is not something
to be done for 2.1).
It will probably be a small change (to switch in the core) but with a
huge impace (concerning plugins).

Matthias
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] new vector api select features

[Martin Dobias]

Hi Marco

On Wed, Feb 27, 2013 at 9:57 AM, Marco Bernasocchi
 wrote:
> On Wed, Feb 6, 2013 at 9:53 AM, Richard Duivenvoorde
>  wrote:
>> THANKS! for this very thorough answer! I think it would be a pity to let
>> it
>> rest here in the dev-list, we should put it somewhere in he cookbook/wiki
>> I
>
> Hi Martin and Richard,
> I'm just teaching some plugin stuff and I remembered that somewhere in the
> dev list there was a super nice email by Marting explaining everythin, it
> took me about 10-15min to find it by skimming the archives.
>
> Would it be possible for either of you to temporarly put the link to it [0]
> in the cookbook (1.8) mentioning that if working with master that is the way
> to go? I think I'm not the only one developing for master.

Instead of a just a link it would be better to have the explanation
directly in the cookbook. For 2.x release we should start another
branch of the cookbook so that people will not get confused by many
"before 2.0" and "after 2.0" sections.

I should have done that already, but I am somehow late with these
updates of documentation :-/
(unfortunately I'm faster typing in c++ than in English)
At some point I will eventually update the cookbook, but now I am
mostly focused on code updates for 2.0. Anyway I would highly welcome
any activity by others to bring the cookbook up to date.

> I'd do it myself, but I never edited the cookbook, how to do it?

The cookbook is now located in the QGIS-Documentation repository:
https://github.com/qgis/QGIS-Documentation/tree/master/source/docs/pyqgis_developer_cookbook
It is in RST format for sphinx, like the rest of the documentation.

Cheers
Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] new vector api select features

[Marco Bernasocchi]

On 02/27/2013 09:57 AM, Marco Bernasocchi wrote:

On Wed, Feb 6, 2013 at 9:53 AM, Richard Duivenvoorde
 wrote:
 > THANKS! for this very thorough answer! I think it would be a pity to
let it
 > rest here in the dev-list, we should put it somewhere in he
cookbook/wiki I

Hi Martin and Richard,
I'm just teaching some plugin stuff and I remembered that somewhere in
the dev list there was a super nice email by Marting explaining
everythin, it took me about 10-15min to find it by skimming the archives.

Would it be possible for either of you to temporarly put the link to it
[0] in the cookbook (1.8) mentioning that if working with master that is
the way to go? I think I'm not the only one developing for master.

I'd do it myself, but I never edited the cookbook, how to do it?

ciao

Marco

[0]
http://lists.osgeo.org/pipermail/qgis-developer/2013-February/024278.html


PS, I added it at least in the api changes page [0]

ciao
Marco

[0] http://hub.qgis.org/wiki/quantum-gis/API_changes_for_version_20

--
Marco Bernasocchi
http://opengis.ch
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] new vector api select features

[Marco Bernasocchi]

On Wed, Feb 6, 2013 at 9:53 AM, Richard Duivenvoorde
 wrote:
> THANKS! for this very thorough answer! I think it would be a pity to 
let it
> rest here in the dev-list, we should put it somewhere in he 
cookbook/wiki I


Hi Martin and Richard,
I'm just teaching some plugin stuff and I remembered that somewhere in 
the dev list there was a super nice email by Marting explaining 
everythin, it took me about 10-15min to find it by skimming the archives.


Would it be possible for either of you to temporarly put the link to it 
[0] in the cookbook (1.8) mentioning that if working with master that is 
the way to go? I think I'm not the only one developing for master.


I'd do it myself, but I never edited the cookbook, how to do it?

ciao

Marco

[0] 
http://lists.osgeo.org/pipermail/qgis-developer/2013-February/024278.html

--
Marco Bernasocchi
http://opengis.ch
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] new vector api select features

[Martin Dobias]

On Wed, Feb 6, 2013 at 9:53 AM, Richard Duivenvoorde
 wrote:
> THANKS! for this very thorough answer! I think it would be a pity to let it
> rest here in the dev-list, we should put it somewhere in he cookbook/wiki I
> think.
>
> Bringing me to the question: how to handle the versioning in the cookbook.
> As it is mostly your book (isn't it?) what do you prefer.
> Leave current cookbook in 'manual_en_v1.8' branch as is, and REPLACE the old
> vector select way in the 'master' branch?
> Or should we write up both ways in one cookbook?

It makes sense to branch 1.x cookbook off master and keep in master
only what is valid in 2.0 to avoid further confusion. Then we will
just need a second instance of the cookbook for online viewing. To
avoid breakage of links from other sites, cookbook for 1.x should stay
at the same URL and cookbook for 2.0 should get a new URL.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] new vector api select features

[Richard Duivenvoorde]

On 05-02-13 22:51, Martin Dobias wrote:

Hi Richard

On Tue, Feb 5, 2013 at 7:27 PM, Richard Duivenvoorde
 wrote:

Hi Devs,

I used the following in a python plugin to select features:

self.provider.select(self.provider.attributeIndexes(), mapCanvasExtent,
True, True)

is there somewhere an example how to do this in the new api using

QgsFeatureIterator QgsOgrProvider::getFeatures( const QgsFeatureRequest&
request )

and python?



I hope I haven't forgotten any important use case. If more examples
are necessary, I will try to provide them. Please do not hesitate to
ask if things are not clear yet.


Hi Martin,

THANKS! for this very thorough answer! I think it would be a pity to let 
it rest here in the dev-list, we should put it somewhere in he 
cookbook/wiki I think.


Bringing me to the question: how to handle the versioning in the 
cookbook. As it is mostly your book (isn't it?) what do you prefer.
Leave current cookbook in 'manual_en_v1.8' branch as is, and REPLACE the 
old vector select way in the 'master' branch?

Or should we write up both ways in one cookbook?

At the moment it seems to me that we are only building the manual_1.8_en 
branch (on documentation.qgis.org) (I will discuss this in a next email).


And thanks for you vector work also!!

Regards,

Richard Duivenvoorde
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] new vector api select features

[Martin Dobias]

Hi Richard

On Tue, Feb 5, 2013 at 7:27 PM, Richard Duivenvoorde
 wrote:
> Hi Devs,
>
> I used the following in a python plugin to select features:
>
> self.provider.select(self.provider.attributeIndexes(), mapCanvasExtent,
> True, True)
>
> is there somewhere an example how to do this in the new api using
>
> QgsFeatureIterator QgsOgrProvider::getFeatures( const QgsFeatureRequest&
> request )
>
> and python?

I haven't updated the examples yet, sorry. Here we go in Python:

for f in layer.getFeatures():
  print f["name"].toString(), f.geometry().exportToWkt()

This will use give you all features from a layer with geometries and
attributes. If no request is specified in the getFeatures() call, it
will use default request which fetches everything - equal to calling
getFeatures(QgsFeatureRequest()).

The getFeatures() call returns QgsFeatureIterator instance. Python
bindings of this class support python's iterator protocol [1] so you
can use it directly in a 'for' cycle as shown above. Of course it is
still possible to original C++ API style, but it does not feel very
pythonic:

f = QgsFeature()
iter = layer.getFeatures()
while iter.nextFeature(f):
  print f["name"].toString(), f.geometry().exportToWkt()

I strongly recommend the former variant - it's easier to write and
understand, so less prone to errors.

Access to attributes: there is no f.attributeMap() anymore, because
attributes are now stored in a vector (Python: list) instead of a map
(Python: dict). QgsFeature class emulates python container object [2]
so you can access attributes as if QgsFeature instance was a list or
dictionary, with keys being either field indices or field names:
f[0]  ... first attribute
f["type"]  ... attribute named "type"
It is still possible to get all attributes: f.attributes() returns a
list of values.

Attribute values are returned as QVariant's as before, so in order to
access a string attribute you need to do something like
f["name"].toString(), for integers it is the cryptic
f["speed"].toInt()[0]. At some point (before the release of 2.0) I
would like to switch QGIS to use newer PyQt4 API that does automatic
conversion of QVariant to python objects. This will cause that
f["name"] or f["speed"] will directly return a string or int without
the need to call toString()/toInt() methods).

Currently we have three types of requests: 1. no filter (default), 2.
filter by rectangle, 3. filter by feature ID.

Fetching of features within a rectangle is done this way:
rect = QgsRectangle(-10,-10,10,10)
request = QgsFeatureRequest().setFilterRect(rect)
for f in layer.getFeatures(request):
  # usual stuff here with the feature

For the sake of processing speed, some providers may return features
that are not in the rectangle (only their bounding box intersects the
rectangle). Usually this is not a problem (e.g. for rendering), but
for some cases such as identification of features in particular
rectangle this is not wanted. In old API, setting fourth argument of
select() to true returned only features that really intersected the
rectangle. In new API this is done with a flag:
request.setFlags(QgsFeatureRequest.ExactIntersect)

Another type of requests are requests for a particular feature ID. In
these cases we expect just one feature so we could directly fetch just
one feature from the iterator:
request = QgsFeatureRequest().setFilterFid(14)
f = l.getFeatures(request).next()

In case feature with given ID does not exist, python exception
StopIteration is raised.

If developers would like to optimize the requests to avoid fetching
things that are not required, it is possible to:
- avoid fetching of geometry: request.setFlags(QgsFeatureRequest.NoGeometry)
- fetch only some (or no) attributes - e.g. fetch just first two
attributes: request.setSubsetOfAttributes([0,1])

Even though API allows that, it is currently not possible to have
concurrent feature iterators on one layer or data provider, something
like this:
iter1 = layer.getFeatures()
iter2. = layer.getFeatures()
# do something with both iter1 and iter2
This will NOT work - in the second getFeatures() call the iter1 will
get closed and it will not return any more features. In the future
some providers will support concurrent iterators, there will be
probably another provider capability that would determine if a
provider supports that.

I hope I haven't forgotten any important use case. If more examples
are necessary, I will try to provide them. Please do not hesitate to
ask if things are not clear yet.

Regards
Martin

[1] http://docs.python.org/2/library/stdtypes.html#generator-types
[2] http://docs.python.org/2/reference/datamodel.html#object.__getitem__
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] new vector api select features

[Richard Duivenvoorde]

Hi Devs,

I used the following in a python plugin to select features:

self.provider.select(self.provider.attributeIndexes(), mapCanvasExtent, 
True, True)


is there somewhere an example how to do this in the new api using

QgsFeatureIterator QgsOgrProvider::getFeatures( const QgsFeatureRequest& 
request )


and python?

If provided, I could add the info at: 
http://documentation.qgis.org/html/en/docs/pyqgis_developer_cookbook/03_vector.html#iterating-over-vector-layer


(or maybe someone can update that pages instead of replying :-) )

Regards,

Richard Duivenvoorde
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi Larry

On Wed, Jan 23, 2013 at 11:52 PM, Larry Shaffer  wrote:
> Hi Martin,
>
> What do you think about tagging the repository, right before merging your
> branch? That way there would be a reference point for building a pre-merge
> state, useful for nightly build maintainers who can offer an additional
> download of that tagged, semi-stable build.

No problem to tag master branch before the merge.

Regards
Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Larry Shaffer]

Hi Martin,

What do you think about tagging the repository, right before merging your
branch? That way there would be a reference point for building a pre-merge
state, useful for nightly build maintainers who can offer an additional
download of that tagged, semi-stable build.

Currently tagging seems to only be done for releases:
https://github.com/qgis/Quantum-GIS/tags

Larry


On Wed, Jan 23, 2013 at 3:37 PM, Martin Dobias  wrote:

> Hi Matthias
>
> On Mon, Jan 21, 2013 at 10:17 AM, Matthias Kuhn 
> wrote:
> > Martin,
> >
> > Works for me. My plugin is ready for the new vector API then. Thank you
> for
> > your work.
> >
> > What are your plans for merging?
>
> I will be merging new vector API in next few days.
>
> Martin
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi Matthias

On Mon, Jan 21, 2013 at 10:17 AM, Matthias Kuhn  wrote:
> Martin,
>
> Works for me. My plugin is ready for the new vector API then. Thank you for
> your work.
>
> What are your plans for merging?

I will be merging new vector API in next few days.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Martin,

Works for me. My plugin is ready for the new vector API then. Thank you 
for your work.


What are your plans for merging?

Regards,
Matthias

On 01/17/2013 12:18 AM, Martin Dobias wrote:

On Tue, Jan 8, 2013 at 9:46 AM, Matthias Kuhn  wrote:

Layers are rendered correctly again.

However, dataProvider.getFeatures() doesn't return any features in my plugin
( the same instructions used to work before your last two commits ).

Matthias,
my recent update to feature iterators should fix this and some more issues, see:

https://github.com/qgis/Quantum-GIS/commit/a6c5fd875bc73e4775b825640a7cedcc2f5a8832

The default behavior for providers is now that they close any previous
active iterator. Later I will be adding a new provider capability
where the provider could state that it supports concurrent iterators.

Regards
Martin


___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin,

see answers below

On 01/16/2013 10:13 PM, Martin Dobias wrote:

Hi Matthias

On Tue, Jan 15, 2013 at 11:46 PM, Matthias Kuhn  wrote:

Hi Martin,

This mail concerns the vectorlayercache outlined in section 8) of your mail.

I'm currently working on the attribute table which has some internal caching
already. As I'm currently rewriting the attribute table, I moved this code
outside and generalized it as good as possible. This work is currently
available in my branch vectorlayercache [0]. Do you think this could be used
instead of your geometry cache? It should be able to cache geometries of
features (although I only tested attributes so far) and update and
invalidate them if required.

I have also the idea to create a QgsVectorDataProviderCacheProxy class which
inherits QgsVectorDataProvider and which you can use as a proxy between your
app and the DataProvider. I've some unfinished code here which provides such
functionality. But for now, the cache in front of the VectorLayer should
already help a lot.

Just out of curiosity, what are you rewriting in the attribute table -
is it just the cache or does it involve more changes?


Have a look at the dual view discussion on this list. It's explained 
more there.


Your idea of having a cache proxy vector data provider is interesting.
Such transparent approach would enable us to remove caching from
vector data provider implementations (e.g. wfs) and transparently add
caching to the ones where caching would be useful (e.g.
delimitedtext). I need to give it some more thoughts what impact that
would have.
Let me know when you do. I have now implemented such a thing in my 
plugin and would love to have this functionality in core.


The code for cache in your github repository implements cache with
feature ids as keys. That is useful for attribute table queries,
however for rendering it does not help much. In the cache we will need
also a spatial index for fast spatial queries used for map rendering,
identification and geometry editing. Without spatial index, the cache
is quite slow if the whole cache need to be traversed in order to find
features in a specified rectangle. As of now, the cache you've done is
bound to a one particular query it gets: I would be most interested in
a general cache to avoid having situations where cache would be filled
by results of a first query and then flushed and filled with results
of a following query.
Yes it is bound to feature ids by now. My idea was, that you will keep 
track of keys of features of interest anyway. Let's say, the keys of 
features of a particular rectangle.



The caching behavior should be ideally configurable: e.g. which
layers/providers to cache (probably we don't want to cache huge
layers), what to cache (everything / only geometries / only attributes
/ geometries and some attributes), cache size limit.
My cache approach does most of this: only geometries, only attributes or 
both (A subset of attributes is still solved unsatisfactory, but that 
wouldn't be too hard to add) and cache limits. The only thing missing is 
the possibility to configure which layers to cache (Needs to be a higher 
level approach maybe? Layer settings?)


The next question is how to invalidate the cache, so that if data have
been changed in the backend, the user will not end up with a cached
version forever. In some providers I think it's possible to detect the
updates automatically, but some providers do not have any such
notifications. Maybe we should allow users to set a timeout - after
specified interval the cache would need to check again with the
backend whether the features have not been altered.
It would be nice to get this information from providers which do have 
the possibility, for others, see below.


Back to your original question whether such improved cache could be
used instead of the current geometry cache: yes, it could, in case it
will include a spatial index. We are really in a need of a general
caching mechanism, so I would welcome any work in this area. The
speedup of rendering from cache should be significant. The current
caching of geometries is there just to preserve the functionality of
moveVertex() and other functions from vector layer that were using
cached geometries before.


To solve these problems I was thinking about adding an abstract 
interface ( QgsCachItemManager or something) which provides at least the 
method

* bool requiresCaching( QgsFeature& )

It could then also be extended to check if the cache is able to answer a 
certain select operation (in which case it would return a QgsFeatureIds 
list). So it could keep track of a spatial index, cache age and so on. 
Speak custom tailored caching depending on the special use-case. 
Accomplishing that would require some small internal changes, as we need 
to keep track of removed items. It's using QCache now, which as far as I 
know does not communicate, when a certain item is removed (it cleans 
silently). So either we'd have to repl

Re: [Qgis-developer] New Vector API

[Martin Dobias]

On Tue, Jan 8, 2013 at 9:46 AM, Matthias Kuhn  wrote:
>
> Layers are rendered correctly again.
>
> However, dataProvider.getFeatures() doesn't return any features in my plugin
> ( the same instructions used to work before your last two commits ).

Matthias,
my recent update to feature iterators should fix this and some more issues, see:

https://github.com/qgis/Quantum-GIS/commit/a6c5fd875bc73e4775b825640a7cedcc2f5a8832

The default behavior for providers is now that they close any previous
active iterator. Later I will be adding a new provider capability
where the provider could state that it supports concurrent iterators.

Regards
Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi Matthias

On Tue, Jan 15, 2013 at 11:46 PM, Matthias Kuhn  wrote:
> Hi Martin,
>
> This mail concerns the vectorlayercache outlined in section 8) of your mail.
>
> I'm currently working on the attribute table which has some internal caching
> already. As I'm currently rewriting the attribute table, I moved this code
> outside and generalized it as good as possible. This work is currently
> available in my branch vectorlayercache [0]. Do you think this could be used
> instead of your geometry cache? It should be able to cache geometries of
> features (although I only tested attributes so far) and update and
> invalidate them if required.
>
> I have also the idea to create a QgsVectorDataProviderCacheProxy class which
> inherits QgsVectorDataProvider and which you can use as a proxy between your
> app and the DataProvider. I've some unfinished code here which provides such
> functionality. But for now, the cache in front of the VectorLayer should
> already help a lot.

Just out of curiosity, what are you rewriting in the attribute table -
is it just the cache or does it involve more changes?

Your idea of having a cache proxy vector data provider is interesting.
Such transparent approach would enable us to remove caching from
vector data provider implementations (e.g. wfs) and transparently add
caching to the ones where caching would be useful (e.g.
delimitedtext). I need to give it some more thoughts what impact that
would have.

The code for cache in your github repository implements cache with
feature ids as keys. That is useful for attribute table queries,
however for rendering it does not help much. In the cache we will need
also a spatial index for fast spatial queries used for map rendering,
identification and geometry editing. Without spatial index, the cache
is quite slow if the whole cache need to be traversed in order to find
features in a specified rectangle. As of now, the cache you've done is
bound to a one particular query it gets: I would be most interested in
a general cache to avoid having situations where cache would be filled
by results of a first query and then flushed and filled with results
of a following query.

The caching behavior should be ideally configurable: e.g. which
layers/providers to cache (probably we don't want to cache huge
layers), what to cache (everything / only geometries / only attributes
/ geometries and some attributes), cache size limit.

The next question is how to invalidate the cache, so that if data have
been changed in the backend, the user will not end up with a cached
version forever. In some providers I think it's possible to detect the
updates automatically, but some providers do not have any such
notifications. Maybe we should allow users to set a timeout - after
specified interval the cache would need to check again with the
backend whether the features have not been altered.

Back to your original question whether such improved cache could be
used instead of the current geometry cache: yes, it could, in case it
will include a spatial index. We are really in a need of a general
caching mechanism, so I would welcome any work in this area. The
speedup of rendering from cache should be significant. The current
caching of geometries is there just to preserve the functionality of
moveVertex() and other functions from vector layer that were using
cached geometries before.

Regards
Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin,

This mail concerns the vectorlayercache outlined in section 8) of your mail.

I'm currently working on the attribute table which has some internal 
caching already. As I'm currently rewriting the attribute table, I moved 
this code outside and generalized it as good as possible. This work is 
currently available in my branch vectorlayercache [0]. Do you think this 
could be used instead of your geometry cache? It should be able to cache 
geometries of features (although I only tested attributes so far) and 
update and invalidate them if required.


I have also the idea to create a QgsVectorDataProviderCacheProxy class 
which inherits QgsVectorDataProvider and which you can use as a proxy 
between your app and the DataProvider. I've some unfinished code here 
which provides such functionality. But for now, the cache in front of 
the VectorLayer should already help a lot.


[0] https://github.com/matthias-kuhn/Quantum-GIS/tree/vectorlayercache

Regards,
Matthias

On 12/28/2012 05:03 PM, Martin Dobias wrote:

Hi everyone,

it seems that now it is a good time to ask for a broader review of the
work I have been doing during recent months: essentially making QGIS
vector API more flexible and more ready for introduction of threading
for rendering. That resulted in a greater refactoring of some parts of
QgsVectorLayer class and QgsVectorDataProvider implementations.
Everything is in new_vector_api branch in QGIS repository on GitHub
[1].

There are few things that are not finished, but should not take too
much work to sort out:
- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated to new API
- disabled mapserver - there are few things to update

If no serious problems will be found, I would like to merge the branch
to master and continue working on that on master branch to avoid the
possibility to drift (again) too much from the quick development that
happens on master. In short term I plan to do some polishing and
fixing bugs, then eventually start looking at the threading again.

For more details about what happened in the branch, see the text below
(long read!). A great help from early testers would be to compile the
branch, try to do some "usual" stuff with vectors and see if things
break apart (crashes? data corruption?).

Looking forward to your comments!

Regards
Martin

[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api



QGIS VECTOR API CHANGES

1. QgsFeature (changes)

a) access to attributes by name. Vector data providers and vector
layer assign pointer to the fields in QgsFeature, so it is possible to
set/get attributes by their name. This is not as efficient as with
direct access by index, but often this convenience is useful for
users.

b) attributes are stored in a QVector container rather than a QMap.
The major advantage is simplification of logic related to handling of
attributes: it's not possible to have "holes" in the array of
fields/attributes. Currently it is possible that for example layer
with three fields returns indexes 0,1,3 - but it is not so common nor
obvious, so it's a common source of errors. After refactoring there
must not be any holes, so a layer with three fields always returns
indexes 0,1,2. When iterating over layer's features, QgsFeature always
contains a vector of all layer's attributes. In case the client has
not requested attributes to be fetched, the attributes contain invalid
values.


2. QgsFields (new class)

Just like attributes within a feature are stored in a vector instead
of map, also layer's fields are now stored in a vector. QgsFields is a
new class that mimics QVector API and adds two more pieces of
functionality:

a) fast lookup of field index from name. When QgsFields is populated
with fields, it creates a map of fields that facilitates fast lookups

b) detection of field origin. When working with a vector layer, it is
sometimes useful to find out origin of the field - whether it comes
from provider, from a join or whether it is a newly added field (not
committed). In the future we could add also expression-based fields,
creating a field calculator that calculates the values on the fly.


3. QgsFeatureRequest (new class)

Class that encapsulates requests for features to a vector layer or
vector data provider. Right now in master branch, the request is
defined by arguments to select() method. That's not very flexible nor
simple to use. Feature request class allows easier extensibility in
future (support generic expression filter, native provider's SQL
filter...).

Without any customization, the request will ask for all features with
geometries and attributes - somehow better default that the current
one that does not fetch attributes if their list is not explicitly
given.

(I'm not yet completely happy with the API of this class, so it may be
changed to some degree. Suggestions are welcome.)

Examples:
- fetch all features:
 QgsFeatureRequest()
- fetch all features, only one attribute
 QgsFeatureRequest().setSubse

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin,

On 01/07/2013 11:04 PM, Martin Dobias wrote:

On Mon, Jan 7, 2013 at 12:42 PM, Matthias Kuhn  wrote:

Hi Martin,

as already mentioned by Marco, there seems to be a problem with the latest
fix. I have multiple postgres layers and after every redraw only one layer
(random) is drawn (sometimes not even one layer).

When I quit QGIS I get a segmentation fault in the postgres provider
(QgsPostgresConn, member methods called on a NULL pointer), backtrace
attached.

Matthias,

the latest fix should solve both problems. In the previous commit I
have removed automatic closing of postgres cursor when iterating over
features finishes and this caused that a new cursor (with the same
name) was being opened while the old one still existed.

Martin

Layers are rendered correctly again.

However, dataProvider.getFeatures() doesn't return any features in my 
plugin ( the same instructions used to work before your last two commits ).


The postgres provider logs the following:

-
2 cursor states lost.
SQL: DECLARE qgisf6 BINARY CURSOR FOR SELECT 
asbinary(force_2d("situation_geometry"),'NDR'),"gid","obj_id"::text,"type"::text,"node_type"::text,"level"::text,"usage_current"::text,"cover_level"::text,"description"::text,asewkt("detail_geometry") 
FROM "qgep"."vw_network_node"

Result: 7 (ERROR:  cursor "qgisf6" already exists
)
-
Query: move absolute 0 in qgisf6 returned 7 [ERROR:  cursor "qgisf6" 
does not exist

]
-

Regards,
Matthias
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Martin Dobias]

On Mon, Jan 7, 2013 at 12:42 PM, Matthias Kuhn  wrote:
> Hi Martin,
>
> as already mentioned by Marco, there seems to be a problem with the latest
> fix. I have multiple postgres layers and after every redraw only one layer
> (random) is drawn (sometimes not even one layer).
>
> When I quit QGIS I get a segmentation fault in the postgres provider
> (QgsPostgresConn, member methods called on a NULL pointer), backtrace
> attached.

Matthias,

the latest fix should solve both problems. In the previous commit I
have removed automatic closing of postgres cursor when iterating over
features finishes and this caused that a new cursor (with the same
name) was being opened while the old one still existed.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin,

as already mentioned by Marco, there seems to be a problem with the 
latest fix. I have multiple postgres layers and after every redraw only 
one layer (random) is drawn (sometimes not even one layer).


When I quit QGIS I get a segmentation fault in the postgres provider 
(QgsPostgresConn, member methods called on a NULL pointer), backtrace 
attached.


Regards,
Matthias


On 01/06/2013 10:01 PM, Martin Dobias wrote:

On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn  wrote:

I've just been able to segfault:

a = dataProvider.getFeatures( QgsFeatureRequest().setFilterFid( featureId )
)
a.rewind()
a.next()

Hi Matthias

I was able to replicate the problem in postgres provider - I have
fixed the issues, please check again. By the way, funny thing is that
rewind() call obviously never worked correctly in postgres provider
until now (it executed "move 0" instead of "move absolute 0",
resulting in no actual move).

Please let me know if you encounter any other problems.

Martin



Thread 1 (Thread 0x750f9880 (LWP 8014)):
#0  0x7fffe214db8e in QgsPostgresConn::PQstatus (this=0x0) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresconn.cpp:920
No locals.
#1  0x7fffe214c469 in QgsPostgresConn::PQexec (this=0x0, query="CLOSE 
qgisf0", logError=false) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresconn.cpp:748
res = 0x5550a6
#2  0x7fffe214ce5b in QgsPostgresConn::PQexecNR (this=0x0, query="CLOSE 
qgisf0", retry=true) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresconn.cpp:825
res = {
  mRes = 0x7fffc4a0
}
errorStatus = 32767
#3  0x7fffe214cd2b in QgsPostgresConn::closeCursor (this=0x0, 
cursorName="qgisf0") at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresconn.cpp:811
No locals.
#4  0x7fffe215aae2 in QgsPostgresFeatureIterator::close (this=0x4892620) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresfeatureiterator.cpp:192
No locals.
#5  0x7fffe2159f57 in 
QgsPostgresFeatureIterator::~QgsPostgresFeatureIterator (this=0x4892620, 
__in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresfeatureiterator.cpp:69
No locals.
#6  0x7fffe215a000 in 
QgsPostgresFeatureIterator::~QgsPostgresFeatureIterator (this=0x4892620, 
__in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/providers/postgres/qgspostgresfeatureiterator.cpp:70
No locals.
#7  0x75c88f7a in QgsAbstractFeatureIterator::deref (this=0x4892620) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsfeatureiterator.cpp:23
No locals.
#8  0x75d372f9 in QgsFeatureIterator::~QgsFeatureIterator 
(this=0xb8159d8, __in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsfeatureiterator.h:95
No locals.
#9  0x75d69903 in 
QgsVectorLayerFeatureIterator::~QgsVectorLayerFeatureIterator (this=0xb815940, 
__in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsvectorlayerfeatureiterator.cpp:48
No locals.
#10 0x75d699d0 in 
QgsVectorLayerFeatureIterator::~QgsVectorLayerFeatureIterator (this=0xb815940, 
__in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsvectorlayerfeatureiterator.cpp:51
No locals.
#11 0x75c88f7a in QgsAbstractFeatureIterator::deref (this=0xb815940) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsfeatureiterator.cpp:23
No locals.
#12 0x75d372f9 in QgsFeatureIterator::~QgsFeatureIterator 
(this=0x48b8170, __in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsfeatureiterator.h:95
No locals.
#13 0x75d412b3 in QgsVectorLayer::~QgsVectorLayer (this=0x48b7fb0, 
__in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsvectorlayer.cpp:179
No locals.
#14 0x75d4167e in QgsVectorLayer::~QgsVectorLayer (this=0x48b7fb0, 
__in_chrg=) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsvectorlayer.cpp:204
No locals.
#15 0x75cd5f9e in QgsMapLayerRegistry::removeMapLayers (this=0x125bc70, 
theLayerIds=QStringList = {...}, theEmitSignal=false) at 
/home/kk/dev/cpp/qgis/Quantum-GIS/src/core/qgsmaplayerregistry.cpp:116
myId = "bodenbedeckungsflaechen_symbole20120316162620304"
_container_ = {
  c = QStringList = {
[0] = "bodenbedeckungsflaechen_symbole20120316162620304",
[1] = "gebaeude20120316163628490",
[2] = "grenzpunkte20120316164004001",
[3] = "liegenschaften20120316163830734",
[4] = "liegenschaften_beschriftung20120316164202086",
[5] = "vw_manhole20120813141942568",
[6] = "vw_network_node20121129124610435",
[7] = "vw_network_segment20121129124546955",
[8] = "vw_reach20120813141922936",
[9] = "vw_special_structure20120817151756659"
  }, 
  brk = 0, 
  i = {
i = 0xb802518
  }, 
  e = {
i = 0xb802568
  }
}
#16 0x75cd6264 in QgsMa

Re: [Qgis-developer] New Vector API

[Marco Hugentobler]

Hi Martin

Finally I had a look at the vector changes and I really like the new 
architecture. I agree it is important to merge soon. Merging as soon as 
server and the remaining providers are ported would be good in my opinion.
Btw, there is a problem with postgis layers, probably related to the 
recent fix: After zooming to a new extent, I always need to refresh a 
second time to get the features (with the refresh button in the main 
window). Otherwise, no features are drawn.


Regards,
Marco




On 28.12.2012 17:03, Martin Dobias wrote:

Hi everyone,

it seems that now it is a good time to ask for a broader review of the
work I have been doing during recent months: essentially making QGIS
vector API more flexible and more ready for introduction of threading
for rendering. That resulted in a greater refactoring of some parts of
QgsVectorLayer class and QgsVectorDataProvider implementations.
Everything is in new_vector_api branch in QGIS repository on GitHub
[1].

There are few things that are not finished, but should not take too
much work to sort out:
- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated to new API
- disabled mapserver - there are few things to update

If no serious problems will be found, I would like to merge the branch
to master and continue working on that on master branch to avoid the
possibility to drift (again) too much from the quick development that
happens on master. In short term I plan to do some polishing and
fixing bugs, then eventually start looking at the threading again.

For more details about what happened in the branch, see the text below
(long read!). A great help from early testers would be to compile the
branch, try to do some "usual" stuff with vectors and see if things
break apart (crashes? data corruption?).

Looking forward to your comments!

Regards
Martin

[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api



QGIS VECTOR API CHANGES

1. QgsFeature (changes)

a) access to attributes by name. Vector data providers and vector
layer assign pointer to the fields in QgsFeature, so it is possible to
set/get attributes by their name. This is not as efficient as with
direct access by index, but often this convenience is useful for
users.

b) attributes are stored in a QVector container rather than a QMap.
The major advantage is simplification of logic related to handling of
attributes: it's not possible to have "holes" in the array of
fields/attributes. Currently it is possible that for example layer
with three fields returns indexes 0,1,3 - but it is not so common nor
obvious, so it's a common source of errors. After refactoring there
must not be any holes, so a layer with three fields always returns
indexes 0,1,2. When iterating over layer's features, QgsFeature always
contains a vector of all layer's attributes. In case the client has
not requested attributes to be fetched, the attributes contain invalid
values.


2. QgsFields (new class)

Just like attributes within a feature are stored in a vector instead
of map, also layer's fields are now stored in a vector. QgsFields is a
new class that mimics QVector API and adds two more pieces of
functionality:

a) fast lookup of field index from name. When QgsFields is populated
with fields, it creates a map of fields that facilitates fast lookups

b) detection of field origin. When working with a vector layer, it is
sometimes useful to find out origin of the field - whether it comes
from provider, from a join or whether it is a newly added field (not
committed). In the future we could add also expression-based fields,
creating a field calculator that calculates the values on the fly.


3. QgsFeatureRequest (new class)

Class that encapsulates requests for features to a vector layer or
vector data provider. Right now in master branch, the request is
defined by arguments to select() method. That's not very flexible nor
simple to use. Feature request class allows easier extensibility in
future (support generic expression filter, native provider's SQL
filter...).

Without any customization, the request will ask for all features with
geometries and attributes - somehow better default that the current
one that does not fetch attributes if their list is not explicitly
given.

(I'm not yet completely happy with the API of this class, so it may be
changed to some degree. Suggestions are welcome.)

Examples:
- fetch all features:
 QgsFeatureRequest()
- fetch all features, only one attribute
 QgsFeatureRequest().setSubsetOfAttributes(QStringList("myfield"),
provider->fields())
- fetch all features, without geometries
 QgsFeatureRequest().setFlags(QgsFeatureRequest::NoGeometry)
- fetch only features from particular extent
 QgsFeatureRequest().setFilterRect(QgsRectangle(0,0,1,1))
- fetch only one feature
 QgsFeatureRequest().setFilterFid(45)



4. QgsFeatureIterator (new class)

The iterator class allows iteration over features of a vector layer or
a provider. It contains the usual nextFe

Re: [Qgis-developer] New Vector API

[Martin Dobias]

On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn  wrote:
> I've just been able to segfault:
>
> a = dataProvider.getFeatures( QgsFeatureRequest().setFilterFid( featureId )
> )
> a.rewind()
> a.next()

Hi Matthias

I was able to replicate the problem in postgres provider - I have
fixed the issues, please check again. By the way, funny thing is that
rewind() call obviously never worked correctly in postgres provider
until now (it executed "move 0" instead of "move absolute 0",
resulting in no actual move).

Please let me know if you encounter any other problems.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin,

On 01/04/2013 10:19 PM, Martin Dobias wrote:

Hi Matthias

thanks for your mail.

On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn  wrote:

You've been talking about updating to PyQt4 SIP version 2. I noticed this
change did not yet happen. What are your plans regarding this update?

Yes, the plans are still there. I'm only wondering what would be the
best time to switch - right now within the branch or a bit later once
the branch is merged and the new code settles down a bit? I can see a
slight advantage of the latter approach of having a greater chance of
separating problems caused by the merge and problems caused by the SIP
API change, but I do not have a strong opinion here.
I think doing it now would be the best time, as the API is already and 
is even more going to be broken. For plugin devs it will probably be the 
easiest to take care only once. No strong opinion here either.

I'm also looking forward to seeing QgsFeatureRequest support QgsExpression,
I think this is not yet supported?

No, it's not yet supported. It should be possible to add features like
this one easily even later without breaking any API. It is not my
priority now, of course any contributions in that direction are
welcome! (and also for support of native expressions for postgres,
spatialite and other DBMSs)

Maybe I find the time to take care of this.
What I would like to see (but I think that's a rather tough one): to 
make the providers translate QgsExpression to native backend commands. 
Although this can lead to some nasty side-effects.



PS:
I've just been able to segfault:

a = dataProvider.getFeatures( QgsFeatureRequest().setFilterFid( featureId )
)
a.rewind()
a.next()

Which provider was that? Just tried with OGR and memory providers and it worked.



PPS:
Another strange thing (maybe related), calling next() directly on a fid
filter returns a feature, storing into a QgsFeatureIterator and calling next
on this throws a StopIteration:

dataProvider.getFeatures(QgsFeatureRequest().setFilterFid( featureId )
).next()


features = dataProvider.getFeatures(QgsFeatureRequest().setFilterFid(
featureId ) )
f = features.next()
Traceback (most recent call last):
   File "/home/kk/.qgis//python/plugins/remotedebug/pysrc/pydevd_exec.py",
line 2, in Exec
 exec exp in global_vars, local_vars
   File "", line 1, in 
StopIteration

Again, works for me correctly with OGR and memory provider - maybe
some provider other has that problem.

Both tests have been made with a postgres backend.



Regards
Martin

Regards
Matthias
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi Matthias

thanks for your mail.

On Thu, Jan 3, 2013 at 10:37 AM, Matthias Kuhn  wrote:
>
> You've been talking about updating to PyQt4 SIP version 2. I noticed this
> change did not yet happen. What are your plans regarding this update?

Yes, the plans are still there. I'm only wondering what would be the
best time to switch - right now within the branch or a bit later once
the branch is merged and the new code settles down a bit? I can see a
slight advantage of the latter approach of having a greater chance of
separating problems caused by the merge and problems caused by the SIP
API change, but I do not have a strong opinion here.


> I'm also looking forward to seeing QgsFeatureRequest support QgsExpression,
> I think this is not yet supported?

No, it's not yet supported. It should be possible to add features like
this one easily even later without breaking any API. It is not my
priority now, of course any contributions in that direction are
welcome! (and also for support of native expressions for postgres,
spatialite and other DBMSs)


> PS:
> I've just been able to segfault:
>
> a = dataProvider.getFeatures( QgsFeatureRequest().setFilterFid( featureId )
> )
> a.rewind()
> a.next()

Which provider was that? Just tried with OGR and memory providers and it worked.


> PPS:
> Another strange thing (maybe related), calling next() directly on a fid
> filter returns a feature, storing into a QgsFeatureIterator and calling next
> on this throws a StopIteration:
>
> dataProvider.getFeatures(QgsFeatureRequest().setFilterFid( featureId )
> ).next()
> 
>
> features = dataProvider.getFeatures(QgsFeatureRequest().setFilterFid(
> featureId ) )
> f = features.next()
> Traceback (most recent call last):
>   File "/home/kk/.qgis//python/plugins/remotedebug/pysrc/pydevd_exec.py",
> line 2, in Exec
> exec exp in global_vars, local_vars
>   File "", line 1, in 
> StopIteration

Again, works for me correctly with OGR and memory provider - maybe
some provider other has that problem.


Regards
Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Matthias Kuhn]

Hi Martin,

Thank you for your work.

I really appreciate these features. Some of them will allow me to 
simplify my code drastically.
I'm updating the plugin I'm working on right now and will let you know 
about any problems which occur.


You've been talking about updating to PyQt4 SIP version 2. I noticed 
this change did not yet happen. What are your plans regarding this update?


I'm also looking forward to seeing QgsFeatureRequest support 
QgsExpression, I think this is not yet supported?


Cheers

PS:
I've just been able to segfault:

a = dataProvider.getFeatures( QgsFeatureRequest().setFilterFid( 
featureId ) )

a.rewind()
a.next()

I'm not sure if the featureId existed or not ( I think it did )

PPS:
Another strange thing (maybe related), calling next() directly on a fid 
filter returns a feature, storing into a QgsFeatureIterator and calling 
next on this throws a StopIteration:


dataProvider.getFeatures(QgsFeatureRequest().setFilterFid( featureId ) 
).next()



features = dataProvider.getFeatures(QgsFeatureRequest().setFilterFid( 
featureId ) )

f = features.next()
Traceback (most recent call last):
  File 
"/home/kk/.qgis//python/plugins/remotedebug/pysrc/pydevd_exec.py", line 
2, in Exec

exec exp in global_vars, local_vars
  File "", line 1, in 
StopIteration


On 12/28/2012 05:03 PM, Martin Dobias wrote:

Hi everyone,

it seems that now it is a good time to ask for a broader review of the
work I have been doing during recent months: essentially making QGIS
vector API more flexible and more ready for introduction of threading
for rendering. That resulted in a greater refactoring of some parts of
QgsVectorLayer class and QgsVectorDataProvider implementations.
Everything is in new_vector_api branch in QGIS repository on GitHub
[1].

There are few things that are not finished, but should not take too
much work to sort out:
- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated to new API
- disabled mapserver - there are few things to update

If no serious problems will be found, I would like to merge the branch
to master and continue working on that on master branch to avoid the
possibility to drift (again) too much from the quick development that
happens on master. In short term I plan to do some polishing and
fixing bugs, then eventually start looking at the threading again.

For more details about what happened in the branch, see the text below
(long read!). A great help from early testers would be to compile the
branch, try to do some "usual" stuff with vectors and see if things
break apart (crashes? data corruption?).

Looking forward to your comments!

Regards
Martin

[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api



QGIS VECTOR API CHANGES

1. QgsFeature (changes)

a) access to attributes by name. Vector data providers and vector
layer assign pointer to the fields in QgsFeature, so it is possible to
set/get attributes by their name. This is not as efficient as with
direct access by index, but often this convenience is useful for
users.

b) attributes are stored in a QVector container rather than a QMap.
The major advantage is simplification of logic related to handling of
attributes: it's not possible to have "holes" in the array of
fields/attributes. Currently it is possible that for example layer
with three fields returns indexes 0,1,3 - but it is not so common nor
obvious, so it's a common source of errors. After refactoring there
must not be any holes, so a layer with three fields always returns
indexes 0,1,2. When iterating over layer's features, QgsFeature always
contains a vector of all layer's attributes. In case the client has
not requested attributes to be fetched, the attributes contain invalid
values.


2. QgsFields (new class)

Just like attributes within a feature are stored in a vector instead
of map, also layer's fields are now stored in a vector. QgsFields is a
new class that mimics QVector API and adds two more pieces of
functionality:

a) fast lookup of field index from name. When QgsFields is populated
with fields, it creates a map of fields that facilitates fast lookups

b) detection of field origin. When working with a vector layer, it is
sometimes useful to find out origin of the field - whether it comes
from provider, from a join or whether it is a newly added field (not
committed). In the future we could add also expression-based fields,
creating a field calculator that calculates the values on the fly.


3. QgsFeatureRequest (new class)

Class that encapsulates requests for features to a vector layer or
vector data provider. Right now in master branch, the request is
defined by arguments to select() method. That's not very flexible nor
simple to use. Feature request class allows easier extensibility in
future (support generic expression filter, native provider's SQL
filter...).

Without any customization, the request will ask for all features with
geometries and attributes - somehow better default that the current
one that does n

Re: [Qgis-developer] New Vector API

[Paolo Cavallini]

Il 01/01/2013 20:41, Werner Macho ha scritto:
> Hi!
> 
> Just for the logs .. I agree with Jürgen in that case..
> It does not make any sense to release a beta now - with some new implemented 
> things
> and some not ..
> From my point of view - the only last thing we are all waiting for is Martins
> threading implementation ..


Of course I agree with you to avoid useless work. I'm just worried that if we 
keep on
adding new features and breaking the API the 2.0 release will take too long.
AFAIK the blocking factors we agreed upon are:

  * Symbology overhaul
  * Labelling overhaul
  * New web site and docs
  * SEXTANTE integration
  * Interface cleanup and consistency
  * Plugins migration

I think there is still work to be done on most of these, if not all.

All the best.
-- 
Paolo Cavallini - Faunalia
www.faunalia.eu
Full contact details at www.faunalia.eu/pc
Nuovi corsi QGIS e PostGIS: http://www.faunalia.it/calendario
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Werner Macho]

Hi!

Just for the logs .. I agree with Jürgen in that case..
It does not make any sense to release a beta now - with some new
implemented things and some not ..
>From my point of view - the only last thing we are all waiting for is
Martins threading implementation ..
After merging his branch I'd vote for a feature freeze and a longer
stabilization phase ..
And then .. Lets' do the 2.0 ;)

regards
Werner



On Tue, Jan 1, 2013 at 5:00 PM, Jürgen E.  wrote:

> Hi Paolo,
>
> On Sat, 29. Dec 2012 at 18:38:26 +0100, Paolo Cavallini wrote:
> > Why not releasing a beta before merging?
>
> What for?
>
> Either we want that stuff in the release, then we need to merge it before
> the
> release and releasing a beta now of stuff that we're probably going to
> break
> IMHO doesn't make any sense.
>
> Then again I don't see a big point in releasing betas at all.  We have
> nightly
> builds.  And given that we have a feature freeze before a release those
> will
> also settle.  If that isn't enough we could also schedule switch from
> nightly
> to weekly builds before a release to have an easier target for testing.
>
> Doing extra betas (and we like won't get away with just one) would just
> imply
> more work and extra infrastructure.
>
>
> Jürgen
>
> --
> Jürgen E. Fischer norBIT GmbH   Tel. +49-4931-918175-31
> Dipl.-Inf. (FH)   Rheinstraße 13Fax. +49-4931-918175-50
> Software Engineer D-26506 Norden
> http://www.norbit.de
> committ(ed|ing) to Quantum GIS IRC: jef on FreeNode
>
> --
> norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
> Rheinstrasse 13, 26506 Norden
> GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502
>
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Jürgen E . Fischer]

Hi Paolo,

On Sat, 29. Dec 2012 at 18:38:26 +0100, Paolo Cavallini wrote:
> Why not releasing a beta before merging?

What for?

Either we want that stuff in the release, then we need to merge it before the
release and releasing a beta now of stuff that we're probably going to break
IMHO doesn't make any sense.

Then again I don't see a big point in releasing betas at all.  We have nightly
builds.  And given that we have a feature freeze before a release those will
also settle.  If that isn't enough we could also schedule switch from nightly
to weekly builds before a release to have an easier target for testing.

Doing extra betas (and we like won't get away with just one) would just imply
more work and extra infrastructure.


Jürgen 

-- 
Jürgen E. Fischer norBIT GmbH   Tel. +49-4931-918175-31
Dipl.-Inf. (FH)   Rheinstraße 13Fax. +49-4931-918175-50
Software Engineer D-26506 Norden   http://www.norbit.de
committ(ed|ing) to Quantum GIS IRC: jef on FreeNode 


-- 
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Paolo Cavallini]

Il 31/12/2012 17:15, Nathan Woodrow ha scritto:
> I don't think that is a great idea. A beta implies that there is a new
> release coming with just bugs fixes this would not be the case if we
> released a beta and then merged the changes.

besides the name, I think waiting until after the merge, and the subsequent 
fixes,
could delay a release for a long time.
all the best.
-- 
Paolo Cavallini - Faunalia
www.faunalia.eu
Full contact details at www.faunalia.eu/pc
Nuovi corsi QGIS e PostGIS: http://www.faunalia.it/calendario
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Nathan Woodrow]

I don't think that is a great idea. A beta implies that there is a new
release coming with just bugs fixes this would not be the case if we
released a beta and then merged the changes.
From: Giovanni Manghi
Sent: 1/01/2013 2:10 AM
To: qgis-developer@lists.osgeo.org
Subject: Re: [Qgis-developer] New Vector API
> Why not releasing a beta before merging?

I completely agree.


cheers


-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Giovanni Manghi]

> Why not releasing a beta before merging?

I completely agree.


cheers


-- Giovanni --
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Alexander Bruy]

Hi Martin,

thanks, this fix works fine, now all compiled without errors

On Sat, 29 Dec 2012 02:35:36 +0100
Martin Dobias  wrote:

> I haven't tested with SpatiaLite 4 yet. I've just committed a quick
> fix that _may_ fix your problems, but there may be more of them -
> please let me know in that case, I will try to compile QGIS with
> SpatiaLite 4 to fix the remaining problems.

-- 
Alexander Bruy
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Paolo Cavallini]

Why not releasing a beta before merging?
All the best.

Marco Hugentobler  ha scritto:

>
>Hi Martin
>
>Cool, great news! Can't wait to try out the new vector classes. I'm
>away from the computer until 4th of January. Probably other core devs
>are out of contact too, so please make the review period long enough.
>
>Regards, Marco
>
>
>
>Von Samsung Galaxy Note gesendetMartin Dobias  hat
>geschrieben:Hi everyone,
>
>it seems that now it is a good time to ask for a broader review of the
>work I have been doing during recent months: essentially making QGIS
>vector API more flexible and more ready for introduction of threading
>for rendering. That resulted in a greater refactoring of some parts of
>QgsVectorLayer class and QgsVectorDataProvider implementations.
>Everything is in new_vector_api branch in QGIS repository on GitHub
>[1].
>
>There are few things that are not finished, but should not take too
>much work to sort out:
>- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated
>to new API
>- disabled mapserver - there are few things to update
>
>If no serious problems will be found, I would like to merge the branch
>to master and continue working on that on master branch to avoid the
>possibility to drift (again) too much from the quick development that
>happens on master. In short term I plan to do some polishing and
>fixing bugs, then eventually start looking at the threading again.
>
>For more details about what happened in the branch, see the text below
>(long read!). A great help from early testers would be to compile the
>branch, try to do some "usual" stuff with vectors and see if things
>break apart (crashes? data corruption?).
>
>Looking forward to your comments!
>
>Regards
>Martin
>
>[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api
>
>
>
>QGIS VECTOR API CHANGES
>
>1. QgsFeature (changes)
>
>a) access to attributes by name. Vector data providers and vector
>layer assign pointer to the fields in QgsFeature, so it is possible to
>set/get attributes by their name. This is not as efficient as with
>direct access by index, but often this convenience is useful for
>users.
>
>b) attributes are stored in a QVector container rather than a QMap.
>The major advantage is simplification of logic related to handling of
>attributes: it's not possible to have "holes" in the array of
>fields/attributes. Currently it is possible that for example layer
>with three fields returns indexes 0,1,3 - but it is not so common nor
>obvious, so it's a common source of errors. After refactoring there
>must not be any holes, so a layer with three fields always returns
>indexes 0,1,2. When iterating over layer's features, QgsFeature always
>contains a vector of all layer's attributes. In case the client has
>not requested attributes to be fetched, the attributes contain invalid
>values.
>
>
>2. QgsFields (new class)
>
>Just like attributes within a feature are stored in a vector instead
>of map, also layer's fields are now stored in a vector. QgsFields is a
>new class that mimics QVector API and adds two more pieces of
>functionality:
>
>a) fast lookup of field index from name. When QgsFields is populated
>with fields, it creates a map of fields that facilitates fast lookups
>
>b) detection of field origin. When working with a vector layer, it is
>sometimes useful to find out origin of the field - whether it comes
>from provider, from a join or whether it is a newly added field (not
>committed). In the future we could add also expression-based fields,
>creating a field calculator that calculates the values on the fly.
>
>
>3. QgsFeatureRequest (new class)
>
>Class that encapsulates requests for features to a vector layer or
>vector data provider. Right now in master branch, the request is
>defined by arguments to select() method. That's not very flexible nor
>simple to use. Feature request class allows easier extensibility in
>future (support generic expression filter, native provider's SQL
>filter...).
>
>Without any customization, the request will ask for all features with
>geometries and attributes - somehow better default that the current
>one that does not fetch attributes if their list is not explicitly
>given.
>
>(I'm not yet completely happy with the API of this class, so it may be
>changed to some degree. Suggestions are welcome.)
>
>Examples:
>- fetch all features:
>    QgsFeatureRequest()
>- fetch all features, only one attribute
>    QgsFeatureRequest().setSubsetOfAttributes(QStringList("myfield"),
>provider->fields())
>- fetch all features, without geometries
>    QgsFeatureRequest().setFlags(QgsFeatureRequest::NoGeometry)
>- fetch only features from particular extent
>    QgsFeatureRequest().setFilterRect(QgsRectangle(0,0,1,1))
>- fetch only one feature
>    QgsFeatureRequest().setFilterFid(45)
>
>
>
>4. QgsFeatureIterator (new class)
>
>The iterator class allows iteration over features of a vector layer or
>a provider. It contains the usual nextFeature() method that fills
>given QgsFea

Re: [Qgis-developer] New Vector API

[Marco Hugentobler]

Hi Martin

Cool, great news! Can't wait to try out the new vector classes. I'm away from 
the computer until 4th of January. Probably other core devs are out of contact 
too, so please make the review period long enough.

Regards, Marco



Von Samsung Galaxy Note gesendetMartin Dobias  hat 
geschrieben:Hi everyone,

it seems that now it is a good time to ask for a broader review of the
work I have been doing during recent months: essentially making QGIS
vector API more flexible and more ready for introduction of threading
for rendering. That resulted in a greater refactoring of some parts of
QgsVectorLayer class and QgsVectorDataProvider implementations.
Everything is in new_vector_api branch in QGIS repository on GitHub
[1].

There are few things that are not finished, but should not take too
much work to sort out:
- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated to new API
- disabled mapserver - there are few things to update

If no serious problems will be found, I would like to merge the branch
to master and continue working on that on master branch to avoid the
possibility to drift (again) too much from the quick development that
happens on master. In short term I plan to do some polishing and
fixing bugs, then eventually start looking at the threading again.

For more details about what happened in the branch, see the text below
(long read!). A great help from early testers would be to compile the
branch, try to do some "usual" stuff with vectors and see if things
break apart (crashes? data corruption?).

Looking forward to your comments!

Regards
Martin

[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api



QGIS VECTOR API CHANGES

1. QgsFeature (changes)

a) access to attributes by name. Vector data providers and vector
layer assign pointer to the fields in QgsFeature, so it is possible to
set/get attributes by their name. This is not as efficient as with
direct access by index, but often this convenience is useful for
users.

b) attributes are stored in a QVector container rather than a QMap.
The major advantage is simplification of logic related to handling of
attributes: it's not possible to have "holes" in the array of
fields/attributes. Currently it is possible that for example layer
with three fields returns indexes 0,1,3 - but it is not so common nor
obvious, so it's a common source of errors. After refactoring there
must not be any holes, so a layer with three fields always returns
indexes 0,1,2. When iterating over layer's features, QgsFeature always
contains a vector of all layer's attributes. In case the client has
not requested attributes to be fetched, the attributes contain invalid
values.


2. QgsFields (new class)

Just like attributes within a feature are stored in a vector instead
of map, also layer's fields are now stored in a vector. QgsFields is a
new class that mimics QVector API and adds two more pieces of
functionality:

a) fast lookup of field index from name. When QgsFields is populated
with fields, it creates a map of fields that facilitates fast lookups

b) detection of field origin. When working with a vector layer, it is
sometimes useful to find out origin of the field - whether it comes
from provider, from a join or whether it is a newly added field (not
committed). In the future we could add also expression-based fields,
creating a field calculator that calculates the values on the fly.


3. QgsFeatureRequest (new class)

Class that encapsulates requests for features to a vector layer or
vector data provider. Right now in master branch, the request is
defined by arguments to select() method. That's not very flexible nor
simple to use. Feature request class allows easier extensibility in
future (support generic expression filter, native provider's SQL
filter...).

Without any customization, the request will ask for all features with
geometries and attributes - somehow better default that the current
one that does not fetch attributes if their list is not explicitly
given.

(I'm not yet completely happy with the API of this class, so it may be
changed to some degree. Suggestions are welcome.)

Examples:
- fetch all features:
    QgsFeatureRequest()
- fetch all features, only one attribute
    QgsFeatureRequest().setSubsetOfAttributes(QStringList("myfield"),
provider->fields())
- fetch all features, without geometries
    QgsFeatureRequest().setFlags(QgsFeatureRequest::NoGeometry)
- fetch only features from particular extent
    QgsFeatureRequest().setFilterRect(QgsRectangle(0,0,1,1))
- fetch only one feature
    QgsFeatureRequest().setFilterFid(45)



4. QgsFeatureIterator (new class)

The iterator class allows iteration over features of a vector layer or
a provider. It contains the usual nextFeature() method that fills
given QgsFeature class with data.

Internally, this class is a wrapper around QgsAbstractFeatureIterator
interface that is implemented by each vector data provider and by
vector layer. In theory we could use directly
QgsAb

Re: [Qgis-developer] New Vector API

[Martin Dobias]

Hi Alex

On Fri, Dec 28, 2012 at 8:27 PM, Alexander Bruy
 wrote:
> Hi Martin,
>
> just tried to build new_vector_api branch under 32bit Slackware 14.0:
> GCC 4.7.1
> glibs 2.15
> Qt 4.8.2
> GDAL 1.9.2
> SpatiaLite 4.0.0
> Python 2.7.3
>
> And get next error:
>
> /home/alex/devel/cpp/qgis/src/providers/spatialite/qgsspatialiteprovider.cpp:
> In member function 'void
> QgsSpatiaLiteProvider::loadFieldsAbstractInterface(gaiaVectorLayerPtr)':
> /home/alex/devel/cpp/qgis/src/providers/spatialite/qgsspatialiteprovider.cpp:619:23:
> error: 'class QgsFields' has no member named 'insert'

I haven't tested with SpatiaLite 4 yet. I've just committed a quick
fix that _may_ fix your problems, but there may be more of them -
please let me know in that case, I will try to compile QGIS with
SpatiaLite 4 to fix the remaining problems.

Martin
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[John C. Tull]

On Dec 28, 2012, at 11:27 AM, Alexander Bruy  wrote:

> Hi Martin,
> 
> just tried to build new_vector_api branch under 32bit Slackware 14.0:
> GCC 4.7.1
> glibs 2.15
> Qt 4.8.2
> GDAL 1.9.2
> SpatiaLite 4.0.0
> Python 2.7.3
> 
> And get next error:
> 
> /home/alex/devel/cpp/qgis/src/providers/spatialite/qgsspatialiteprovider.cpp:
> In member function 'void
> QgsSpatiaLiteProvider::loadFieldsAbstractInterface(gaiaVectorLayerPtr)':
> /home/alex/devel/cpp/qgis/src/providers/spatialite/qgsspatialiteprovider.cpp:619:23:
> error: 'class QgsFields' has no member named 'insert'
> make[2]: *** 
> [src/providers/spatialite/CMakeFiles/spatialiteprovider.dir/qgsspatialiteprovider.cpp.o]
> Error 1
> make[1]: *** [src/providers/spatialite/CMakeFiles/spatialiteprovider.dir/all]
> Error 2
> make[1]: *** Waiting for unfinished jobs
> 
> -- 
> Alexander Bruy

Hi all,

I ran into the same compile issue on my OS X system.

Regards,
John

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Alexander Bruy]

Hi Martin,

just tried to build new_vector_api branch under 32bit Slackware 14.0:
GCC 4.7.1
glibs 2.15
Qt 4.8.2
GDAL 1.9.2
SpatiaLite 4.0.0
Python 2.7.3

And get next error:

/home/alex/devel/cpp/qgis/src/providers/spatialite/qgsspatialiteprovider.cpp:
In member function 'void
QgsSpatiaLiteProvider::loadFieldsAbstractInterface(gaiaVectorLayerPtr)':
/home/alex/devel/cpp/qgis/src/providers/spatialite/qgsspatialiteprovider.cpp:619:23:
error: 'class QgsFields' has no member named 'insert'
make[2]: *** 
[src/providers/spatialite/CMakeFiles/spatialiteprovider.dir/qgsspatialiteprovider.cpp.o]
Error 1
make[1]: *** [src/providers/spatialite/CMakeFiles/spatialiteprovider.dir/all]
Error 2
make[1]: *** Waiting for unfinished jobs

-- 
Alexander Bruy
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] New Vector API

[Régis Haubourg]

Hi Martin, 
all this looks great! I really had troubles with FieldMap and vector
handling in python plugin developpement. I really appriciate that step
towards more usability for basic plugin developpers.  Ready for testing and
correcting some plugins of course.
 Some classes were removed some days ago and many plugins are already
broken. From my point of view, it is a good timing for merging your branch,
before qe spend to much efforts on correcting them. (And I stick with
osgeo4w for testing, so it's hard to test your branch otherwirse ;-))
Régis



--
View this message in context: 
http://osgeo-org.1560.n6.nabble.com/New-Vector-API-tp5024891p5024910.html
Sent from the Quantum GIS - Developer mailing list archive at Nabble.com.
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] New Vector API

[Martin Dobias]

Hi everyone,

it seems that now it is a good time to ask for a broader review of the
work I have been doing during recent months: essentially making QGIS
vector API more flexible and more ready for introduction of threading
for rendering. That resulted in a greater refactoring of some parts of
QgsVectorLayer class and QgsVectorDataProvider implementations.
Everything is in new_vector_api branch in QGIS repository on GitHub
[1].

There are few things that are not finished, but should not take too
much work to sort out:
- disabled providers - mssql, osm, sqlanywhere, wfs - not yet updated to new API
- disabled mapserver - there are few things to update

If no serious problems will be found, I would like to merge the branch
to master and continue working on that on master branch to avoid the
possibility to drift (again) too much from the quick development that
happens on master. In short term I plan to do some polishing and
fixing bugs, then eventually start looking at the threading again.

For more details about what happened in the branch, see the text below
(long read!). A great help from early testers would be to compile the
branch, try to do some "usual" stuff with vectors and see if things
break apart (crashes? data corruption?).

Looking forward to your comments!

Regards
Martin

[1] https://github.com/qgis/Quantum-GIS/tree/new_vector_api



QGIS VECTOR API CHANGES

1. QgsFeature (changes)

a) access to attributes by name. Vector data providers and vector
layer assign pointer to the fields in QgsFeature, so it is possible to
set/get attributes by their name. This is not as efficient as with
direct access by index, but often this convenience is useful for
users.

b) attributes are stored in a QVector container rather than a QMap.
The major advantage is simplification of logic related to handling of
attributes: it's not possible to have "holes" in the array of
fields/attributes. Currently it is possible that for example layer
with three fields returns indexes 0,1,3 - but it is not so common nor
obvious, so it's a common source of errors. After refactoring there
must not be any holes, so a layer with three fields always returns
indexes 0,1,2. When iterating over layer's features, QgsFeature always
contains a vector of all layer's attributes. In case the client has
not requested attributes to be fetched, the attributes contain invalid
values.


2. QgsFields (new class)

Just like attributes within a feature are stored in a vector instead
of map, also layer's fields are now stored in a vector. QgsFields is a
new class that mimics QVector API and adds two more pieces of
functionality:

a) fast lookup of field index from name. When QgsFields is populated
with fields, it creates a map of fields that facilitates fast lookups

b) detection of field origin. When working with a vector layer, it is
sometimes useful to find out origin of the field - whether it comes
from provider, from a join or whether it is a newly added field (not
committed). In the future we could add also expression-based fields,
creating a field calculator that calculates the values on the fly.


3. QgsFeatureRequest (new class)

Class that encapsulates requests for features to a vector layer or
vector data provider. Right now in master branch, the request is
defined by arguments to select() method. That's not very flexible nor
simple to use. Feature request class allows easier extensibility in
future (support generic expression filter, native provider's SQL
filter...).

Without any customization, the request will ask for all features with
geometries and attributes - somehow better default that the current
one that does not fetch attributes if their list is not explicitly
given.

(I'm not yet completely happy with the API of this class, so it may be
changed to some degree. Suggestions are welcome.)

Examples:
- fetch all features:
QgsFeatureRequest()
- fetch all features, only one attribute
QgsFeatureRequest().setSubsetOfAttributes(QStringList("myfield"),
provider->fields())
- fetch all features, without geometries
QgsFeatureRequest().setFlags(QgsFeatureRequest::NoGeometry)
- fetch only features from particular extent
QgsFeatureRequest().setFilterRect(QgsRectangle(0,0,1,1))
- fetch only one feature
QgsFeatureRequest().setFilterFid(45)



4. QgsFeatureIterator (new class)

The iterator class allows iteration over features of a vector layer or
a provider. It contains the usual nextFeature() method that fills
given QgsFeature class with data.

Internally, this class is a wrapper around QgsAbstractFeatureIterator
interface that is implemented by each vector data provider and by
vector layer. In theory we could use directly
QgsAbstractFeatureIterator pointers, but this wrapper allows us to use
it as a local variable, so it gets destroyed automatically when it
gets out of scope, not requiring an explicit "delete" call that would
be necessary with a pointer (easy to forget, right?)


5. Access to features

Vector layer and provide