Re: [Qgis-developer] Status of Q3 migration?

[Neumann, Andreas]

Hi, 

I would also prefer a double dev cycle, followed by a double test
period. So just shift the one month from the first dev cycle towards the
end. 

In Bonn we discussed this and I think the double dev cycle was an
option, but we said we would decide in December/January when we know
more about the state of the 3.0. 

I had a look at the past couple meeting minutes and apparently there
haven't been any decisions in the minutes about the release date of QGIS
3.0. 

Ultimately it is Jürgen who decides as the release manager. 

Greetings, 

Andreas 

On 2016-12-06 00:02, Nyall Dawson wrote:

> On 3 December 2016 at 19:51, Paolo Cavallini  wrote: 
> 
>> Hi all,
>> I'm regularly compiling QGIS master, and I'm surprised how usable and
>> smooth it is - obviously not suitable for production, but apparently not
>> so far from it. It would be great if those more involved with the
>> migration could give us a quick overview:
>> * are there blocking factors?
>> * which are the main areas which require much work?
>> * are the deadlines proposed originally feasible
>> * how can we speed up the migration?
> 
> Hmm I've brought up previously on the list my thoughts that we
> should *extend* the timeline to mid next year, as opposed to speeding
> it up.
> 
> See https://lists.osgeo.org/pipermail/qgis-developer/2016-October/045554.html
> 
> I'm still strongly of the view that extending the timeline is a good
> idea and will give us a much better 3.x series.
> 
> Nyall
> 
>> Thanks a lot.
>> --
>> Paolo Cavallini - www.faunalia.eu [1]
>> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
>> https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
>> ___
>> Qgis-developer mailing list
>> Qgis-developer@lists.osgeo.org
>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

  

Links:
--
[1] http://www.faunalia.eu
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Proposal: Having all Processing providers as independent components

[Paolo Cavallini]

Hi Victor,

Il 06/12/2016 07:33, Victor Olaya ha scritto:
> Let me know what you think

I like the idea of modularity. However, requiring the user to do extra
steps to use GRASS, SAGA, GDAL/OGR, that are already shipped with our
packages, seems a bit of hindrance.
All the best.
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Proposal: Having all Processing providers as independent components

[Victor Olaya]

Hi all


Before writing a QEP, I would like to discuss quickly here one idea
about Processing, that I think could be interesting and feasible for
3.0

Currently, Processing has some “core” providers (those that are part
of the Processing code itself and are installed with QGIS), ranging
from basic ones such as GDAL or the provider with native QGIS
algorithms, to more uncommon ones, such as LiDAR tools or TauDEM.

Apart from that, more algorithms can be added to the Processing
toolbox, via separate plugins.

To homogenize that and improve user experience, it could be good to
have all those providers as separate plugins, so a bare-bones
installation of Processing would have maybe just the native algorithms
(or native + GDAL), and all remaining ones would be installed by the
users. Installation can be done directly using the plugin manager, or
maybe from Processing itself, if all Processing-based plugins are
conveniently tagged (which is a good idea in any case).

This would have some advantages. Providers would be better organized,
and they could be updated and managed independently. In the future, we
might even think about packaging both the provider itself and the
backend app, which would be a much better user experience. Also, for
beginners it would be easier and less intimidating to have a simpler
toolbox (and with just the algorithms that are more robust and work
always out of the box), and then extend it as needed.

So, in short, I would like to discuss the idea of taking out most of
the providers and have them outside of the core code, so they can be
installed as complements that enhance Processing, instead of some of
them being core and some of them not.

Let me know what you think

Thanks!
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Plugin [995] dzetsaka : Classification tool approval notification.

[noreply]

Plugin dzetsaka : Classification tool approval by pcav.
The plugin version "[995] dzetsaka : Classification tool 2.1" is now approved
Link: http://plugins.qgis.org/plugins/dzetsaka/
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Status of Q3 migration?

[Paolo Cavallini]

Hi all,

Il 06/12/2016 00:02, Nyall Dawson ha scritto:

> Hmm I've brought up previously on the list my thoughts that we
> should *extend* the timeline to mid next year, as opposed to speeding
> it up.
> 
> See https://lists.osgeo.org/pipermail/qgis-developer/2016-October/045554.html
> 
> I'm still strongly of the view that extending the timeline is a good
> idea and will give us a much better 3.x series.

thanks Nyall for your reply. I meant in fact to revive that thread,
thanks for digging it out.
So if I understand correctly we are aiming at:
* feature freeze around May
* release around July
or:
* wait until ready?
I love cleanups, but of course they are never really finished: do we
have a specific goal, i.e. a minimum set of tasks to be finished before
freezing? Could we set some of the issues [0] as blockers?
Sorry to bother, but I believe making the whole process more transparent
to developers, users, and donors will help us going ahead smoothly.
All the best, and thanks for thoughts.

[0]https://github.com/qgis/qgis3.0_api/issues
-- 
Paolo Cavallini - www.faunalia.eu
QGIS & PostGIS courses: http://www.faunalia.eu/training.html
https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] [Qgis-psc] Task Manager has landed!

[Tim Sutton]

Hi Nyall

Wow this is absolutely fantastic! Thanks so much for all your hard work! When 
you have all your social media bits ready let me know and I will make a blog 
post on http://blog.qgis.org - I think it is a really great demonstration of 
how we use the funds donated to QGIS for the good of the project.

Regards

Tim

> On 05 Dec 2016, at 10:04 AM, Nyall Dawson  wrote:
> 
> Hi all,
> 
> Thanks to tons of valuable feedback from the QGIS community and the
> generous funding grant from the QGIS organisation, I've just completed
> and merged the task manager framework branch.
> 
> I'm very keen on feedback from QGIS devs, plugin authors, and users as
> they start the play with this new feature. While the bulk of the work
> is now complete, I'll continue refining this as we had toward 3.0.
> 
> Outstanding tasks include:
> - a PR for documentation of the task manager within the python cookbook
> - some final UI polish and tweaks
> - some blog posts + social media push advertising this feature
> - aiding the community with porting their work to task manager
> 
> I'd like to once again thank the QGIS org for this grant and everyone
> who has donated funds to QGIS which made the grants possible.
> 
> (Personally, I think this work is a great demonstration once again of
> how healthy the QGIS project is. It's involved donations from end
> users, the initiative and leadership of the PSC in creating this grant
> program, the new QGIS voting structure + the associated involvement
> from local user groups, and a PR with nearly 60 comments from
> interested developers in the QGIS community!)
> 
> Nyall
> ___
> Qgis-psc mailing list
> qgis-...@lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-psc

—









Tim Sutton

Co-founder: Kartoza
Project chair: QGIS.org

Visit http://kartoza.com  to find out about open source:

Desktop GIS programming services
Geospatial web development
GIS Training
Consulting Services

Skype: timlinux 
IRC: timlinux on #qgis at freenode.net

Kartoza is a merger between Linfiniti and Afrispatial

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Status of Q3 migration?

[Nyall Dawson]

On 3 December 2016 at 19:51, Paolo Cavallini  wrote:
> Hi all,
> I'm regularly compiling QGIS master, and I'm surprised how usable and
> smooth it is - obviously not suitable for production, but apparently not
> so far from it. It would be great if those more involved with the
> migration could give us a quick overview:
> * are there blocking factors?
> * which are the main areas which require much work?
> * are the deadlines proposed originally feasible
> * how can we speed up the migration?

Hmm I've brought up previously on the list my thoughts that we
should *extend* the timeline to mid next year, as opposed to speeding
it up.

See https://lists.osgeo.org/pipermail/qgis-developer/2016-October/045554.html

I'm still strongly of the view that extending the timeline is a good
idea and will give us a much better 3.x series.

Nyall

> Thanks a lot.
> --
> Paolo Cavallini - www.faunalia.eu
> QGIS & PostGIS courses: http://www.faunalia.eu/training.html
> https://www.google.com/trends/explore?date=all=IT=qgis,arcgis
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing NG/V2 - brainstorming

[Nyall Dawson]

On 6 December 2016 at 05:46, Matthias Kuhn  wrote:
> Thanks for writing this up Nyall,
>
> This is something that I has been on my list of ideas for a long time
> already and that I have never got round to get down to.
>
> I think you have covered the topic quite well already, so I don't think
> there's a lot I have to add right now.

Ah - that reminds me of point 4 which I forgot:


4 - Make more use of iterators in QGIS api

What I mean here is that things like QgsVectorFileWriter,
QgsVectorDataProvider, QgsVectorLayer should all have an addFeatures(
QgsFeatureIterator it... ) method to make it easier to bulk insert
features.

There's also the possibility of adding a "QgsFeatureIteratorProvider"
interface which implements a getFeatures() call. QgsVectorLayer would
then implement the interface, as well as other classes like
QgsFeatureList and QgsFeatureStore, and the processing algorithms.

Nyall



>
> On 12/05/2016 06:41 PM, Alexander Bruy wrote:
>
>> Regarding iterators approach... It sounds interesting, but we need to think
>> also about possibility to work with layers, which are not loaded in QGIS.
>> In this case as I understand we need to construct layer and then create
>> an iterator for Processing from it.
>
> If the algorithm is a QGIS core one, that's correct. But I assume that's
> done already now, so it shouldn't be much of a difference.
>
> For anything else, the iterator approach will not be usable either way
> and it will have to end up in a file anyway. So the model processor
> should be smart enough to handle that.
>
> Case 1:
> qgis alg -> qgis alg
>   iterator
>
> Case 2:
> qgis alg -> external alg
>   find a suitable file format and route the iterator to this file
>
> Case 3:
>   external alg -> external alg
> check if they have a common file type, if yes, use it, if not convert
> between something usable.
>
> If an external file is involved as source, a "file loader" algorithm
> should be transparently inserted to handle conversion to an iterator or
> find the best intermediate format or just pass the file as-is.
>
>> Another idea (obvious and already discussed a bit) is to adopt Processing
>> to use recently added Task Manager, so algs can be executed in background
>> and models can be executed using multiple threads when this is possible.
>
> I think that's something that will happen in any case. We should be able
> to combine the two ideas.
>
> The different possibilities here are probably:
>
> * Always use a file to pass data between algs:
> Heavy on disk I/O and which has performance impacts.
>
> * Use a memory layer to pass data between algs:
> Heavy on memory which may fail with big data (*)
>
> * Iterator approach:
> Performance win. Most complex because it adds a lot of
> task-interdependency, scheduling effort and possibly will still use a
> lot of memory if the consumer algorithm is not fast enough to handle the
> data produced by the generator algorithm. So there should be some kind
> of coordination between the iterator and the producer to pause the
> producer task if the queue gets too big.
>
> Best regards
> Matthias
>
> (*) I always like to use the term big data. It makes me feel like a
> sales person.
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Add virtual layer equivalent in Processing

[Tom Chadwin]

I think qgis-dev is now a zombie, as it's v2 master, where there has nor been
any work for a while. 



-
Buy Pie Spy: Adventures in British pastry 2010-11 on Amazon 
--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/Re-Add-virtual-layer-equivalent-in-Processing-tp5298535p5298722.html
Sent from the Quantum GIS - Developer mailing list archive at Nabble.com.
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Any UI/UX plans for QGIS3?

[Chris Nicholas]

I would really like an option to allow advanced users to manually just SET the 
extents of a vector layer when first adding it, so that a full table-scan can 
be avoided to derive that result.

This still seems to happen, even though the “use estimated metadata” flag is 
checked, when first adding a layer.

Gotta say that I too have built the new QGIS3 from source and it does seem 
quite usable … EXCELLENT work folks!!

thanks -
Chris



> On Dec 2, 2016, at 7:54 PM, Nyall Dawson  wrote:
> 
> 
> 
> On 25 Nov 2016 9:22 PM, "Alexandre Neto"  > wrote:
> Hello all,
> 
> QGIS 3 is getting lots of love, and we are seeing new features being added, 
> some legacy code being clean up, and so on. We are all aware that this will 
> break the 2.x API, but now it's the time for it.
> 
> So I wondered if this is not the perfect time to think about some UI/UX 
> changes as well, as it will lead to more work in translation and 
> Documentation. Could we do a list of UI/UX stuff that we could improve?
> 
> I can think of a few things:
> 
> Consistency
>  - "search", "select" and "browse" for browsing files;
> - "Add saved file to map", "Load into canvas when finished", "Open output 
> file after running algorithm";
> - there is more of these for sure;
> 
> Help and Documentation
> - Most of our help buttons lead to nowhere, we already talked about pointing 
> them to the User's Manual online (although we would need a solution for 
> working with it offline and in different languages)
> 
> Defaults
> - Should default selection color not be opaque?
> - Showing only selected features vertexes by default?
> - Map template, we could provide a very simple template as default (I think 
> there was already some discussion about this); Most people I see gets very 
> confused when they see a completely empty page.
> 
> 
> Here's another idea for a great project t which doesn't involve c++/python: 
> check through and delete unused png icons.
> 
> A lot of the old png icons have been replaced with proper svg versions, but 
> the pngs are still included and in some cases they are incorrectly used 
> instead of the svg replacements.
> 
> Identifying these would be a matter of going though each png icon in the repo 
> and searching through the code (eg via "git grep mactionrefresh.png") to see 
> if they are still used anywhere. (Ignore any hits in the resources files!) . 
> If they aren't they can be safely removed. If they are, you could check 
> whether a suitable svg replacement can be used.
> 
> The more pngs we remove the easier it will be to identify which remaining 
> icons require vector versions.
> 
> Nyall
> 
> 
> Alexandre Neto
> --
> Alexandre Neto
> -
> @AlexNetoGeo
> http://sigsemgrilhetas.wordpress.com 
> http://gisunchained.wordpress.com 
> 
> 
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org 
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer 
> 
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer 
> 
> 
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing NG/V2 - brainstorming

[Matthias Kuhn]

Thanks for writing this up Nyall,

This is something that I has been on my list of ideas for a long time
already and that I have never got round to get down to.

I think you have covered the topic quite well already, so I don't think
there's a lot I have to add right now.

On 12/05/2016 06:41 PM, Alexander Bruy wrote:

> Regarding iterators approach... It sounds interesting, but we need to think
> also about possibility to work with layers, which are not loaded in QGIS.
> In this case as I understand we need to construct layer and then create
> an iterator for Processing from it.

If the algorithm is a QGIS core one, that's correct. But I assume that's
done already now, so it shouldn't be much of a difference.

For anything else, the iterator approach will not be usable either way
and it will have to end up in a file anyway. So the model processor
should be smart enough to handle that.

Case 1:
qgis alg -> qgis alg
  iterator

Case 2:
qgis alg -> external alg
  find a suitable file format and route the iterator to this file

Case 3:
  external alg -> external alg
check if they have a common file type, if yes, use it, if not convert
between something usable.

If an external file is involved as source, a "file loader" algorithm
should be transparently inserted to handle conversion to an iterator or
find the best intermediate format or just pass the file as-is.

> Another idea (obvious and already discussed a bit) is to adopt Processing
> to use recently added Task Manager, so algs can be executed in background
> and models can be executed using multiple threads when this is possible.

I think that's something that will happen in any case. We should be able
to combine the two ideas.

The different possibilities here are probably:

* Always use a file to pass data between algs:
Heavy on disk I/O and which has performance impacts.

* Use a memory layer to pass data between algs:
Heavy on memory which may fail with big data (*)

* Iterator approach:
Performance win. Most complex because it adds a lot of
task-interdependency, scheduling effort and possibly will still use a
lot of memory if the consumer algorithm is not fast enough to handle the
data produced by the generator algorithm. So there should be some kind
of coordination between the iterator and the producer to pause the
producer task if the queue gets too big.

Best regards
Matthias

(*) I always like to use the term big data. It makes me feel like a
sales person.
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing NG/V2 - brainstorming

[Alexander Bruy]

I really like the idea of having basic Processing code in the core library
with Python bindings available.

Regarding iterators approach... It sounds interesting, but we need to think
also about possibility to work with layers, which are not loaded in QGIS.
In this case as I understand we need to construct layer and then create
an iterator for Processing from it.

Another idea (obvious and already discussed a bit) is to adopt Processing
to use recently added Task Manager, so algs can be executed in background
and models can be executed using multiple threads when this is possible.


2016-12-05 11:03 GMT+02:00 Nyall Dawson :
> Hi all,
>
> I've recently been informally chatting about possible enhancements to
> processing with a few QGIS team members, and I thought it'd be worth
> starting a public brainstorm about these ideas.
>
> This is really just "thinking aloud" about what the next logical steps
> are for processing and how we can make it more competitive against
> programs like FME.
>
> So here we go... a bunch of random ideas on future processing enhancements:
>
>
> 1. Rework native algorithms to avoid layer input/outputs
>
> (Full credit goes to Matthias here). One current inefficiency with
> processing models is that every step is exported to a file based
> format, which is then reread in for the next algorithm. This means
> that a simple set of steps like buffer->reproject involves multiple
> conversions from OGR formats to QgsFeature/QgsGeometry and back to the
> OGR output format, when it could be simplified to just two operations
> on a QgsFeature's geometry and then a final write to disk. In addition
> to the inefficiency here we also lose things like long field names and
> full support for z/m/curves (depending on the intermediate file
> format).
>
> So... how to address this... Matthias came up with the idea that
> native processing algs could accept a feature iterator instead of an
> input layer, and themselves be a feature iterator. This effectively
> would make a processing model a chain of iterators which features are
> "pulled" through from a final writer step. Ie, the source
> layer->buffer->save to layer->load layer->reproject->save to layer
> model becomes:
>
> a writer
> -> which reads features from the iterator provided a transform alg
> -> which reprojects the geometry from features provided by a buffer
> alg's output iterator
> -> which buffers the geometry on features from an iterator from the
> original source layer
>
> (Obviously, anytime a non-native algorithm (eg saga/grass/ogr) is used
> then the features would need to be written to disk first. But this is
> no different to the current behaviour so there shouldn't be any extra
> cost incurred.)
>
> This gets a little trickier when we want to multithread something, eg.
> 2 input layers-> each buffered -> intersection of the two. But we
> could handle this by using a form of "pipe" iterator, which sucks in
> features as fast as possible from its input iterator and stores them
> in one thread, and then an algorithm in another thread consumes these
> features as they become available. Ie:
>
> thread a:
> input layer 1 iterator -> buffer 1 alg iterator -> "pipe" iterator a ->
>
>
> thread c: intersection alg
> thread b:
> input layer 2 iterator -> buffer 2 alg iterator  -> "pipe" iterator b ->
>
> where thread c reads the features from "pipe iterator a" and "b" as
> they become available, and then does its processing on them.
>
> (hope that makes sense!)
>
> 2. Georeferenced geometries
>
> I think for the approach in 1 to work we'd also need to introduce the
> concept of "referenced" geometries. This would basically be
> QgsGeometry + a QgsCoordinateReferenceSystem. It would allow retrieval
> of a geometry's CRS without requiring any knowledge of its source
> layer (or where no layer exists, eg the canvas extent as a geometry).
>
> I've pondered several approaches to this, such as:
> - QgsReferencedFeature (QgsFeature + crs): This doesn't work for
> non-feature based geometries or allow features with multiple
> geometries in different CRSes. (See
> https://github.com/qgis/qgis3.0_api/issues/21).
> - QgsReferencedGeometry: subclass of QgsGeometry with a CRS member.
> This approach would avoid adding any extra overhead to QgsGeometry.
> But given that the main use of QgsGeometry (geometry attached to a
> feature from a layer) will always have a CRS associated, this seems
> like it unnecessarily complicates the API.
>
> So my current preference would be for QgsGeometry to gain a
> QgsCoordinateReferenceSystem member variable, which is an invalid crs
> if the geometry is not referenced. This should still be quite
> lightweight given that QgsCoordinateReferenceSystem is implicitly
> shared.
>
>
> 3. Porting components of processing to core
>
> There's demand (from eg QField) to reuse parts of processing outside of 
> PyQGIS.
>
> I think good candidates for porting to core would 

Re: [Qgis-developer] Add virtual layer equivalent in Processing

[Anita Graser]

Thanks Victor!

On Mon, Dec 5, 2016 at 1:54 PM, Victor Olaya  wrote:

> Anita, Mathieu
>
> I added both of your requests and it should be possible now to use
> tables on both "execute SQL" and "build virtual vector" algorithms
>
> Let me know if you try it (it is on master)
>


​As far as I can see OGR's Execute SQL takes only one input layer/table.
Therefore it's not suitable for the task of joining a vector layer with a
table.

QGIS' Build virtual vector takes multiple inputs but I'm not sure what it
does with them. The help text is a bit short: "This algorithm creates a
virtual layer that contains a set of vector layer." That does not sound
like it will do the job either.

QGIS' Execute SQL seems to allow multiple inputs but doesn't list tables
and I didn't see any changes to this algorithm in your recent commits but
maybe I missed something?

Note: I'm running qgis-dev from OSGeo4W which is probably not up to date
with master.

The SQL statement I want to execute (and which runs well in Layer | Add
Layer | Add Virtual Layer) is:

select s_out.y1_statefips, s_out.y2_statefips,
  st_centroid(f.geometry)
from fips as f
join stateoutflow1415 as s_out
on f.fips = s_out.y1_statefips​

​where fips is a polygon layer and stateoutflow1415 is a CSV.

Best wishes,
Anita





​










>
> Cheers
>
> 2016-12-05 11:07 GMT+01:00 Mathieu Pellerin :
> > Victor,
> >
> > I was looking into this; it'd be nice for the "build vector layer"
> algorithm
> > would support/show geometryless layers in the parametermultiinput panel.
> > AFAIK, geometryless layers aren't (yet? :) ) supported by the
> > parametermultiinput.
> >
> > Math
> >
> > On Mon, Dec 5, 2016 at 2:51 PM, Victor Olaya  wrote:
> >>
> >> No, there is no such functionality AFAIK, but it should be easy to add,
> i
> >> think
> >>
> >> Can you open a feature request?
> >>
> >> Also, the execute SQL algorithm can be modified to accept non-spatial
> >> tables. That makes sense to me
> >>
> >> Cheers
> >>
> >> 2016-12-04 22:28 GMT+01:00 Anita Graser :
> >> > Hi,
> >> >
> >> > Since the user list doesn't seen to have an answer, I'd like to ask
> >> > here:
> >> > Is there an equivalent to the "add virtual layer" functionality in the
> >> > Processing toolbox?
> >> >
> >> > I want to join multiple csv files to a vector layer (and at the same
> >> > time
> >> > build a new geometry WKT). This seems most easily done with a virtual
> >> > layer.
> >> > But I need to script this workflow.
> >> >
> >> > In the Processing toolbox, I only found the "execute SQL" tool but it
> >> > does
> >> > not allow selection of non-spatial tables as input.
> >> >
> >> > Best wishes,
> >> > Anita
> >> >
> >> >
> >> >
> >> > ___
> >> > Qgis-developer mailing list
> >> > Qgis-developer@lists.osgeo.org
> >> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> ___
> >> Qgis-developer mailing list
> >> Qgis-developer@lists.osgeo.org
> >> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> >
> >
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Geo-Information with QGIS

[a.dasilvamano]

Dear all,



The Faculty ITC [1] of the University of Twente, The Netherlands, is starting a 
9 month distance course in Geo-Information Science and Earth Observation (15 EC 
spread over a 9-month period)  that can be taken solo or as a first step to a 
full MSc degree..



A decision was made to use QGIS (or associated providers) designed exercises to 
explore the concepts of Geo-Information Science. We believe this will 
contribute decisively for the quality of the course, and therefore appreciate 
if you can share this among your contacts.



The course consists of three 10-week separate but connected 5 EC modules. The 
course will extend from February to November 2017 in part-time (14 hours a 
week) modality. Enrollments are accepted until the 6th of January 2017



Details about content and enrollments can be found at[2].

Alternately, you can also inquire Andre Mano 
(a.dasilvam...@utwente.nl)



On behalf of ITC Faculty,

Andre Mano



[1] https://www.itc.nl

[2] 
https://www.itc.nl/C17-EDU-DE-01?utm_campaign=touch_medium=email_source=DECoreFeb2017intake_content=QGIS

___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Add virtual layer equivalent in Processing

[Victor Olaya]

Anita, Mathieu

I added both of your requests and it should be possible now to use
tables on both "execute SQL" and "build virtual vector" algorithms

Let me know if you try it (it is on master)

Cheers

2016-12-05 11:07 GMT+01:00 Mathieu Pellerin :
> Victor,
>
> I was looking into this; it'd be nice for the "build vector layer" algorithm
> would support/show geometryless layers in the parametermultiinput panel.
> AFAIK, geometryless layers aren't (yet? :) ) supported by the
> parametermultiinput.
>
> Math
>
> On Mon, Dec 5, 2016 at 2:51 PM, Victor Olaya  wrote:
>>
>> No, there is no such functionality AFAIK, but it should be easy to add, i
>> think
>>
>> Can you open a feature request?
>>
>> Also, the execute SQL algorithm can be modified to accept non-spatial
>> tables. That makes sense to me
>>
>> Cheers
>>
>> 2016-12-04 22:28 GMT+01:00 Anita Graser :
>> > Hi,
>> >
>> > Since the user list doesn't seen to have an answer, I'd like to ask
>> > here:
>> > Is there an equivalent to the "add virtual layer" functionality in the
>> > Processing toolbox?
>> >
>> > I want to join multiple csv files to a vector layer (and at the same
>> > time
>> > build a new geometry WKT). This seems most easily done with a virtual
>> > layer.
>> > But I need to script this workflow.
>> >
>> > In the Processing toolbox, I only found the "execute SQL" tool but it
>> > does
>> > not allow selection of non-spatial tables as input.
>> >
>> > Best wishes,
>> > Anita
>> >
>> >
>> >
>> > ___
>> > Qgis-developer mailing list
>> > Qgis-developer@lists.osgeo.org
>> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> ___
>> Qgis-developer mailing list
>> Qgis-developer@lists.osgeo.org
>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing NG/V2 - brainstorming

[Bernhard Ströbl]

 PyQGIS.

I think good candidates for porting to core would be:
- parameters
- inputs
- the algorithm base class

In addition to allowing use outside of python this would also help
strengthen these components by the static typing which would result of
porting to c++.

I'd also like to see the results + history dialogs merged and moved to
core so that they can also be reused for non-processing tasks (eg
composer exports).



So there we go. What's everyone's thoughts? Are these ideas worth
pursing? Is there other things we should be looking at investigation
for future processing enhancements?

Nyall
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer





__ Information from ESET Mail Security, version of virus signature 
database 14553 (20161205) __

The message was checked by ESET Mail Security.
http://www.eset.com


___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Processing NG/V2 - brainstorming

[johnrobot]

Hi
I would very much like to see improvements for Processing. A few additional
ideas:

- Display the number of features going through the model. This would help
the user understand the data better.

- Add comments to model components without opening each component. A chance
to explain why a specific tool is used.

- Support clustering/grouping for a set of model components that together
perform a task. Placing a few components inside a visual box would make the
model more structured and easier to understand.

- A server side component where users can log in and run a model/job, such
as extracting data for a selected area and downloading it in a format that
suits the user. This environment should also provide an admin interface, log
files, scheduled jobs etc.

- Watch a file/folder/database table and run the model when a change is
detected. Option for sending the user an email when the job is completed.



Magnus




--
View this message in context: 
http://osgeo-org.1560.x6.nabble.com/Processing-NG-V2-brainstorming-tp5298579p5298592.html
Sent from the Quantum GIS - Developer mailing list archive at Nabble.com.
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] The icon has Santa's cap ...

[Nathan Woodrow]

Klaus,

You can see this post on how to replace them if you want.

http://gis.stackexchange.com/questions/220138/removing-santa-hat-from-qgis-icon/220222#220222

Regards,
Nathan

On Mon, Dec 5, 2016 at 7:32 PM, Mithöfer  wrote:

> Dear group,
> not everybody was happy about it. We have cusumer who are a bit
> irritated asking where it suddenly came from (and what other
> changes were applied...).
>
> Regards,
> Klaus
>
> Am Thu, 1 Dec 2016 12:10:12 +0100
> schrieb Geo DrinX :
>
> > This morning, I had the surprise of see that the QGIS icon has the
> > Santa Claus red cap.
> >
> > Do you noticed ?
> >
> > Who had this nice idea ?   :)
> >
> >
> > Great !   Perhaps, we will have some other new ?
> >
> >
> > Regards and Happy new year (in advance)
> >
> > Geo
>
>
>
> --
> Geoinformatikbüro Dassau GmbH
> Rethelstrasse 153
> D - 40237 Düsseldorf
> Tel: +49-211-69937750
> Fax: +49-211-69937751
> Mobile  +49-171-4687540
> http://www.gbd-consult.de
>
> Registergericht: Amtsgericht Düsseldorf, HR B 74022
> Geschäftsführer: Otto Dassau
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Add virtual layer equivalent in Processing

[Mathieu Pellerin]

Victor,

I was looking into this; it'd be nice for the "build vector layer"
algorithm would support/show geometryless layers in the parametermultiinput
panel. AFAIK, geometryless layers aren't (yet? :) ) supported by the
parametermultiinput.

Math

On Mon, Dec 5, 2016 at 2:51 PM, Victor Olaya  wrote:

> No, there is no such functionality AFAIK, but it should be easy to add, i
> think
>
> Can you open a feature request?
>
> Also, the execute SQL algorithm can be modified to accept non-spatial
> tables. That makes sense to me
>
> Cheers
>
> 2016-12-04 22:28 GMT+01:00 Anita Graser :
> > Hi,
> >
> > Since the user list doesn't seen to have an answer, I'd like to ask here:
> > Is there an equivalent to the "add virtual layer" functionality in the
> > Processing toolbox?
> >
> > I want to join multiple csv files to a vector layer (and at the same time
> > build a new geometry WKT). This seems most easily done with a virtual
> layer.
> > But I need to script this workflow.
> >
> > In the Processing toolbox, I only found the "execute SQL" tool but it
> does
> > not allow selection of non-spatial tables as input.
> >
> > Best wishes,
> > Anita
> >
> >
> >
> > ___
> > Qgis-developer mailing list
> > Qgis-developer@lists.osgeo.org
> > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] The icon has Santa's cap ...

[Mithöfer]

Dear group,
not everybody was happy about it. We have cusumer who are a bit
irritated asking where it suddenly came from (and what other
changes were applied...). 

Regards,
Klaus

Am Thu, 1 Dec 2016 12:10:12 +0100
schrieb Geo DrinX :

> This morning, I had the surprise of see that the QGIS icon has the
> Santa Claus red cap.
> 
> Do you noticed ?
> 
> Who had this nice idea ?   :)
> 
> 
> Great !   Perhaps, we will have some other new ?
> 
> 
> Regards and Happy new year (in advance)
> 
> Geo



-- 
Geoinformatikbüro Dassau GmbH 
Rethelstrasse 153
D - 40237 Düsseldorf
Tel: +49-211-69937750
Fax: +49-211-69937751
Mobile  +49-171-4687540
http://www.gbd-consult.de

Registergericht: Amtsgericht Düsseldorf, HR B 74022
Geschäftsführer: Otto Dassau
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Processing NG/V2 - brainstorming

[Nyall Dawson]

Hi all,

I've recently been informally chatting about possible enhancements to
processing with a few QGIS team members, and I thought it'd be worth
starting a public brainstorm about these ideas.

This is really just "thinking aloud" about what the next logical steps
are for processing and how we can make it more competitive against
programs like FME.

So here we go... a bunch of random ideas on future processing enhancements:


1. Rework native algorithms to avoid layer input/outputs

(Full credit goes to Matthias here). One current inefficiency with
processing models is that every step is exported to a file based
format, which is then reread in for the next algorithm. This means
that a simple set of steps like buffer->reproject involves multiple
conversions from OGR formats to QgsFeature/QgsGeometry and back to the
OGR output format, when it could be simplified to just two operations
on a QgsFeature's geometry and then a final write to disk. In addition
to the inefficiency here we also lose things like long field names and
full support for z/m/curves (depending on the intermediate file
format).

So... how to address this... Matthias came up with the idea that
native processing algs could accept a feature iterator instead of an
input layer, and themselves be a feature iterator. This effectively
would make a processing model a chain of iterators which features are
"pulled" through from a final writer step. Ie, the source
layer->buffer->save to layer->load layer->reproject->save to layer
model becomes:

a writer
-> which reads features from the iterator provided a transform alg
-> which reprojects the geometry from features provided by a buffer
alg's output iterator
-> which buffers the geometry on features from an iterator from the
original source layer

(Obviously, anytime a non-native algorithm (eg saga/grass/ogr) is used
then the features would need to be written to disk first. But this is
no different to the current behaviour so there shouldn't be any extra
cost incurred.)

This gets a little trickier when we want to multithread something, eg.
2 input layers-> each buffered -> intersection of the two. But we
could handle this by using a form of "pipe" iterator, which sucks in
features as fast as possible from its input iterator and stores them
in one thread, and then an algorithm in another thread consumes these
features as they become available. Ie:

thread a:
input layer 1 iterator -> buffer 1 alg iterator -> "pipe" iterator a ->


thread c: intersection alg
thread b:
input layer 2 iterator -> buffer 2 alg iterator  -> "pipe" iterator b ->

where thread c reads the features from "pipe iterator a" and "b" as
they become available, and then does its processing on them.

(hope that makes sense!)

2. Georeferenced geometries

I think for the approach in 1 to work we'd also need to introduce the
concept of "referenced" geometries. This would basically be
QgsGeometry + a QgsCoordinateReferenceSystem. It would allow retrieval
of a geometry's CRS without requiring any knowledge of its source
layer (or where no layer exists, eg the canvas extent as a geometry).

I've pondered several approaches to this, such as:
- QgsReferencedFeature (QgsFeature + crs): This doesn't work for
non-feature based geometries or allow features with multiple
geometries in different CRSes. (See
https://github.com/qgis/qgis3.0_api/issues/21).
- QgsReferencedGeometry: subclass of QgsGeometry with a CRS member.
This approach would avoid adding any extra overhead to QgsGeometry.
But given that the main use of QgsGeometry (geometry attached to a
feature from a layer) will always have a CRS associated, this seems
like it unnecessarily complicates the API.

So my current preference would be for QgsGeometry to gain a
QgsCoordinateReferenceSystem member variable, which is an invalid crs
if the geometry is not referenced. This should still be quite
lightweight given that QgsCoordinateReferenceSystem is implicitly
shared.


3. Porting components of processing to core

There's demand (from eg QField) to reuse parts of processing outside of PyQGIS.

I think good candidates for porting to core would be:
- parameters
- inputs
- the algorithm base class

In addition to allowing use outside of python this would also help
strengthen these components by the static typing which would result of
porting to c++.

I'd also like to see the results + history dialogs merged and moved to
core so that they can also be reused for non-processing tasks (eg
composer exports).



So there we go. What's everyone's thoughts? Are these ideas worth
pursing? Is there other things we should be looking at investigation
for future processing enhancements?

Nyall
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [Qgis-developer] Task Manager has landed!

[Victor Olaya]

Nice!

Definitely, a major step ahead, and will be very important for making
Processing more responsive

thanks for your great work!

2016-12-05 9:04 GMT+01:00 Nyall Dawson :
> Hi all,
>
> Thanks to tons of valuable feedback from the QGIS community and the
> generous funding grant from the QGIS organisation, I've just completed
> and merged the task manager framework branch.
>
> I'm very keen on feedback from QGIS devs, plugin authors, and users as
> they start the play with this new feature. While the bulk of the work
> is now complete, I'll continue refining this as we had toward 3.0.
>
> Outstanding tasks include:
> - a PR for documentation of the task manager within the python cookbook
> - some final UI polish and tweaks
> - some blog posts + social media push advertising this feature
> - aiding the community with porting their work to task manager
>
> I'd like to once again thank the QGIS org for this grant and everyone
> who has donated funds to QGIS which made the grants possible.
>
> (Personally, I think this work is a great demonstration once again of
> how healthy the QGIS project is. It's involved donations from end
> users, the initiative and leadership of the PSC in creating this grant
> program, the new QGIS voting structure + the associated involvement
> from local user groups, and a PR with nearly 60 comments from
> interested developers in the QGIS community!)
>
> Nyall
> ___
> Qgis-developer mailing list
> Qgis-developer@lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer

[Qgis-developer] Task Manager has landed!

[Nyall Dawson]

Hi all,

Thanks to tons of valuable feedback from the QGIS community and the
generous funding grant from the QGIS organisation, I've just completed
and merged the task manager framework branch.

I'm very keen on feedback from QGIS devs, plugin authors, and users as
they start the play with this new feature. While the bulk of the work
is now complete, I'll continue refining this as we had toward 3.0.

Outstanding tasks include:
- a PR for documentation of the task manager within the python cookbook
- some final UI polish and tweaks
- some blog posts + social media push advertising this feature
- aiding the community with porting their work to task manager

I'd like to once again thank the QGIS org for this grant and everyone
who has donated funds to QGIS which made the grants possible.

(Personally, I think this work is a great demonstration once again of
how healthy the QGIS project is. It's involved donations from end
users, the initiative and leadership of the PSC in creating this grant
program, the new QGIS voting structure + the associated involvement
from local user groups, and a PR with nearly 60 comments from
interested developers in the QGIS community!)

Nyall
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer