Hi Camilo, Hi all,
First of all congratulation for your GSoC project !
Some apologies for the delay in my response, we are approaching the next
OTB release so I'm busy on fixing little stuff/builds here and there.
Please don't expect real time answers in the next week or two.
But it's nice to see you left the starting blocks a while ago :)
For OTB we are developing a "module" framework also, with auto-generated
interface. As I told people at the HF, we have made a first step with
our last 3.8 release, but are not satisfied enough with the GUI part, so
we are currently revamping the core to add a lot of functionnalities.
Our ideas/wish-list is summed up here :
http://wiki.orfeo-toolbox.org/index.php/OTB-Wrapper
The current code for what we want to push into QGis is here (please note
that it is in development, i've completed the first usefull app just
after the HackFest) :
http://hg.orfeo-toolbox.org/OTB-Wrapper
Don't hesitate to ask me for help to build it if you need.
Some general comments :
[1]
We should be very careful about not being limited by "too much genericity".
The SAGA and GRASS modules have each their own design/parameter
types/inner but also outer logic.
On the OTB side, we have some liberty since we are currently working on
them, but we have some plans to integrate them in other tools, possibly
benefiting from their additional possibilities (the pipeline
capabilities of Monteverdi for example).
We have some ideas for making the GUI as dynamic as possible (and it is
with their Qt/Qgis implementation that we plan to have the most
full-featured GUI) but I guess other libs will have capabilities that we
don't have.
Trying to fit everyone in the same framework will be hard : either each
library (SAGA/GRASS/OTB) will only use a subset of the provided
functionnalities, either we will end up taking the common factor of all
libs and everyone will feel limited. I'm worried than we fall into one
of these two pitfalls. Please do not impose to much constraints !
Reactions very welcome on this point !
What seems reasonable to me at first sight :
- using similar widgets for similar functionnalities/parameter types,
maybe with base classes to reuse (but again make them really high level !).
- having a common way to call the modules
- all in all, make them all look the same
This will give the user the illusion that they all comes from the same
library, but will give each implementation the necessary liberty to fine
tune their GUIs using the specificities of their library/internal
"module" framework.
We have the chance of being in a similar phase of development for both
project, so i'm sure we will find the right common ground.
By the way, after SAGA, Grass and OTB, maybe someone will consider Ossim
: they also have a similar framework for modules with autogenerated
interfaces, to do raster processing.
Seeing Maxim presenting a Qgis classification plugin based on OpenCV,
maybe you will also end up with some OpenCV modules in the future...
[2]
Chaining modules from different libraries seems a little ambitious to
me, but this would be awesome !
As you said Camilo, I think we can leave pipelining out as a first step.
We are not really focusing on that part now neither, though the current
implementation should allow this.
[3]
You seem to consider parameter validation only in an independent way
From our experience it is insufficient (for example, absolute default
values is not enough).
How we plan to solve this (trying to keep it simple also...), is to give
our modules 3 main entry points :
- description of the parameters
- a global method to update all the parameters, called as soon as one
parameter is changed
- an execution method
Let's imagine a simple orthorectification application with the following
params :
- input/output image
- the output CRS
- output origin, size, and spacing
We want the origin, size and spacing to be auto-computed each time the
input or CRS changes
If CRS changes from UTM to WGS84, we expect the outputorigin
units/display to also change from meter to degree, for example.
[4] (related to [3])
Our first working-and-usefull application in this new framework we're
building shows also dynamic parameters list in the GUI (see the
screenshot on the Wiki).
It is a smoothing application with 3 modes: Median, Gaussian,
Anisotropic Filtering.
Each mode has associated params. Median and Gaussian needs a radius,
Anisotropic Filtering needs a time step and a number of iterations.
So below the combobox, the widget displayed depend on the combobox value.
[5]
The model we imagined for the parameter list will be a tree more than a
flat list.
A group of parameters is just one node of the tree, containing a
parameters list, where each parameter can also be a group of parameters.
Please find also some other more specific comments below :
Le 27/04/2011 11:20, Martin Dobias a écrit :
Hi Camilo
On Tue, Apr 26, 2011 at 9:46 PM, Camilo Polymeris<[email protected]> wrote:
Hello everyone,
I was accepted as a student in GSoC to implement a "SAGA interface for QGIS".
First of all congratulations to you and to other accepted students for
the Summer of Code programme!
There are also proyects that try to interface QGIS with GRASS and the
Orfeo Toolbox (OTB). Paolo has informed me that the creation of a
common framework for those systems was discussed and approved during
HF. A very interesting idea, IMO, that completely changes the way I
would have to work on my project.
Great that you like the idea of generic framework for processing tools
in QGIS. Helping us to bring this framework from ideas into code would
be another great benefit of your project.
While I didn't participate in those discussions and don't know what
conclusions were drawn, as a brainstorming exercise I am writing a
short description of what a very simple API to that framework could
look like:
https://github.com/polymeris/qgis/wiki/QGIS-Processing-Framework
Would that more or less fit what you have in mind? Please comment/criticize.
The discussion was not going into implementation details, we were
mostly collecting the ideas how the things should work.
Framework implementation: as you write in the proposal, ideally it
should be implemented in C++ with Python bindings. I would suggest you
to start the implementation in Python and skip C++ coding at least at
the beginning. My experience is that Python is great for this kind of
prototyping (no compiling, shorter code etc). If you will end up with
sufficient amount of time at the end of the summer, you can move that
into C++ and add Python bindings. The idea is not to waste time with
writing and rewriting various c++ classes when the API is not clear
yet. Consider this as a suggestion, not a command how to do it :-)
Forms/Parameters: I would start with with generic input fields (line
edit / spin box) for integers, floats, strings that can be constrained
with a range (numbers) or regular expressions (strings) - easily
achieved with QValidator classes. Then some custom widgets can be
added for better handling of some types: layer selector, enumeration
etc. The framework may be extensible so that if a library often uses a
particular custom type, the plugin integrating the library could add a
custom widget for it.
Please see [3]
I fear it will be too limiting to consider parameters only independently
Advanced parameters: some modules may allow users set lots of
parameters. Although this is handy, many times the default values work
well and only few basic parameters are changed regularly. It should be
possible to mark some parameters as advanced and they would be hidden
by default, they would show up after clicking a checkbox or a button.
For example, when creating a vector buffer, the distance would be a
basic parameter while the number of segments per quadrant would be
advanced parameter.
Yes basic/advanced parameters seems a must-have to me.
We want not experienced users to use the modules, but we surely want to
give all the possible finetuning possibilities to those who "know what
they're doing"
We would also want checkable parameters.
mandatory parameters should have a value.
non-mandatory parameters would have a checkbox to be activated.
Categorization: currently we are starting to face some trouble with
organization of plugins and the "modules" they provide. We thought
about creating fixed categories as in GRASS (e.g. Raster, Vector,
Database) where the modules would be placed. We also considered custom
categories and subcategories. The general problem with categories is
that many modules do not suit them well if they are not coming from a
well-organized library such as grass, saga or otb.
The conclusion that I recall from the discussion at hackfest is that
tagging is probably the only way to go: each module could have several
tags assigned to it, then the list of modules would be a tree with top
level items containing tags and modules would be shown inside these
tags. Bigger libraries requiring deeper hierarchy might come with
hierarchical tags like "SAGA/Grid - Tools".
Hierarchical tags seems very reasonable to me.
The names of tags will be
left for developers' creativity.
A list of default available tags should be provided.
If the developer feel it can't fit in those, then it can create a new tag.
This list of processing modules should later get some advanced
functionality like searching, favourite modules or even attaching most
used modules on a toolbar, but that is something for the future, we
are not expecting you to do that during the summer :-) A simple tree
listing tags and modules assigned to them will suffice.
Agreed.
Formats and format conversion: for import and export of map layers the
framework should support any layer loaded (or loadable) by QGIS. In
case the library uses a different format for input/output, it should
take care of import/export.
I'm not sure this can be handled in the framework, but more in the
different implementations.
The input images to our modules will be OTB specific data objects, and
the transformation from QGis raster (= filename) to OTB raster would be
done the OTB-QGis plugins implementation (you can check the current
Smoothing application code).
Running modules from scripts: you probably have that in mind but let's
keep it explicit: the framework should provide API that would allow
user to run modules from python console or from a plugin, QGIS would
basically act just as a default GUI using that API. This will finally
allow users to run the processing modules in batches - poor man's
pipelining :-)
Feedback: many modules are able to return some feedback - either in
the form of a progress to indicate how long the process will take - or
to return some messages that will be shown to user. Support for
cancelling a running module would be helpful too, but not really
necessary right now.
Must-have also !
Having a common framework to report progress and log messages would be
great.
If its in Qt, it should be through signals.
I already saw some code making a bridge between the OTB events framework
and Qt signals, I'd like to reuse this.
Regards
Martin
_______________________________________________
Qgis-developer mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
Qgis-developer mailing list
[email protected]
http://lists.osgeo.org/mailman/listinfo/qgis-developer
