Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Tue, Jul 29, 2014 at 11:00 PM, Glynn Clements
gl...@gclements.plus.com wrote:
Markus Neteler wrote:
Can you try the attached patch? The new function should be used like:
if (setjmp(*G_fatal_longjmp(1))) {
// this will be executed on fatal errors
}
Works. Can I hope the patch will find a way to 7.0.0?
Committed in r60590.
Glynn, should I backport that to relbranch70?
I suppose so.
Done in r61451 (relbranch70).
Markus
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, May 30, 2014 at 12:38 AM, Glynn Clements
gl...@gclements.plus.com wrote:
Radim Blazek wrote:
Can you try the attached patch? The new function should be used like:
if (setjmp(*G_fatal_longjmp(1))) {
// this will be executed on fatal errors
}
Works. Can I hope the patch will find a way to 7.0.0?
Committed in r60590.
Glynn, should I backport that to relbranch70?
Markus
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, May 30, 2014 at 12:38 AM, Glynn Clements
gl...@gclements.plus.com wrote:
Radim Blazek wrote:
Can you try the attached patch? The new function should be used like:
if (setjmp(*G_fatal_longjmp(1))) {
// this will be executed on fatal errors
}
Works. Can I hope the patch will find a way to 7.0.0?
Committed in r60590.
The only aspect which might be invasive is that defs/gis.h now
includes setjmp.h for the jmp_buf declaration. Although this is a
standard C header, it's not necessarily as straightforward as other
headers, being tied to the compiler's internals. I would advise
checking whether there are any build issues on less-common platforms
(e.g. AIX) before promoting the change to release.
I have patched my relbr7 locally on the AIX supercomputer and it compiles.
Markus
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, May 30, 2014 at 9:33 PM, Markus Neteler nete...@osgeo.org wrote: On Fri, May 30, 2014 at 12:38 AM, Glynn Clements ... Committed in r60590. The only aspect which might be invasive is that defs/gis.h now includes setjmp.h for the jmp_buf declaration. Although this is a standard C header, it's not necessarily as straightforward as other headers, being tied to the compiler's internals. I would advise checking whether there are any build issues on less-common platforms (e.g. AIX) before promoting the change to release. I have patched my relbr7 locally on the AIX supercomputer and it compiles. I should have added that I use gcc there. The change should be tested with compilers as well. Markus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Wed, May 28, 2014 at 1:49 AM, Glynn Clements
gl...@gclements.plus.com wrote:
But I still think that the global error handler is the wrong place for
a longjmp(). Its original purpose was to allow the standard
notification mechanism (stderr, log file, and/or email) to be replaced
with a custom mechanism.
Can you try the attached patch? The new function should be used like:
if (setjmp(*G_fatal_longjmp(1))) {
// this will be executed on fatal errors
}
Works. Can I hope the patch will find a way to 7.0.0?
Thanks
Radim
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Sat, May 17, 2014 at 7:16 PM, Glynn Clements gl...@gclements.plus.com wrote: Also, currently the global error handler (G_set_error_routine) is called before the non-exclusive handlers (G_add_error_handler), so those will never be called if the global handler lonjmp()s out. The global error handler isn't limited to fatal errors, but is also used for messages and warnings. G_add_error_handler() is used by GRASS itself? The raster library installs a handler which calls Rast__unopen_all(). The vector and DBMI libraries provide interfaces to it, although I don't know how (or if) these are used at present. It is desired that Rast__unopen_all() is not called before longjmp(), I think. I don't want to close all maps successfully opened before. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Apr 18, 2014 at 12:27 PM, Paolo Cavallini cavall...@faunalia.it wrote: Thanks for this. If I understand well, this means: * we can load GRASS7 vectors through QGIS browser * we cannot load GRASS7 rasters Raster provider is also upgraded (bf6effdd). * the qgis-grass-plugin is not functional with GRASS7 Left for 2.6. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Dne 16.5.2014 18:36 Glynn Clements gl...@gclements.plus.com napsal(a): Radim Blazek wrote: There is a problem in G_fatal_error(): static int busy if (busy) exit(EXIT_FAILURE); busy = 1; second G_fatal_error() call always exits. The busy check is there for the case when print error routine itself ends up with call to G_fatal_error()? It's there to prevent infinite recursion, whether from the printing code, a fatal error handler, or whatever. May be the busy variable made global and added a function to clear it? That might be going a bit too far. OTOH, it can't be cleared prior to calling the handler because it's intended to protect against the case of a handler generating a fatal error. I ment some G_fatal_error_recover() which would be called as last thing from QGIS error routine. One possibility would be to add explicit support for longjmp()ing out, e.g. adding G_error_longjmp(jmp_buf env, int val) which would clear the busy flag then perform the longjmp. That would avoid exposing the internals more than is necessary. Sounds good. Also, currently the global error handler (G_set_error_routine) is called before the non-exclusive handlers (G_add_error_handler), so those will never be called if the global handler lonjmp()s out. The global error handler isn't limited to fatal errors, but is also used for messages and warnings. G_add_error_handler() is used by GRASS itself? Perhaps we need a dedicated fatal error handler for this purpose? It would be called immediately prior to the final exit() call. It would not be allowed to call GRASS functions or to return. The busy flag could be cleared before calling it. Whatever allows to recover from fatal error will help. I would prefer however not changing semantics too much so that GRASS 6, 7.0.0beta1 and 7.x.x may be easily supported by the same code with only few ifdefs. Radim -- Glynn Clements gl...@gclements.plus.com ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Sat, Apr 19, 2014 at 12:43 AM, Glynn Clements gl...@gclements.plus.com wrote: 3) use setjmp()/longjmp() That is the sane option. But bear in mind that subsequently calling any GRASS function is entirely at your own risk. A call to G_fatal_error() often means that internal data structures will contain garbarge. If you want to submit patches to perform clean-up in the event of fatal errors, they will presumably be accepted. Failing that, it may be advisable to have G_fatal_error() zero-out certain core data structures (e.g. R__) so that people don't make the mistake of trying to carry on as if nothing happened. There is a problem in G_fatal_error(): static int busy if (busy) exit(EXIT_FAILURE); busy = 1; second G_fatal_error() call always exits. The busy check is there for the case when print error routine itself ends up with call to G_fatal_error()? May be the busy variable made global and added a function to clear it? Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 23/04/2014 15:12, Radim Blazek ha scritto: I am not even trying to ask that question. As long as there are GRASS users using the QGIS plugin, it should not be thrown away. agreed fully, of course. The issue is: how much work is required for this? and: who is goig to invest on it? There is the GRASS Direct lib which allows GRASS raster modules to read/write data through QGIS providers. I is currently disabled, needs some fixes but it was mostly working. If fixed, it can be easily used in Processing, I think. I think rasters are not a big issue (AFAIK they are read through v.external, so no import); vectors seem more diffucult to handle though. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Wed, Apr 23, 2014 at 4:02 PM, Sören Gebbert Vector lib does not call G_fatal_error(), it was written in parallel with QGIS plugin with vector editing in mind. I hope it is still true. I fear that might be not the case anymore. For example, all memory allocation functions G_calloc(), G_malloc() and G_realloc() will call G_fatal_error() in case the memory allocation fails. As far as i can tell the vector library make often use of this functions. So we have the problem that basic core functions call G_fatal_error() that are used all over the place in GRASS GIS. I don't care about allocation error. If it really runs out of memory then exit() is maybe the best. Any other action (saving project, saving changes) will also need memory and thus fail. Are you often running out of memory? It made me curious how Qt handles allocation problem, because I think that allocation results are not checked in the whole QGIS at all. For example QString::realloc(int) on the pointer returned by malloc() calls Q_CHECK_PTR() macro which prints error if pointer is 0. Then it continues execution even if allocation failed. If you patch GRASS to use setjmp/longjmp to implement an exception like behavior then you need to write clean up code for static variables and dynamically allocated memory. A G_fatal_error() call can be deeply nested in a library function. Cleaning only the static variables to be in usable state will lead to problematic memory leaks. So it is a complex task in GRASS. I don't want to patch GRASS until it is really inevitable. The next one is my choice: Using setjmp/longjmp in QGIS wrapping all GRASS functions that might call G_fatal_error() is complex, since a G_fatal_error() call can be deeply nested in a library function. You need to keep track of this resulting in plenty setjmp wrappers around GRASS functions. setjmp will be called just before the top level function we need. You need to cleanup global static structures as well and you will have no control over pointers pointing to dynamic allocated memory that got lost in a longjmp. and will not prevent QGIS from terminating or segfaulting eventually in case of a fatal error. Why? Because of the undefined state of global variables and structures and because of potential memory leaks that may sum up making QGIS unusable. In theory maybe, in practice it will work and it DOES work with GRASS 6. I don't care about few bytes lost. G_fatal_error() is really rare in QGIS provider/plugin, it should be rare. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 22/04/2014 15:55, Radim Blazek ha scritto: Thanks for this. If I understand well, this means: * we can load GRASS7 vectors through QGIS browser * we cannot load GRASS7 rasters * the qgis-grass-plugin is not functional with GRASS7 Yes. We are talking about current master, not about intended final implementation. BTW, GRASS6 vectors apparently cannot be loaded from QGIS browser as a whole. This, coupled with several serious regressions I found, may mean that GRASS support in QGIS is essentially broken, You are talking about GRASS 6 support? What are serious regressions? * creating a new location is partly broken (cannot take the extent from the canvas, at least for some projections); a minor issue, but a serious concern for new users * the region resolution cannot be changed interactively. The Processing plugin may substitute GRASS Tools part of the plugin (i.e. modules GUI), it cannot help with: - mapset creation - vector/raster maps visualization - vector digitizing - region visualization and editing The Processing plugin is good for users who don't want to use GRASS data format at all. The GRASS plugin is GRASS GUI alternative for true GRASS users. Yes, got it. The question is: what is the real advantage to use grass alone, instead of jointly with other tols in Processing? I see two major ones: * Processing is blocking the main QGIS canvas, which for very long analyses is unacceptable * import-export overhead is a serious issue for complex vectors. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
* Processing is blocking the main QGIS canvas, which for very long analyses is unacceptable That should be a matter of putting Processing algorithm in a different thread. We had it like that, but it proved to be rather unstable, so it was termporarily disabled, but i hope i can do further work on that and rework that part * import-export overhead is a serious issue for complex vectors. Agree. That seems harder to solve :-( ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
2014-04-20 11:38 GMT+02:00 Martin Landa landa.mar...@gmail.com: Hi, 2014-04-20 1:34 GMT+02:00 Sören Gebbert soerengebb...@googlemail.com: Hence the wrapper will not only do simple function wrapping, it will also implement a higher level interface: [...] what about to create a trac wiki page dedicated to RFC and collect this notes there... ? https://trac.osgeo.org/grass/wiki/Grass7/RPCInterface Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Wed, Apr 23, 2014 at 1:11 AM, Glynn Clements 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions By itself, this will do nothing. I mean to allow to throw exception from the function set by G_set_error_routine(). Does using a C++ exception have any advantage over longjmp()? I don't know, probably no. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Tue, Apr 22, 2014 at 4:49 PM, Sören Gebbert soerengebb...@googlemail.com wrote: IMHO we GRASS developers are too stubborn to change the design of GRASS GIS to work as library with persistent applications. This would require rewriting plenty of core functionality and modification of all C-modules. But there is a solution for persistent applications, called Remote Procedure Call (RPC), more below. It sounds interesting but it seems to be a huge work an far future. I Its far less work then modifying GRASS libraries to run with persistent applications. We have already the providers and the plugin working with GRASS 6, I just want to upgrade it to GRASS 7. The only problem I found so far is that G__get_window() and Rast_get_cellhd() call G_fatal_error(). I can only dedicate few hours/max days of my free time to that, so I have to consider well what is less work, RPC is not. We need such an approach anyway, since vector editing is a GUI task - GUI is a persistent application - we need an exit safe interface to the GRASS libraries to implement a reliable vector editing GUI in GRASS itself. Vector lib does not call G_fatal_error(), it was written in parallel with QGIS plugin with vector editing in mind. I hope it is still true. am also concerned about complexity, e.g. starting / keeping running servers on various platforms. I can imagine endless bug reports about server not running. I am looking for something which will not Since only QGIS is spawning the server process, it is always able to check if the process is running and can restart it when needed. We already make use of this concept in GRASS and it works nicely[1]. Spawning of processes and keeping track of them is an easy task with Qt, and it is completely OS independent. The Qt Bitcoin client for example makes use of RPC and runs on Windows, Mac OS and various Linux systems. So i don't see a problem here. The raster provider is already using something like that. Raster data are read from executed GRASS module output (a module written for that in QGIS in fact) and identify tool keeps another GRASS module running and it is communicating with that through a pipe. I wrote that as an experiment and because I know, how much unnecessary additional code and potential problems it is, I don't want to continue in that direction. require permanent assistance. It is not KISS enough for me. What would be a better KISS solution? IMO the RPC approach will make the implementation of the QGIS-GRASS data provider and the plugin simpler. There will be no mixing of C++ and C-code anymore. We are no starting the plugin from scratch, it is already written, we just need to make it work with GRASS 7. No setjmp/longjmp/exception workaround that is really complex Why? and will not prevent QGIS from terminating or segfaulting eventually in case of a fatal error. Why? Radim The QGIS GRASS data provider/plugin will be a clean C++ implementation using the thrift library. It will be better maintainable since developers do not need to know all the different GRASS libraries (gis, raster, vector, database) but only the RPC interface that make use of a subset of the GRASS library functions, bundling them in an object oriented API. QGIS do not need to link against GRASS libraries anymore, future GRASS versions will be supported out of the box. This is much more KISS then the current approach. Best regards Soeren [1] http://grass.osgeo.org/programming7/namespacepython_1_1temporal_1_1c__libraries__interface.html Radim 2014-04-18 11:31 GMT+02:00 Radim Blazek radim.bla...@gmail.com: I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. Basically more and more low level functions are calling exit() instead of returning error code if something failed. As I am not willing to implement GRASS module call for each simple function, we have to think again about hacks we can use: 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS 3) use setjmp()/longjmp() 4) let QGIS crash whenever GRASS lib function fails I fear that none of these suggestions are good working solutions, since in case a fatal error or a segfault occurs nothing (no exception, no setjmp) will prevent QGIS from crashing. So i would like to suggest the following: 5.) Using a RPC interface for map metadata, vector and raster map access. Hence one or several GRASS server processes provide an RPC interface to access GRASS library functions that are needed in a persistent
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Wed, Apr 23, 2014 at 10:09 AM, Paolo Cavallini cavall...@faunalia.it wrote: The Processing plugin may substitute GRASS Tools part of the plugin (i.e. modules GUI), it cannot help with: - mapset creation - vector/raster maps visualization - vector digitizing - region visualization and editing The Processing plugin is good for users who don't want to use GRASS data format at all. The GRASS plugin is GRASS GUI alternative for true GRASS users. Yes, got it. The question is: what is the real advantage to use grass alone, instead of jointly with other tols in Processing? I am not even trying to ask that question. As long as there are GRASS users using the QGIS plugin, it should not be thrown away. I see two major ones: * Processing is blocking the main QGIS canvas, which for very long analyses is unacceptable * import-export overhead is a serious issue for complex vectors. There is the GRASS Direct lib which allows GRASS raster modules to read/write data through QGIS providers. I is currently disabled, needs some fixes but it was mostly working. If fixed, it can be easily used in Processing, I think. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Radim, 2014-04-23 14:50 GMT+02:00 Radim Blazek radim.bla...@gmail.com: On Tue, Apr 22, 2014 at 4:49 PM, Sören Gebbert soerengebb...@googlemail.com wrote: IMHO we GRASS developers are too stubborn to change the design of GRASS GIS to work as library with persistent applications. This would require rewriting plenty of core functionality and modification of all C-modules. But there is a solution for persistent applications, called Remote Procedure Call (RPC), more below. It sounds interesting but it seems to be a huge work an far future. I Its far less work then modifying GRASS libraries to run with persistent applications. We have already the providers and the plugin working with GRASS 6, I just want to upgrade it to GRASS 7. The only problem I found so far is that G__get_window() and Rast_get_cellhd() call G_fatal_error(). I can only dedicate few hours/max days of my free time to that, so I have to consider well what is less work, RPC is not. Yes, i absolutely understand this. We need such an approach anyway, since vector editing is a GUI task - GUI is a persistent application - we need an exit safe interface to the GRASS libraries to implement a reliable vector editing GUI in GRASS itself. Vector lib does not call G_fatal_error(), it was written in parallel with QGIS plugin with vector editing in mind. I hope it is still true. I fear that might be not the case anymore. For example, all memory allocation functions G_calloc(), G_malloc() and G_realloc() will call G_fatal_error() in case the memory allocation fails. As far as i can tell the vector library make often use of this functions. So we have the problem that basic core functions call G_fatal_error() that are used all over the place in GRASS GIS. am also concerned about complexity, e.g. starting / keeping running servers on various platforms. I can imagine endless bug reports about server not running. I am looking for something which will not Since only QGIS is spawning the server process, it is always able to check if the process is running and can restart it when needed. We already make use of this concept in GRASS and it works nicely[1]. Spawning of processes and keeping track of them is an easy task with Qt, and it is completely OS independent. The Qt Bitcoin client for example makes use of RPC and runs on Windows, Mac OS and various Linux systems. So i don't see a problem here. The raster provider is already using something like that. Raster data are read from executed GRASS module output (a module written for that in QGIS in fact) and identify tool keeps another GRASS module running and it is communicating with that through a pipe. I wrote that as an experiment and because I know, how much unnecessary additional code and potential problems it is, I don't want to continue in that direction. Ok. require permanent assistance. It is not KISS enough for me. What would be a better KISS solution? IMO the RPC approach will make the implementation of the QGIS-GRASS data provider and the plugin simpler. There will be no mixing of C++ and C-code anymore. We are no starting the plugin from scratch, it is already written, we just need to make it work with GRASS 7. No setjmp/longjmp/exception workaround that is really complex Why? If you patch GRASS to use setjmp/longjmp to implement an exception like behavior then you need to write clean up code for static variables and dynamically allocated memory. A G_fatal_error() call can be deeply nested in a library function. Cleaning only the static variables to be in usable state will lead to problematic memory leaks. So it is a complex task in GRASS. Using setjmp/longjmp in QGIS wrapping all GRASS functions that might call G_fatal_error() is complex, since a G_fatal_error() call can be deeply nested in a library function. You need to keep track of this resulting in plenty setjmp wrappers around GRASS functions. You need to cleanup global static structures as well and you will have no control over pointers pointing to dynamic allocated memory that got lost in a longjmp. IMHO booth cases require plenty of work to make them usable. Having a kind of garbage collector in GRASS, implemented in the G_*alloc() functions, may solve many problems. But i have no idea howto implement the garbage collector to detect pointers that got orphaned in a longjmp to automagically clean them. The next step would be to centralize global variables to clean them up more easily (as Glynn mentioned). and will not prevent QGIS from terminating or segfaulting eventually in case of a fatal error. Why? Because of the undefined state of global variables and structures and because of potential memory leaks that may sum up making QGIS unusable. Best regards Soeren Radim The QGIS GRASS data provider/plugin will be a clean C++ implementation using the thrift library. It will be better maintainable since developers do not need to know all the different
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Apr 18, 2014 at 12:27 PM, Paolo Cavallini cavall...@faunalia.it wrote: Il 18/04/2014 11:31, Radim Blazek ha scritto: I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Thanks for this. If I understand well, this means: * we can load GRASS7 vectors through QGIS browser * we cannot load GRASS7 rasters * the qgis-grass-plugin is not functional with GRASS7 Yes. We are talking about current master, not about intended final implementation. BTW, GRASS6 vectors apparently cannot be loaded from QGIS browser as a whole. This, coupled with several serious regressions I found, may mean that GRASS support in QGIS is essentially broken, You are talking about GRASS 6 support? What are serious regressions? and things may only get worse with the arrival of GRASS7, if nobody works to fix it. Am I too pessimistic? The most likely solution seems to use GRASS only through Processing. This also has a number of major limitations, however. The Processing plugin may substitute GRASS Tools part of the plugin (i.e. modules GUI), it cannot help with: - mapset creation - vector/raster maps visualization - vector digitizing - region visualization and editing The Processing plugin is good for users who don't want to use GRASS data format at all. The GRASS plugin is GRASS GUI alternative for true GRASS users. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Apr 18, 2014 at 2:19 PM, Sören Gebbert soerengebb...@googlemail.com wrote: Hi Radim, IMHO we GRASS developers are too stubborn to change the design of GRASS GIS to work as library with persistent applications. This would require rewriting plenty of core functionality and modification of all C-modules. But there is a solution for persistent applications, called Remote Procedure Call (RPC), more below. It sounds interesting but it seems to be a huge work an far future. I am also concerned about complexity, e.g. starting / keeping running servers on various platforms. I can imagine endless bug reports about server not running. I am looking for something which will not require permanent assistance. It is not KISS enough for me. Radim 2014-04-18 11:31 GMT+02:00 Radim Blazek radim.bla...@gmail.com: I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. Basically more and more low level functions are calling exit() instead of returning error code if something failed. As I am not willing to implement GRASS module call for each simple function, we have to think again about hacks we can use: 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS 3) use setjmp()/longjmp() 4) let QGIS crash whenever GRASS lib function fails I fear that none of these suggestions are good working solutions, since in case a fatal error or a segfault occurs nothing (no exception, no setjmp) will prevent QGIS from crashing. So i would like to suggest the following: 5.) Using a RPC interface for map metadata, vector and raster map access. Hence one or several GRASS server processes provide an RPC interface to access GRASS library functions that are needed in a persistent application. Most important is the vector editing, vector reading/writing, database access, raster reading and map metadata support. Everything else can be done using modules. Hence the RPC interface will support only a limited subset of the GRASS library functions. This RPC interface should only be used in the GRASS provider classes, the GRASS plugin itself can be written in C++ or Python. This approach will decouple GRASS from QGIS, since all communication is done via Inter Process Communication (IPC) using pipes or sockets. There is no need anymore to catch a fatal error or to link GRASS libraries directly to QGIS. The GRASS plugin can be implemented GRASS version independently and so the GRASS data provider, since they will not use GRASS functions directly only the RPC interface. The RPC implementation on the side of GRASS will provide a consistent interface that will not change in case the underlying GRASS API changes. I strongly suggest to use an existing RPC framework to implement fast binary data exchange and the IPC. My favorite is apache thrift[1], since it supports plenty of programming languages (C/C++, Python, Java, JavaScript, ...) and provides operating system independent client and server functionality. It supports exceptions on client side that can be emitted in case a GRASS server process died because of a fatal error, SIGINT or segfault. I am absolutely willing to implement the RPC server side in GRASS using thrift and C++, providing plenty of Python unit tests that show howto use it on the client side. We just need to decide what kind of GRASS library functionality is needed as RPC interface and what can be done in QGIS with C++/Python directly (gisrc creation, GRASS environmental variable settings, ...) or using GRASS modules (g.region, g.gisenv, ...). [1] https://thrift.apache.org/ Best regards Soeren Radim [1]https://trac.osgeo.org/grass/ticket/869#comment:1 On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini cavall...@faunalia.it wrote: Hi all. I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) * some of the module options will be different * new modules will not be available in QGIS. I think we can deal with this in several ways: * dropping the plugin, and concentrate the work on Processing * upgrading both the plugin and Processing. In the first case, we have two major issues: * upgrading Processing GRASS modules * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. While the first issue can be solved easily by a
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Sat, Apr 19, 2014 at 12:43 AM, Glynn Clements gl...@gclements.plus.com wrote: Basically more and more low level functions are calling exit() instead of returning error code if something failed. As I am not willing to implement GRASS module call for each simple function, we have to think again about hacks we can use: 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions By itself, this will do nothing. I mean to allow to throw exception from the function set by G_set_error_routine(). 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS If you're talking about replacing the exit() with a return, that will typically just result in the caller crashing when G_fatal_error() fails to perform its primary function (prevening any code which follows it from being executed). Right, nonsense. 3) use setjmp()/longjmp() That is the sane option. But bear in mind that subsequently calling any GRASS function is entirely at your own risk. A call to G_fatal_error() often means that internal data structures will contain garbarge. If you want to submit patches to perform clean-up in the event of fatal errors, they will presumably be accepted. Good. Failing that, it may be advisable to have G_fatal_error() zero-out certain core data structures (e.g. R__) so that people don't make the mistake of trying to carry on as if nothing happened. Do you mean to add some Rast_clean() to GRASS lib and call it from G_fatal_error()? Something to be done in GRASS library? Still, any dynamically allocated memory, to which there is no pointer from static structures like R__, will never be released. Radim ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Radim, 2014-04-22 16:03 GMT+02:00 Radim Blazek radim.bla...@gmail.com: On Fri, Apr 18, 2014 at 2:19 PM, Sören Gebbert soerengebb...@googlemail.com wrote: Hi Radim, IMHO we GRASS developers are too stubborn to change the design of GRASS GIS to work as library with persistent applications. This would require rewriting plenty of core functionality and modification of all C-modules. But there is a solution for persistent applications, called Remote Procedure Call (RPC), more below. It sounds interesting but it seems to be a huge work an far future. I Its far less work then modifying GRASS libraries to run with persistent applications. We need such an approach anyway, since vector editing is a GUI task - GUI is a persistent application - we need an exit safe interface to the GRASS libraries to implement a reliable vector editing GUI in GRASS itself. am also concerned about complexity, e.g. starting / keeping running servers on various platforms. I can imagine endless bug reports about server not running. I am looking for something which will not Since only QGIS is spawning the server process, it is always able to check if the process is running and can restart it when needed. We already make use of this concept in GRASS and it works nicely[1]. Spawning of processes and keeping track of them is an easy task with Qt, and it is completely OS independent. The Qt Bitcoin client for example makes use of RPC and runs on Windows, Mac OS and various Linux systems. So i don't see a problem here. require permanent assistance. It is not KISS enough for me. What would be a better KISS solution? IMO the RPC approach will make the implementation of the QGIS-GRASS data provider and the plugin simpler. There will be no mixing of C++ and C-code anymore. No setjmp/longjmp/exception workaround that is really complex and will not prevent QGIS from terminating or segfaulting eventually in case of a fatal error. The QGIS GRASS data provider/plugin will be a clean C++ implementation using the thrift library. It will be better maintainable since developers do not need to know all the different GRASS libraries (gis, raster, vector, database) but only the RPC interface that make use of a subset of the GRASS library functions, bundling them in an object oriented API. QGIS do not need to link against GRASS libraries anymore, future GRASS versions will be supported out of the box. This is much more KISS then the current approach. Best regards Soeren [1] http://grass.osgeo.org/programming7/namespacepython_1_1temporal_1_1c__libraries__interface.html Radim 2014-04-18 11:31 GMT+02:00 Radim Blazek radim.bla...@gmail.com: I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. Basically more and more low level functions are calling exit() instead of returning error code if something failed. As I am not willing to implement GRASS module call for each simple function, we have to think again about hacks we can use: 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS 3) use setjmp()/longjmp() 4) let QGIS crash whenever GRASS lib function fails I fear that none of these suggestions are good working solutions, since in case a fatal error or a segfault occurs nothing (no exception, no setjmp) will prevent QGIS from crashing. So i would like to suggest the following: 5.) Using a RPC interface for map metadata, vector and raster map access. Hence one or several GRASS server processes provide an RPC interface to access GRASS library functions that are needed in a persistent application. Most important is the vector editing, vector reading/writing, database access, raster reading and map metadata support. Everything else can be done using modules. Hence the RPC interface will support only a limited subset of the GRASS library functions. This RPC interface should only be used in the GRASS provider classes, the GRASS plugin itself can be written in C++ or Python. This approach will decouple GRASS from QGIS, since all communication is done via Inter Process Communication (IPC) using pipes or sockets. There is no need anymore to catch a fatal error or to link GRASS libraries directly to QGIS. The GRASS plugin can be implemented GRASS version independently and so the GRASS data provider, since they will not use GRASS functions directly only the RPC interface. The RPC implementation on the side of GRASS will provide a consistent interface that will not change in case the underlying GRASS API changes. I strongly suggest to use
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi, 2014-04-20 1:34 GMT+02:00 Sören Gebbert soerengebb...@googlemail.com: Hence the wrapper will not only do simple function wrapping, it will also implement a higher level interface: [...] what about to create a trac wiki page dedicated to RFC and collect this notes there... ? Martin -- Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
2014-04-19 0:59 GMT+02:00 Glynn Clements gl...@gclements.plus.com:
[...]
You're misunderstanding a key point.
Misunderstanding is usually a question of the point of view or level
of doggedness. Especially when someone is speaking about a key
point ;-)
Returning from G_fatal_error() won't leave GRASS libraries in an
unpredictable state. It's escaping from G_fatal_error() via e.g.
Returning from G_fatal_error() will typically result in a
(more-or-less immediate) segfault when the caller tries to use data
structures which contain garbage (because the callee never initialised
them, but raised a fatal error instead).
Well, and what do you think that I meant by unpredictable state?
The simplest example is G__malloc():
buf = malloc(n);
if (!buf) {
...
G_fatal_error(_(G_malloc: unable to allocate %lu bytes of memory at
%s:%d),
(unsigned long) n, file, line);
}
return buf;
If malloc() returns NULL and the subsequent G_fatal_error() call
returns, G__malloc() will return NULL to its caller. Which will
segfault as soon as it tries to store something in the allocated
memory.
You chose very specific case of course. There is probably something
you don't want to understand. In the core libraries (gis, vector,
raster, imagery) there are 553(!) places where G_fatal_error() is
called. Some time ago you introduced G_fatal_error() even to the
function which tries to open raster map [1]. So if the raster map
doesn't exists the function is calling exit()! This is something which
makes life harder for anyone who want to use GRASS libraries (as you
already mentioned) with one exception - modules - standalone programs,
where you don't need to specify `if` statement. You can hardly explain
to someone that our libraries call fatal exit even when raster map is
not found. Yes, you can if don't want to change your opinion
regardless any input from outside world. For such cases, like opening
raster map would G_set_fatal_error() simply make sense.
It achieves this by calling exit() (or optionally raising SIGSEGV; why
it didn't use SIGABRT, I have no idea). Calling longjmp() would work.
If it was C++, raising an exception would work. Returning will *not*
work.
[...]
Martin
[1]
http://trac.osgeo.org/grass/changeset/40209/grass/trunk/lib/raster/opencell.c
--
Martin Landa * http://geo.fsv.cvut.cz/gwiki/Landa
___
Qgis-developer mailing list
Qgis-developer@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Martin, On Fri, 18. Apr 2014 at 12:20:15 +0200, Martin Landa wrote: [...] I still think that we should provide in GRASS API function like G_set_fatal_error() with default value G_FATAL_ERROR_EXIT (current behaviour). The second option would be G_FATAL_ERROR_RETURN with big big warning in the API manual that you are living GRASS libraries in completely unpredictable state and you should immediately stop an application which is using GRASS libraries. I also don't think that would help much. You would still need to use G_set_error_routine to track that there was an fatal error and in turn avoid a crash by not touching the result when the call returns - and keep away from calling other library function because of the unpredictable state of the library. On Fri, 18. Apr 2014 at 23:59:25 +0100, Glynn Clements wrote: It achieves this by calling exit() (or optionally raising SIGSEGV; why it didn't use SIGABRT, I have no idea). Calling longjmp() would work. If it was C++, raising an exception would work. Returning will *not* work. I guess there are also library calls inside the libraries themselves and those would not be aware that there is a need to handle invalid results and might crash before the toplevel caller has a chance know of the error. I think that's Glynn's keypoint. Using setjmp()/longjmp() or throwing C++ exceptions (with the libraries built with -fexceptions) in the error routine would take you back to the caller that is aware of the error. That avoids the crash, but also leaves the library in an unpredictable state. To avoid that there's no other way than returning and handling all the errors properly. Probably huge amount of work with not much gain for GRASS. The RPC approach will IMHO be less work and also cause less friction. Jürgen -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS PSC member (RM) Germany IRC: jef on FreeNode -- norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH Rheinstrasse 13, 26506 Norden GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502 ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
G_fatal_error() is too low level not only for messages but in general, for Python libraries wrapping GRASS library, it does not allow to throw an exception in case of an error. And the same applies for C++ wrappers. If you want to turn fatal errors into exceptions, the correct way to do it is to use setjmp/longjmp (and contribute patches to make any intervening code exception-safe). As I wrote before, I hope that RPC approach suggested here by Soeren will work for languages with exceptions and for persistent applications while GRASS modules will benefit from the advantages of GRASS library with G_fatal_error() with exit(). Vaclav ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Glynn, An RPC wrapper would move the execution of GRASS functions into a new process (i.e. a server). If the call generates a fatal error, the server dies, the client detects this and reports an error rather than a result. The main problem with this is that any error loses the entire state held on the server, e.g. any open maps no longer exist. Exactly! Hence the wrapper will not only do simple function wrapping, it will also implement a higher level interface: * map metadata read access as single client function will do on the server side: open map; read metadata; close map; send data * fast raster and vector map read access as single client function will do on server side: open map; read requested map into memory based on bounding box and render resolution; close map; send data * vector editing: keep a single vector map in update state open in a dedicated grass vector editing process. In case of a fatal error, this map may be lost or corrupted. It is important that this process will not be used for other purposes than single vector map editing. * vector analysis functionality that need to keep vector maps open in read only state for fast topological access. Information's about the open vector maps (i. e. the position of the next line to be read) may be lost in case the server process terminates. * Same for raster map analysis The client will detect if the server was terminated and will raise an exception. In QGIS that will be the data provider classes. The dedicated raster display module in QGIS is a nice starting point to implement a RPC function that will read a raster map as RGB color array based on extent and resolution information into a byte array that will be send via thrift (using thrift binary protocol) to the client. The RPC interface should only support meaningful functions to be used in a GUI, that's its only purpose. It should not be used to implement processing algorithms or GRASS modules. But it will in addition provide capabilities to use several processes to read different chunks of the same map in the GUI, hence parallel read only map access. This can be nicely done using a pool of GRASS server processes. Using setjmp/longjmp would be more useful, but it would require some effort to add clean-up code to functions so that core data structures aren't left in an inconsistent state. There are plenty of functions using global variables in GRASS. Also, persistent applications may change the location and therefore the projection several times on run-time. Are the GRASS libraries able to handle this correctly? The suggested RPC server processes can be restarted in this case. Best regards Soeren ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. Basically more and more low level functions are calling exit() instead of returning error code if something failed. As I am not willing to implement GRASS module call for each simple function, we have to think again about hacks we can use: 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS 3) use setjmp()/longjmp() 4) let QGIS crash whenever GRASS lib function fails Radim [1]https://trac.osgeo.org/grass/ticket/869#comment:1 On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini cavall...@faunalia.it wrote: Hi all. I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) * some of the module options will be different * new modules will not be available in QGIS. I think we can deal with this in several ways: * dropping the plugin, and concentrate the work on Processing * upgrading both the plugin and Processing. In the first case, we have two major issues: * upgrading Processing GRASS modules * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. While the first issue can be solved easily by a couple of people in a few days, the second one is more tricky, and requires hard coding skills. In addition, we'll no longer be able to edit GRASS vectors directly. In the second case, we'll have more work, and a not-so-nice duplication. I would like to have an open discussion on this, avoiding things to just happen, with the possible negative consequences. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ grass-dev mailing list grass-...@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi, 2014-04-18 11:31 GMT+02:00 Radim Blazek radim.bla...@gmail.com: [] Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. Basically more and personally I am not fan of this approach. I still think that we should provide in GRASS API function like G_set_fatal_error() with default value G_FATAL_ERROR_EXIT (current behaviour). The second option would be G_FATAL_ERROR_RETURN with big big warning in the API manual that you are living GRASS libraries in completely unpredictable state and you should immediately stop an application which is using GRASS libraries. There no so big chance that someone will rewrite GRASS libraries to be suitable for long running applications, on the other hand we should allow in API possibility to avoid calling exit with big warning to the programmer. Than it will be his/her responsibility if he/she uses G_FATAL_ERROR_RETURN. Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Radim, Il 18/04/2014 11:31, Radim Blazek ha scritto: I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Thanks for this. If I understand well, this means: * we can load GRASS7 vectors through QGIS browser * we cannot load GRASS7 rasters * the qgis-grass-plugin is not functional with GRASS7 BTW, GRASS6 vectors apparently cannot be loaded from QGIS browser as a whole. This, coupled with several serious regressions I found, may mean that GRASS support in QGIS is essentially broken, and things may only get worse with the arrival of GRASS7, if nobody works to fix it. Am I too pessimistic? 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions this IMVHO does not seem feasible for all OS and distros. 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS same as above 3) use setjmp()/longjmp() 4) let QGIS crash whenever GRASS lib function fails this is obviously unacceptable. The most likely solution seems to use GRASS only through Processing. This also has a number of major limitations, however. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Radim, IMHO we GRASS developers are too stubborn to change the design of GRASS GIS to work as library with persistent applications. This would require rewriting plenty of core functionality and modification of all C-modules. But there is a solution for persistent applications, called Remote Procedure Call (RPC), more below. 2014-04-18 11:31 GMT+02:00 Radim Blazek radim.bla...@gmail.com: I have upgraded the vector provider to GRASS 7, layers may be added by drag from browser. The raster and the plugin are disabled. Be careful about multiple versions on the same system (LD_LIBRARY_PATH..., check with ldd if does not work). Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. Basically more and more low level functions are calling exit() instead of returning error code if something failed. As I am not willing to implement GRASS module call for each simple function, we have to think again about hacks we can use: 1) add a requirement that GRASS 7 used with QGIS must be compiled with -fexceptions 2) add a requirement that a patch (it is a single line comment in fact) must be applied to GRASS 7 to make it usable with QGIS 3) use setjmp()/longjmp() 4) let QGIS crash whenever GRASS lib function fails I fear that none of these suggestions are good working solutions, since in case a fatal error or a segfault occurs nothing (no exception, no setjmp) will prevent QGIS from crashing. So i would like to suggest the following: 5.) Using a RPC interface for map metadata, vector and raster map access. Hence one or several GRASS server processes provide an RPC interface to access GRASS library functions that are needed in a persistent application. Most important is the vector editing, vector reading/writing, database access, raster reading and map metadata support. Everything else can be done using modules. Hence the RPC interface will support only a limited subset of the GRASS library functions. This RPC interface should only be used in the GRASS provider classes, the GRASS plugin itself can be written in C++ or Python. This approach will decouple GRASS from QGIS, since all communication is done via Inter Process Communication (IPC) using pipes or sockets. There is no need anymore to catch a fatal error or to link GRASS libraries directly to QGIS. The GRASS plugin can be implemented GRASS version independently and so the GRASS data provider, since they will not use GRASS functions directly only the RPC interface. The RPC implementation on the side of GRASS will provide a consistent interface that will not change in case the underlying GRASS API changes. I strongly suggest to use an existing RPC framework to implement fast binary data exchange and the IPC. My favorite is apache thrift[1], since it supports plenty of programming languages (C/C++, Python, Java, JavaScript, ...) and provides operating system independent client and server functionality. It supports exceptions on client side that can be emitted in case a GRASS server process died because of a fatal error, SIGINT or segfault. I am absolutely willing to implement the RPC server side in GRASS using thrift and C++, providing plenty of Python unit tests that show howto use it on the client side. We just need to decide what kind of GRASS library functionality is needed as RPC interface and what can be done in QGIS with C++/Python directly (gisrc creation, GRASS environmental variable settings, ...) or using GRASS modules (g.region, g.gisenv, ...). [1] https://thrift.apache.org/ Best regards Soeren Radim [1]https://trac.osgeo.org/grass/ticket/869#comment:1 On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini cavall...@faunalia.it wrote: Hi all. I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) * some of the module options will be different * new modules will not be available in QGIS. I think we can deal with this in several ways: * dropping the plugin, and concentrate the work on Processing * upgrading both the plugin and Processing. In the first case, we have two major issues: * upgrading Processing GRASS modules * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. While the first issue can be solved easily by a couple of people in a few days, the second one is more tricky, and requires hard coding skills. In addition, we'll no longer be able to edit GRASS vectors directly. In the second case, we'll have more work, and a not-so-nice duplication. I would like to have an open discussion on this, avoiding things to just happen, with the possible negative consequences. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses:
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Apr 18, 2014 at 6:59 PM, Glynn Clements gl...@gclements.plus.comwrote: The key feature of G_fatal_error() isn't printing error messages. The key feature is that it doesn't return. This is one of the problems of G_fatal_error(), it does not gives good error messages to the user. User gets G_malloc: unable to allocate %lu bytes of memory at %s:%d but does not have any idea when and why this happened and what to do about it. If the module calling G_malloc() had the chance to report error by itself, it could suggest to user to change some parameter or use a smaller map. With G_fatal_error() user has only the possibility to try to guess from debug messages, ask on mailing list or keep trying with random changes in input parameters. I don't believe that this is efficient. G_fatal_error() is too low level not only for messages but in general, for Python libraries wrapping GRASS library, it does not allow to throw an exception in case of an error. And the same applies for C++ wrappers. As far as I understand, the RPC wrapper suggested earlier by Soeren, would hopefully allow to have exceptions (and no exit()) in Python and C++ and thus potentially higher level error messages and also general error handling would be possible (for those not using the original library). ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Apr 18, 2014 at 6:43 PM, Glynn Clements gl...@gclements.plus.comwrote: Unfortunately GRASS 7 moved ahead towards its aim to make life harder for anyone trying to use the GRASS libraries [1]. That isn't actually the reason why various functions have had status returns changed to fatal errors. The reason is to avoid pushing the burden of error handling onto their callers. The fact is that no library should use exit() or similar things if it wants to be general. GRASS library is intended to be used only for building GRASS modules, thus it is specialized for them, thus it uses G_fatal_error() with exit(). (At least there is a believe that exit makes it easier.) As a result using GRASS functions outside the scope of GRASS modules requires another library. It would be nice to have two libraries, one for GRASS modules (with exit()) and one for everybody else. Hopefully, it would be possible to share the same code to some extent. Alternative is to use the RPC wrapper as suggested earlier by Soeren, as far as I understand this has the same result. The questions are what is less work, less duplication, what has easier maintenance and what is easier to do for all the platforms. ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Thu, Mar 27, 2014 at 12:18 PM, Markus Neteler nete...@osgeo.org wrote: Hi Paolo, (note: I'm not ready qgis-dev, not sure if this email reaches it) ... * new modules will not be available in QGIS. There might be only a few which could be of interest here (e.g. the highly specialized evapotranspiration modules not but some others yes). For an overview, see http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures ... See now also http://trac.osgeo.org/grass/wiki/Release/7.0.0beta-News I expect that we'll come up with a Processing prototype for which supports GRASS 7 soon. Just for the record: my updated files have hit QGIS master via Pirmin and Victor: On Mon, Mar 31, 2014 at 11:47 PM, volaya notificati...@github.com wrote: Merged #1279. https://github.com/qgis/QGIS/pull/1279 https://github.com/qgis/QGIS/tree/master/python/plugins/processing/grass7 Please test... Best Markus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 28/03/2014 08:34, Paolo Cavallini ha scritto: Thanks to all for the interesting comments. Obviously we have a varied and rich ecosystem here, that's why I want to preserve it. Obviously we all want to have all. IMHO the points are: * can we (GRASS+QGIS) agree on a specific course of actions? * can we find the resource to implement it? Of course we could think of sticking with GRASS6, but I think sooner or later this will be untenable (e.g. major distros will switch to GRASS7 once out), so better prepare now. Quick interesting chat with Luca Delucchi and Martin Landa from GRASS dev team. One interesting option would be: * to add GRASS browsing capabilities in QGIS file browser * to add support to direct GRASS reading (possibly writing) in Processing, and avoid import/export whenever possible; v.external and v.out.external could be used for this; this is probably the thoughest part * of course modules should be upgraded anyway * GRASS direct digitizing can be skipped; heavy digitizer can use GRASS directly * keep a GRASS shell? * with all the above, upgrading the plugin may probably be skipped. Any thoughts. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Paolo, 2014-03-27 11:18 GMT+01:00 Paolo Cavallini cavall...@faunalia.it: Hi all. I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) Yes, it will stop working, since the API in GRASS7 has changed (Raster API functions have now Rast_ as prefix). Besides of that must the cmake files be modified to detect GRASS7. * some of the module options will be different * new modules will not be available in QGIS. Yes, but this shouldn't be a problem if the module interface description created by the GRASS modules itself was used to generate the module interface in QGIS. New and modified modules will not work if the interfaces are handcrafted. I think we can deal with this in several ways: * dropping the plugin, and concentrate the work on Processing That is IMHO not a good idea. I think to provide the full functionality of GRASS7 to QGIS user, this plugin should be maintained and updated to support the new GRASS7 API. Handling and processing of massive datasets, especially time series, is only meaningful if GRASS is used as data storage as well. The processing interface will add massively overhead in data processing. The temporary location/mapset creation approach is not well suited to process massive data, even though r.external and v.external are used to link external data temporary into GRASS. * upgrading both the plugin and Processing. Yes, that's the way to go. In the first case, we have two major issues: * upgrading Processing GRASS modules * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. While the first issue can be solved easily by a couple of people in a few days, the second one is more tricky, and requires hard coding skills. In addition, we'll no longer be able to edit GRASS vectors directly. In the second case, we'll have more work, and a not-so-nice duplication. I would like to have an open discussion on this, avoiding things to just happen, with the possible negative consequences. My suggestion would be: Full integration of the GRASS7 into QGIS via C++ or Python plugin. This includes the temporal GIS capabilities as well. The existing plugin is a very good start point, lots its functionality can be reused, especially the vector editing, grass shell and map management. But there is a major problem with the GRASS QGIS plugin: it links directly to the grass libraries and calls plenty of functions that can QGIS cause to crash in case of an error. We face the same problem in GRASS with the vector editing tools. My solution would be to use a RPC (Remote Procedure Call) interface to calls GRASS library functions in a remote process using binary data for inter-process communication. IMHO the best tool for this is apache thrift[1] which allows us to implement a RPC interface in GRASS7 to the needed library functions. IMHO the number of RPC functions is limited since only vector editing, raster map rendering and some map/stds management functions are needed for direct access, all other functionality is provided by GRASS modules. So the first step is to implement an RPC interface in GRASS7, that supports C/C++, Java, Python and JavaScript on the client side out of the box. This interface can be used by the GRASS GUI itself to implement exit safe vector editing and it can be used by QGIS and other nice GIS desktop systems to provide GRASS database access, fast raster rendering and vector edit functionality. The beauty of this approach is, that the client side (the QGIS plugin for example) do not need to link against GRASS libraries, since it will communicate via pipes or sockets with one or several persistent GRASS7 processes, which can be restarted in case of a fatal error. The client side do not need to be updated in case the GRASS7 API changes again, only the server side which will be implemented in GRASS7 must be updated. Implementation effort In case of the QGIS plugin: All direct GRASS dependencies and function calls must be removed and replaced by the client RPC solution. Hence the provider classes needs to be rewritten, the C++ plugin code itself needs to be modified to support the new interface datatypes that are used for inter-process communication. Access to the temporal GIS functionality must be implemented to list space time datasets. What do you think about this approach? Best regards Soeren [1] https://thrift.apache.org/ All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ grass-dev mailing list grass-...@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/grass-dev ___ Qgis-developer mailing list
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
I updated all existing modules incl the newly included r.stream.*. A few more new modules might be interesting to add. That's easy... Markus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Mar 28, 2014 at 11:16 AM, Paolo Cavallini cavall...@faunalia.itwrote: Il 28/03/2014 15:04, Martin Dobias ha scritto: On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini cavall...@faunalia.it wrote: * to add GRASS browsing capabilities in QGIS file browser The support for GRASS is already in the browser - if you enter a directory that is a GRASS database, it will detect it and show locations/mapsets/maps/layers. That's great news, never tried. Works beautifully, thanks. In the GRASS plugin there are a few more features, i.e.: * showing history (important) * copy/rename/delete. The browser does not support these operations, but it would be cool. If we can add these, I think the corresponding section of the plugin would be rather useless. Am I missing something? All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 28/03/2014 15:51, Markus Neteler ha scritto: I have submitted the Processing for GRASS GIS 7 to Pirmin and Victor for git upload. Now the user has GRASS 6 and 7 both supported in Processing. good news, thanks. are you referring to the upgraded and new modules? all the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 28/03/2014 15:04, Martin Dobias ha scritto: On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini cavall...@faunalia.it wrote: * to add GRASS browsing capabilities in QGIS file browser The support for GRASS is already in the browser - if you enter a directory that is a GRASS database, it will detect it and show locations/mapsets/maps/layers. That's great news, never tried. Works beautifully, thanks. In the GRASS plugin there are a few more features, i.e.: * showing history (important) * copy/rename/delete. If we can add these, I think the corresponding section of the plugin would be rather useless. Am I missing something? All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi all, On Thu, Mar 27, 2014 at 12:18 PM, Markus Neteler nete...@osgeo.org wrote: Hi Paolo, (note: I'm not ready qgis-dev, not sure if this email reaches it) On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini cavall...@faunalia.it wrote: Hi all. I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) * some of the module options will be different For the time being people can certainly continue to work with GRASS 6. I think that the plugin needs to be cloned for GRASS 7. I have submitted the Processing for GRASS GIS 7 to Pirmin and Victor for git upload. Now the user has GRASS 6 and 7 both supported in Processing. Cheers from the train, Markus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 28/03/2014 15:16, Paolo Cavallini ha scritto: Il 28/03/2014 15:04, Martin Dobias ha scritto: The support for GRASS is already in the browser - if you enter a directory that is a GRASS database, it will detect it and show locations/mapsets/maps/layers. In the GRASS plugin there are a few more features, i.e.: * showing history (important) * copy/rename/delete. If we can add these, I think the corresponding section of the plugin would be rather useless. Am I missing something? What about writing down the minumum set of requirements to be preserved? Please add it to this thread, we can note it on a wiki page once bolied down. All the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On Fri, Mar 28, 2014 at 11:48 AM, Paolo Cavallini cavall...@faunalia.it wrote: * to add GRASS browsing capabilities in QGIS file browser The support for GRASS is already in the browser - if you enter a directory that is a GRASS database, it will detect it and show locations/mapsets/maps/layers. Regards Martin ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi Paolo, (note: I'm not ready qgis-dev, not sure if this email reaches it) On Thu, Mar 27, 2014 at 11:18 AM, Paolo Cavallini cavall...@faunalia.it wrote: Hi all. I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) * some of the module options will be different For the time being people can certainly continue to work with GRASS 6. I think that the plugin needs to be cloned for GRASS 7. * new modules will not be available in QGIS. There might be only a few which could be of interest here (e.g. the highly specialized evapotranspiration modules not but some others yes). For an overview, see http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures I think we can deal with this in several ways: * dropping the plugin, and concentrate the work on Processing * upgrading both the plugin and Processing. (the second: yes) In the first case, we have two major issues: * upgrading Processing GRASS modules I'll do that. I already started with Pirmin and Victor to discuss it. * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. ... this would not be a good idea. While the first issue can be solved easily by a couple of people in a few days, the second one is more tricky, and requires hard coding skills. In addition, we'll no longer be able to edit GRASS vectors directly. In the second case, we'll have more work, and a not-so-nice duplication. I would like to have an open discussion on this, avoiding things to just happen, with the possible negative consequences. I expect that we'll come up with a Processing prototype for which supports GRASS 7 soon. Best Markus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Il 27/03/2014 12:18, Markus Neteler ha scritto: * upgrading Processing GRASS modules I'll do that. I already started with Pirmin and Victor to discuss it. good news * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. ... this would not be a good idea. could you please explain why? all the best. -- Paolo Cavallini - www.faunalia.eu QGIS PostGIS courses: http://www.faunalia.eu/training.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On 27 March 2014 11:18, Paolo Cavallini cavall...@faunalia.it wrote: Hi all. Hi all, I learned during dinner that GRASS7 RC1 is due very soon. This opens the issue of its functioning in QGIS. IMHO: * the qgis-grass-plugin might stop working (this has to be tested) I test to compile QGIS master with GRASS7 but it doesn't work, the GRASS7 directory it seems not be recognized bu QGIS. Can I open a ticket on QGIS bug tracker? * some of the module options will be different * new modules will not be available in QGIS. I think we can deal with this in several ways: * dropping the plugin, and concentrate the work on Processing * upgrading both the plugin and Processing. In the first case, we have two major issues: * upgrading Processing GRASS modules * changing the current Processing behaviour, avoiding the import-export phase when piping consecutive GRASS commands; this makes GRASS modules slower than the equivalent commands of other backends. While the first issue can be solved easily by a couple of people in a few days, the second one is more tricky, and requires hard coding skills. In addition, we'll no longer be able to edit GRASS vectors directly. In the second case, we'll have more work, and a not-so-nice duplication. I'm not using at all GRASS from QGIS, so I think could be useful make a poll to know what the user need All the best. -- ciao Luca http://gis.cri.fmach.it/delucchi/ www.lucadelu.org ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
That sounds very reasonable to me. Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see http://trac.osgeo.org/gdal/ticket/2953). As for a GRASS data browser, I think, a plugin would be required with regards to user friendliness, because one needs to know what files to access from a GRASS data base when using GDAL. I also understand that at some point in time one will have to use GRASS directly in order to access full functionality (e.g. ortho-rectification, nviz, mapswipe, animation and stuff), which makes the way Moritz suggests maybe even more reasonable... Cheers Stefan -Original Message- From: Moritz Lennert [mailto:mlenn...@club.worldonline.be] Sent: 27. mars 2014 13:36 To: Blumentrath, Stefan Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev Subject: Re: [GRASS-dev] [Qgis-developer] GRASS QGIS: the future I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general point of view I also think that duplication is a problem and that the current way to go in QGIS is the processing framework. So; On 27/03/14 12:49, Blumentrath, Stefan wrote: I understand well the point; however, the plugin has additional functions, e.g.: * a grass shell couldn't this be implemented within the processing environment ? * a grass data browser If we are talking about accessing GRASS data for loading into QGIS, wouldn't it be a better idea to improve the GDAL/OGR handling in order to be able to load GRASS data just like any other data format ? * a grass digitizing environment. Maybe this could be split out into a specific plugin ? Moritz ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
Hi all, Here is my wish list: - To be able to access GRASS data within QGIS. Similar to PostGIS and other GDAL/OGR data sources, it should be up to users to create GRASS geodatabase/location/mapsets. QGIS will be able to simply add layers for viewing and editing - Having access to GRASS shell. It will be a very handy shortcut to run GRASS modules without the need to open GRASS. GRASS modules which are available in Processing toolbox can use the same UI. Otherwise, user has to run it in shell. - Simple import data to the mapsets...similar approach as drag-n-drop for PostGIS databases. Right-click and save as... in QGIS does a great work for Export. In summary, Processing toolbox should be a good one-stop for one-off use of a specific GRASS module. But if a user wants to dig more, GRASS already offers all that. For those who have already got their GRASS set up, they should be able to view and edit their data in QGIS with the added benefit of having access to shell. Some of the stuff can be ported to plugin(s) as mentioned by others: e.g. creating location/mapsets, setting region from qgis canvas or other qgis layers, etc. Cheers, Saber -Original Message- From: grass-dev-boun...@lists.osgeo.org [mailto:grass-dev-boun...@lists.osgeo.org] On Behalf Of Benjamin Ducke Sent: 27 March 2014 14:31 To: grass-dev Subject: Re: [GRASS-dev] [Qgis-developer] GRASS QGIS: the future In the gvSIG CE project, we had to make the same type of decisions and came to our own conclusions. Allow me to summarize our reasoning, maybe it will be useful for the QGIS project, as well: 1. The number one cause of irritations among novice users is having to set up a GRASS mapset and having to understand how GRASS data management works. 2. The number two cause of irritations are the effects of importing vector data with bad topology into a GRASS mapset. 3. The original QGIS plugin does nothing to alleviate (1). No plug-in, however cleverly designed, can do anything about (2): garbage in, garbage out. 4. GRASS power users gain very little (if anything) from running GRASS through a host GIS, such as QGIS or gvSIG. But they do lose functionality, such as the d.* family of modules. Therefore, we gave up trying to design a plug-in for advanced users. We assume that such users will use GRASS through its native CLI and/or native GUI. The resulting design of the original SEXTANTE-GRASS interface (which is now also mirrored in the Python re-write that became QGIS' Processing) addresses users that either don't care much for GRASS' CLI capabilities and internal data management, or are willing to switch to native GRASS as needed. If you want to change this and address another type of user, then you will need to re-examine the entire design of the current SEXTANTE/Processing approach, which is to use only temporary mapsets and perform data import/export every time a GRASS module is run. You can optimize the I/O performance of Processing by using r.external to directly access raster input maps. But there is little you can do about vector data with the current design, as GRASS needs to build its own topological data structures (and rebuild them every time you run a GRASS module on non-topological input!). In any case, I do not think that it is worth maintaining the original QGIS plugin, since it is neither very well suited for novice nor advanced users, IMHO. Best, Ben On 27/03/14 14:38, Blumentrath, Stefan wrote: That sounds very reasonable to me. Proper GDAL/OGR handling for GRASS 7 would be very nice in any case (see http://trac.osgeo.org/gdal/ticket/2953). As for a GRASS data browser, I think, a plugin would be required with regards to user friendliness, because one needs to know what files to access from a GRASS data base when using GDAL. I also understand that at some point in time one will have to use GRASS directly in order to access full functionality (e.g. ortho-rectification, nviz, mapswipe, animation and stuff), which makes the way Moritz suggests maybe even more reasonable... Cheers Stefan -Original Message- From: Moritz Lennert [mailto:mlenn...@club.worldonline.be] Sent: 27. mars 2014 13:36 To: Blumentrath, Stefan Cc: Paolo Cavallini; Nathan Woodrow; qgis-developer; grass-dev Subject: Re: [GRASS-dev] [Qgis-developer] GRASS QGIS: the future I'm not much of a QGIS-as-GRASS-frontend user, either, but from a general point of view I also think that duplication is a problem and that the current way to go in QGIS is the processing framework. So; On 27/03/14 12:49, Blumentrath, Stefan wrote: I understand well the point; however, the plugin has additional functions, e.g.: * a grass shell couldn't this be implemented within the processing environment ? * a grass data browser If we are talking about accessing GRASS data for loading into QGIS, wouldn't it be a better idea to improve the GDAL/OGR handling in order to be able to load GRASS data just
Re: [Qgis-developer] [GRASS-dev] GRASS QGIS: the future
On 03/27/2014 08:05 AM, Saber Razmjooei wrote: Hi all, Here is my wish list: - To be able to access GRASS data within QGIS. Similar to PostGIS and other GDAL/OGR data sources, it should be up to users to create GRASS geodatabase/location/mapsets. QGIS will be able to simply add layers for viewing and editing I have actually always created my locations/mapsets in QGIS. Just made more sense for some reason. - Having access to GRASS shell. It will be a very handy shortcut to run GRASS modules without the need to open GRASS. GRASS modules which are available in Processing toolbox can use the same UI. Otherwise, user has to run it in shell. Then once in GRASS usually went directly to using GRASS commands in a Terminal. But I can see for non-Linux the shell inside the plugin would be much more obvious. Perhaps a GRASS Launcher in QGIS would take it's place. - Simple import data to the mapsets...similar approach as drag-n-drop for PostGIS databases. Right-click and save as... in QGIS does a great work for Export. And as others have mentioned, 99.9% of the time my cartography work with GRASS layers is done in QGIS, including intermediary viewing of results before additional steps. Some of the other discussion about how to chain GRASS functions in Processing without having to dump to file outside GRASS actually gets at the biggest downside to Processing. If one has big data in GRASS or Postgis, it needs to stay inside those tools - shapefile is not an option. If we can some up with some way to encourage the various makers of tools that end up in Processing to implement all GDAL/OGR drivers, then we can pipe between things without having to drop to intermediary formats outside of ram. Thanks, Alex ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
