Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Stephen]

Hi Nyall,

sorry, I've made it past this error -- did you see this post?

http://lists.osgeo.org/pipermail/qgis-user/2018-May/042461.html

On 28.05.2018 07:07, Nyall Dawson wrote:
> On 14 May 2018 at 19:30, Stephen  wrote:
>>
>> Automatically setting output paths to temporary files would be a
>> sensible convenience, I think.
>>
>> In any event, I used processing.tools.system.getTempFilename() to set a
>> filename and that gets me past the 'Incorrect parameter value for
>> output' error.
>>
>> However, I am still getting "Unable to execute algorithm".
>>
>> In AlgorithmExecutor, line 55:
>> results, ok = alg.run(parameters, context, feedback)
>>
>> returns ({}, False)
>>
>> parameters contains:
>>
>> {'GRASS_REGION_CELLSIZE_PARAMETER': 0.008333, 'use': 1,
>> 'GRASS_REGION_PARAMETER':
>> '33.91415594546105,41.91384412832699,-4.679722671256611,5.467319665842772',
>> 'input': ,
>> 'GRASS_MIN_AREA_PARAMETER': 0.0001, 'GRASS_SNAP_TOLERANCE_PARAMETER':
>> -1, 'type': 0, 'output':
>> '/tmp/processing_da49d2e8c1bd4e249d1a5c6d42680149/5af9540a0b2212'}
>>
>> context contains:
>>
>> 
>>
>> and feedback contains:
>>
>> 
>>
>> Without a useful error message, it's very difficult to troubleshoot this
>> on my own.
>>
>> Do you have any idea what might be wrong here?
> 
> I had another look at this. I suspect what might be happening is that
> 'tmp/processing_da49d2e8c1bd4e249d1a5c6d42680149/5af9540a0b2212' does
> not exist, which makes GRASS bomb out with a 'No such file or
> directory' error.

I have observed that sometimes, when using temp files generated by
processing, QGIS (or the underlying algorithm) will not detect the file,
even though it exists.

I need to test this to confirm or falsify your theory. I'll let you know
what I find.

> Are you running this script from within QGIS itself? You should see
> the detailed log, including all the output from grass, by opening the
> log window and switching to the "Processing" tab (alternatively
> running from a terminal window will also show this output on the
> command line).

Sadly, no, this is purely standalone code (the wisdom of this is
arguable, but here we are).

How can I turn on detailed logging in standalone PyQGIS?

Thanks

Stephen
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Nyall Dawson]

On 14 May 2018 at 19:30, Stephen  wrote:
>
> Automatically setting output paths to temporary files would be a
> sensible convenience, I think.
>
> In any event, I used processing.tools.system.getTempFilename() to set a
> filename and that gets me past the 'Incorrect parameter value for
> output' error.
>
> However, I am still getting "Unable to execute algorithm".
>
> In AlgorithmExecutor, line 55:
> results, ok = alg.run(parameters, context, feedback)
>
> returns ({}, False)
>
> parameters contains:
>
> {'GRASS_REGION_CELLSIZE_PARAMETER': 0.008333, 'use': 1,
> 'GRASS_REGION_PARAMETER':
> '33.91415594546105,41.91384412832699,-4.679722671256611,5.467319665842772',
> 'input': ,
> 'GRASS_MIN_AREA_PARAMETER': 0.0001, 'GRASS_SNAP_TOLERANCE_PARAMETER':
> -1, 'type': 0, 'output':
> '/tmp/processing_da49d2e8c1bd4e249d1a5c6d42680149/5af9540a0b2212'}
>
> context contains:
>
> 
>
> and feedback contains:
>
> 
>
> Without a useful error message, it's very difficult to troubleshoot this
> on my own.
>
> Do you have any idea what might be wrong here?

I had another look at this. I suspect what might be happening is that
'tmp/processing_da49d2e8c1bd4e249d1a5c6d42680149/5af9540a0b2212' does
not exist, which makes GRASS bomb out with a 'No such file or
directory' error.

Are you running this script from within QGIS itself? You should see
the detailed log, including all the output from grass, by opening the
log window and switching to the "Processing" tab (alternatively
running from a terminal window will also show this output on the
command line).

Nyall
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Stephen]

Hi Nyall, everyone,

I built 3.1 master d5ffc5a2d9, and now I'm getting failures in parts of
my code that previously worked, plus no error messages.

If I walk through the code, exceptions get raised silently. Do I need to
turn on debugging somewhere?

Here's an example of what I see in the pycharm debugger:

> QSettings::value: Empty key passed
> QSettings::value: Empty key passed
> Computing population...
> sys:1: ResourceWarning: unclosed  family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, 
> laddr=('127.0.0.1', 42474), raddr=('127.0.0.1', 33003)>
> 
> Process finished with exit code 1



On 14.05.2018 11:51, Nyall Dawson wrote:
> On 14 May 2018 at 19:30, Stephen  wrote:
> 
>> Without a useful error message, it's very difficult to troubleshoot this
>> on my own.
> 
> Is this on master? The error messages have been greatly improved for
> 3.2, so I'd be interested to see what results you get there.
> 
> Nyall
> 

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Stephen]

On 14.05.2018 11:51, Nyall Dawson wrote:
> On 14 May 2018 at 19:30, Stephen  wrote:
> 
>> Without a useful error message, it's very difficult to troubleshoot this
>> on my own.
> 
> Is this on master? The error messages have been greatly improved for
> 3.2, so I'd be interested to see what results you get there.

This was on 3.0.1.

I'm on Gentoo, I'll have to try the -ebuild or spin my own to test
3.2...

Stephen

___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Nyall Dawson]

On 14 May 2018 at 19:30, Stephen  wrote:

> Without a useful error message, it's very difficult to troubleshoot this
> on my own.

Is this on master? The error messages have been greatly improved for
3.2, so I'd be interested to see what results you get there.

Nyall
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Stephen]

Hi Nyall,

I'm revisiting this problem now after a month. Thanks for your help on this.

On 11.04.2018 02:03, Nyall Dawson wrote:
>> After correcting the statement to the following:
>>
>>> file_grid_r = QgsRasterLayer()
>>> processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
>>> 'GRASS_REGION_PARAMETER': extent,
>>> 'output': file_grid_r,
>>> 'GRASS_REGION_CELLSIZE_PARAMETER':
>>> 0.008333,
>>> 'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
>>> 'GRASS_MIN_AREA_PARAMETER': 0.000100},
>>>  feedback=QgsProcessingFeedback())
>>
>>
>>  I get the message
>>
>>> Unable to execute algorithm
>>> Incorrect parameter value for output
> 
> The OUTPUT parameter should be set to the path to save the result,
> e.g. 'd:\blahblah.tif":
> 
>  processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
>  'GRASS_REGION_PARAMETER': extent,
>  'output': 'd:\blah.tif',
>  'GRASS_REGION_CELLSIZE_PARAMETER':
>  0.008333,
>  'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
>  'GRASS_MIN_AREA_PARAMETER': 0.000100},
>   feedback=QgsProcessingFeedback())
> 
> processing.run will return a dict a results, and inside that you'll
> automatically have a QgsRasterLayer loaded corresponding to this
> output.
> 
> E.g.
> 
> params = { ., 'OUTPUT': 'd:\blah.tif' }
> results = processing.run('grass7:v.to.rast', params, feedback )
> my_raster_layer = results['OUTPUT']
> assert my_raster_layer.isValid()
> 
> Here my_raster_layer will be a QgsRasterLayer, ready for further
> processing or adding to the project for visualisation.
> 
> I'm pondering whether we should automatically set output locations to
> temporary files if they aren't specified. E.g. if you don't set OUTPUT
> to 'd:\blah.tif', it'll fill it in automatically with a .tif file in
> your temp folder...
> 
> Nyall

Automatically setting output paths to temporary files would be a
sensible convenience, I think.

In any event, I used processing.tools.system.getTempFilename() to set a
filename and that gets me past the 'Incorrect parameter value for
output' error.

However, I am still getting "Unable to execute algorithm".

In AlgorithmExecutor, line 55:
results, ok = alg.run(parameters, context, feedback)

returns ({}, False)

parameters contains:

{'GRASS_REGION_CELLSIZE_PARAMETER': 0.008333, 'use': 1,
'GRASS_REGION_PARAMETER':
'33.91415594546105,41.91384412832699,-4.679722671256611,5.467319665842772',
'input': ,
'GRASS_MIN_AREA_PARAMETER': 0.0001, 'GRASS_SNAP_TOLERANCE_PARAMETER':
-1, 'type': 0, 'output':
'/tmp/processing_da49d2e8c1bd4e249d1a5c6d42680149/5af9540a0b2212'}

context contains:



and feedback contains:



Without a useful error message, it's very difficult to troubleshoot this
on my own.

Do you have any idea what might be wrong here?

Thanks

Stephen Bosch
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Nyall Dawson]

On 10 April 2018 at 20:25, Stephen Bosch  wrote:
> Hi Nyall:
>
> On Tue, Apr 10, 2018 at 2:20 AM, Nyall Dawson 
> wrote:
>>
>> On 10 April 2018 at 01:03, Stephen Bosch  wrote:
>>
>> > I've tried calling the algorithm this way:
>> >
>> >> file_grid_r = processing.run("grass7:v.to.rast", {'INPUT': grid_v,
>> >> 'type':
>> >> 0, 'use': 1, 'GRASS_REGION_PARAMETER': extent}, feedback=None)
>> >
>>
>> Looks correct
>>
>> > But it fails. On the console, I see
>> >
>> >> sys:1: ResourceWarning: unclosed > >> family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
>> >> laddr=('127.0.0.1', 41358), raddr=('127.0.0.1', 37307)>
>>
>> That can be ignored - it's unrelated.
>
>
> (Messages like this occur only when the process exits with a code other than
> 0.)
>
> I should add that, without using the debugger, I wouldn't be able to see the
> error at all. Is that intended behaviour? Is there any way to turn it on?
> The absence of exceptions and tracebacks make things fail silently, which
> can have nasty consequences.
>
>>
>> >
>> > If I step through the code in PyCharm and trace the exception, I see an
>> > AttributeError: 'NoneType' object has no attribute 'messageBar'.
>>
>> Where is this thrown? I suspect the problem is because you're passing
>> "feedback=None" and somewhere there's code which is expecting a valid
>> feedback object. Can you let me know which line the exception is
>> thrown on and I'll fix?
>
>
> It is thrown in line 42 of MessageBarProgress.py:
> __init__ [MessageBarProgress.py:42]  id:94435226437000

Thanks, fixed in
https://github.com/qgis/QGIS/commit/d5573a5a7bf5a32490b50f528da10c6f58871eab

>
> After correcting the statement to the following:
>
>> file_grid_r = QgsRasterLayer()
>> processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
>> 'GRASS_REGION_PARAMETER': extent,
>> 'output': file_grid_r,
>> 'GRASS_REGION_CELLSIZE_PARAMETER':
>> 0.008333,
>> 'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
>> 'GRASS_MIN_AREA_PARAMETER': 0.000100},
>>  feedback=QgsProcessingFeedback())
>
>
>  I get the message
>
>> Unable to execute algorithm
>> Incorrect parameter value for output

The OUTPUT parameter should be set to the path to save the result,
e.g. 'd:\blahblah.tif":

 processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
 'GRASS_REGION_PARAMETER': extent,
 'output': 'd:\blah.tif',
 'GRASS_REGION_CELLSIZE_PARAMETER':
 0.008333,
 'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
 'GRASS_MIN_AREA_PARAMETER': 0.000100},
  feedback=QgsProcessingFeedback())

processing.run will return a dict a results, and inside that you'll
automatically have a QgsRasterLayer loaded corresponding to this
output.

E.g.

params = { ., 'OUTPUT': 'd:\blah.tif' }
results = processing.run('grass7:v.to.rast', params, feedback )
my_raster_layer = results['OUTPUT']
assert my_raster_layer.isValid()

Here my_raster_layer will be a QgsRasterLayer, ready for further
processing or adding to the project for visualisation.

I'm pondering whether we should automatically set output locations to
temporary files if they aren't specified. E.g. if you don't set OUTPUT
to 'd:\blah.tif', it'll fill it in automatically with a .tif file in
your temp folder...

Nyall

> I've tried a couple of variations (specifying 'output' as an index and
> assigning it to a variable while setting 'output': None in the dictionary,
> for example) and get the same error.
>
> In my 2.x code, I just used the index and assigned the return to a variable.
> The implementation of v.to.rast for QGIS 3 Processing seems to require a
> value for output, and simply putting an variable doesn't work either (it
> complains that the variable is undefined).
>
> With a few more hints, I'm sure I'll get the hang of this ;-)
>
> Thanks
>
> Stephen
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Stephen Bosch]

Hi Nyall:

On Tue, Apr 10, 2018 at 2:20 AM, Nyall Dawson 
wrote:

> On 10 April 2018 at 01:03, Stephen Bosch  wrote:
>
> > I've tried calling the algorithm this way:
> >
> >> file_grid_r = processing.run("grass7:v.to.rast", {'INPUT': grid_v,
> 'type':
> >> 0, 'use': 1, 'GRASS_REGION_PARAMETER': extent}, feedback=None)
> >
>
> Looks correct
>
> > But it fails. On the console, I see
> >
> >> sys:1: ResourceWarning: unclosed  >> family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
> >> laddr=('127.0.0.1', 41358), raddr=('127.0.0.1', 37307)>
>
> That can be ignored - it's unrelated.
>

(Messages like this occur only when the process exits with a code other
than 0.)

I should add that, without using the debugger, I wouldn't be able to see
the error at all. Is that intended behaviour? Is there any way to turn it
on? The absence of exceptions and tracebacks make things fail silently,
which can have nasty consequences.


> >
> > If I step through the code in PyCharm and trace the exception, I see an
> > AttributeError: 'NoneType' object has no attribute 'messageBar'.
>
> Where is this thrown? I suspect the problem is because you're passing
> "feedback=None" and somewhere there's code which is expecting a valid
> feedback object. Can you let me know which line the exception is
> thrown on and I'll fix?
>

It is thrown in line 42 of MessageBarProgress.py:
__init__ [MessageBarProgress.py:42]  id:94435226437000

In the meantime try passing a feedback object:
>
>feedback = QgsProcessingFeedback()
>

That seemed to correct the messageBar error, thanks.

After correcting the statement to the following:

file_grid_r = QgsRasterLayer()
> processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
> 'GRASS_REGION_PARAMETER': extent,
> 'output': file_grid_r,
> 'GRASS_REGION_CELLSIZE_PARAMETER':
> 0.008333,
> 'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
> 'GRASS_MIN_AREA_PARAMETER': 0.000100},
>  feedback=QgsProcessingFeedback())
>

 I get the message

Unable to execute algorithm
> Incorrect parameter value for output
>

I've tried a couple of variations (specifying 'output' as an index and
assigning it to a variable while setting 'output': None in the dictionary,
for example) and get the same error.

In my 2.x code, I just used the index and assigned the return to a
variable. The implementation of v.to.rast for QGIS 3 Processing seems to
require a value for output, and simply putting an variable doesn't work
either (it complains that the variable is undefined).

With a few more hints, I'm sure I'll get the hang of this ;-)

Thanks

Stephen
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

Re: [Qgis-user] How do I pass parameters to grass7 algorithms with Processing.run?

[Nyall Dawson]

On 10 April 2018 at 01:03, Stephen Bosch  wrote:

> The input types are described with terms like
> ""; what does this mean exactly? Do I
> need to pass class instances?

Nope, strings/ints/layer refs/etc are fine. The class is shown in the
help to indicate what values are acceptable (e.g. you can't use a
layer reference for a point parameter!)

> I've tried calling the algorithm this way:
>
>> file_grid_r = processing.run("grass7:v.to.rast", {'INPUT': grid_v, 'type':
>> 0, 'use': 1, 'GRASS_REGION_PARAMETER': extent}, feedback=None)
>

Looks correct

> But it fails. On the console, I see
>
>> sys:1: ResourceWarning: unclosed > family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0,
>> laddr=('127.0.0.1', 41358), raddr=('127.0.0.1', 37307)>

That can be ignored - it's unrelated.
>
> If I step through the code in PyCharm and trace the exception, I see an
> AttributeError: 'NoneType' object has no attribute 'messageBar'.

Where is this thrown? I suspect the problem is because you're passing
"feedback=None" and somewhere there's code which is expecting a valid
feedback object. Can you let me know which line the exception is
thrown on and I'll fix?

In the meantime try passing a feedback object:

   feedback = QgsProcessingFeedback()


Nyall

> I'm not sure if this is a bug in QGIS or whether there is something wrong
> with my parameters.
>
> Do you have any suggestions or ideas?
>
> Thanks
>
> Stephen
>
>
>
> ___
> Qgis-user mailing list
> Qgis-user@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
___
Qgis-user mailing list
Qgis-user@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user