Re: [GRASS-dev] i.sentinel.import does not list downloaded data

[Martin Landa]

Hi,

st 15. 8. 2018 v 22:28 odesílatel Markus Neteler  napsal:

> ## ... nothing reported while the data are there:
> GRASS 7.5.svn (ireland):/scratch/ireland > ls -lart data/
> total 3267768
> -rw-rw-r-- 1 mundialis mundialis  784858224 Aug 14 12:32
> S2A_MSIL2A_20180628T115401_N0208_R023_T29ULT_20180628T125228.zip

right, that's because you downloaded Sentinel products by sentinelsat
directly and not by i.sentinel.download.

i.sentinel.download unzips downloaded archives, and i.sentinel.import
expects uncompressed data. I wanted to change this behaviour some time
ago. i.sentinel.import should read zipped archives directly. Will
change it in the next days.

Ma

-- 
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3620: r.buffer crashes when trying to buffer rasterized polygons

[GRASS GIS]

#3620: r.buffer crashes when trying to buffer rasterized polygons
---+
  Reporter:  dido  |  Owner:  grass-dev@…
  Type:  defect| Status:  new
  Priority:  critical  |  Milestone:
 Component:  Raster|Version:  7.4.0
Resolution:|   Keywords:  r.buffer crash
   CPU:  x86-64|   Platform:  MSWindows 7
---+
Changes (by dido):

 * priority:  major => critical


-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3620: r.buffer crashes when trying to buffer rasterized polygons

[GRASS GIS]

#3620: r.buffer crashes when trying to buffer rasterized polygons
-+
  Reporter:  dido|  Owner:  grass-dev@…
  Type:  defect  | Status:  new
  Priority:  major   |  Milestone:
 Component:  Raster  |Version:  7.4.0
Resolution:  |   Keywords:  r.buffer crash
   CPU:  x86-64  |   Platform:  MSWindows 7
-+

Comment (by dido):

 Cannot attach a too-big file, please download it here:
 http://dido.bajhui.org/pics_aquarium/dem_blending_test.zip

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] [GRASS GIS] #3620: r.buffer crashes when trying to buffer rasterized polygons

[GRASS GIS]

#3620: r.buffer crashes when trying to buffer rasterized polygons
+-
 Reporter:  dido|  Owner:  grass-dev@…
 Type:  defect  | Status:  new
 Priority:  major   |  Milestone:
Component:  Raster  |Version:  7.4.0
 Keywords:  r.buffer crash  |CPU:  x86-64
 Platform:  MSWindows 7 |
+-
 At crash time r.buffer outputs this:

 Reading input raster map ...
  100%
 Finding buffer zones...
   57%
 > crash here

 Here is the windows crash log:

 Problem signature:
   Problem Event Name:   APPCRASH
   Application Name: r.buffer.EXE
   Application Version:  7.4.0.0
   Application Timestamp:5a6bcca6
   Fault Module Name:r.buffer.EXE
   Fault Module Version: 7.4.0.0
   Fault Module Timestamp:   5a6bcca6
   Exception Code:   c005
   Exception Offset: 164c
   OS Version:   6.1.7601.2.1.0.256.48
   Locale ID:1026
   Additional Information 1: bbf7
   Additional Information 2: bbf7706b53fbb0fa7695ff337b93e0f9
   Additional Information 3: 9d52
   Additional Information 4: 9d5215a4a4810f4d9a53d895d6bdf7b3

 Read our privacy statement online:
   http://go.microsoft.com/fwlink/?linkid=104288=0x0409

 If the online privacy statement is not available, please read our privacy
 statement offline:
   C:\Windows\system32\en-US\erofflps.txt


 Problem is manifested using a python script. Script and data provided as
 attachment.

 !!! The script must be tweaked slightly in the beginning to match the
 grass install path. A proper GRASS DB must be provided there with a BG_DEM
 location and a PERMANENT mapset projected in EPSG:32635 (UTM Z35N)

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [QGIS-Developer] Running grass algorithms in threads

[Markus Neteler]

On Wed, Aug 15, 2018 at 11:54 AM Rudi von Staden  wrote:
>
> Thanks all for your thoughts on this. I suspect the problem may be race 
> conditions being set up between 
> https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/grass7/Grass7Utils.py
>  and 
> https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/grass7/Grass7Algorithm.py.
>  It attempts to use an existing session and shared mapset which could be 
> problematic from the sounds of things. It might be an idea to add an optional 
> parameter to grass algorithms to not use shared mapsets and create a new 
> mapset for each process?

It just came in yesterday :)

See
https://lists.osgeo.org/pipermail/grass-user/2018-August/078936.html
[GRASS-user] New CLI option --tmp-location in 7.5 to enhance --exec

Perhaps this approach will solve the issues?

Best
Markus
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] i.sentinel.import does not list downloaded data

[Markus Neteler]

On Wed, Aug 15, 2018 at 11:50 AM Martin Landa  wrote:
> Hi,
>
> út 14. 8. 2018 v 14:58 odesílatel Markus Neteler  napsal:
> > which downloaded as desired (since i.sentinel.download does not have a
> > uuid parameter I used sentinelsat):
>
> now possible :-)

Testing:

GRASS 7.5.svn (ireland):~ > i.sentinel.download -l
settings=~/.esa_sentinelhub uuid=98f00e12-3b62-4c54-8554-c4439bb1ebef
98f00e12-3b62-4c54-8554-c4439bb1ebef 2018-06-28T11:54:01Z  0% S2MSI2A

Works - great, thank you!!

> $ i.sentinel.download settings=sentinel.txt
> uuid=98f00e12-3b62-4c54-8554-c4439bb1ebef output=data
>
> > # print
> > i.sentinel.import -p input=data
> > --> nothing, should list the channels?
>
> This seems to work for me.
>
> $ i.sentinel.import -p input=/tmp/sentinel
> /tmp/sentinel/S2A_MSIL2A_20180628T115401_N0208_R023_T29UMU_20180628T125228.SAFE/GRANULE/L2A_T29UMU_A015750_20180628T115400/IMG_DATA/R10m/T29UMU_20180628T115401_B03_10m.jp2
> 0 (EPSG: 32629)
> /tmp/sentinel/S2A_MSIL2A_20180628T115401_N0208_R023_T29UMU_20180628T125228.SAFE/GRANULE/L2A_T29UMU_A015750_20180628T115400/IMG_DATA/R10m/T29UMU_20180628T115401_B08_10m.jp2
> 0 (EPSG: 32629)
>
> Can you try on empty data directory with fresh addons?

# relative path
GRASS 7.5.svn (ireland):/scratch/ireland > i.sentinel.import -p input=data

# absolute path
GRASS 7.5.svn (ireland):/scratch/ireland > i.sentinel.import -p
input=/scratch/ireland/data

## ... nothing reported while the data are there:
GRASS 7.5.svn (ireland):/scratch/ireland > ls -lart data/
total 3267768
-rw-rw-r-- 1 mundialis mundialis  784858224 Aug 14 12:32
S2A_MSIL2A_20180628T115401_N0208_R023_T29ULT_20180628T125228.zip
-rw-rw-r-- 1 mundialis mundialis 1037024033 Aug 14 12:35
S2A_MSIL2A_20180628T115401_N0208_R023_T29UMT_20180628T125228.zip
-rw-rw-r-- 1 mundialis mundialis  955002429 Aug 14 12:51
S2A_MSIL2A_20180628T115401_N0208_R023_T29UMU_20180628T125228.zip
-rw-rw-r-- 1 mundialis mundialis  569305492 Aug 14 13:33
S2A_MSIL2A_20180628T115401_N0208_R023_T29ULU_20180628T125228.zip

# test with empty one:
GRASS 7.5.svn (ireland):/scratch/ireland > mkdir data2
GRASS 7.5.svn (ireland):/scratch/ireland > i.sentinel.import -p
input=/scratch/ireland/data2/
GRASS 7.5.svn (ireland):/scratch/ireland >

Unfortunately nothing yet...

Markus
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Trunk fails at configure phase on Ubuntu 18.04

[Markus Neteler]

On Wed, Aug 15, 2018 at 11:39 AM Maris Nartiss  wrote:
>
> Hello folks,
> I just updated my Ubuntu box from previous LTS to new LTS (18.04).
> When I try to configure a fresh trunk checkout, it fails to pass
> configure phase:
>
> ./configure --enable-debug --with-nls --with-cxx --with-bzlib
> --with-liblas --with-freetype
> --with-freetype-includes=/usr/include/freetype2 --with-X11
> --with-wxwidgets --with-python --with-sqlite --with-geos --with-blas
> --with-openmp --with-lapack --with-pdal --with-zstd
> configure: error: freetype: invalid package name

That's strange. I recently updated the Dockerfile in trunk to use
18.04 and it works smoothly:

https://hub.docker.com/r/neteler/grassgis7/~/dockerfile/

Perhaps compare to your install procedure?

Markus
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] partitioning of vector polygon?

[Anika Bettge]

Hi,

does anyone know how I can split a polygon into n polygons?

For performing area image classification I need to spatially divide my
training data (points in the polygon) into spatial clumps with those I
want to train different classifiers, so that I do not need to load the
image data for the whole polygon at once in the mapset. The image area
contains 4*10^9 pixels.


Best regards,

Anika


-- 
 
  Anika Bettge
  - Anwendungsentwicklerin -

  mundialis GmbH & Co. KG
  Kölnstraße 99
  53111 Bonn

  Tel: +49 (0)228 / 38 75 80 -80
  Fax: +49 (0)228 / 96 28 99 -57

  Email: bet...@mundialis.de
  Web: https://www.mundialis.de

  Amtsgericht Bonn, HRA 8528
  Komplementärin: mundialis Verwaltungsgesellschaft mbH
  vertreten durch: Dr. Markus Neteler, Hinrich Paulsen, Till Adams
  
  Informationen über Ihre gespeicherten Daten finden Sie auf unserer Homepage 
unter folgendem Link: 
  https://www.mundialis.de/datenschutzerklaerung

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [EXTERNAL] GSoC 2018 final report week 13 - GRASS GIS module for Sentinel-2 cloud and shadow detection

[Doug Newcomb]

Roberta,
Thank you for this work!
Doug

On Mon, Aug 13, 2018 at 4:21 AM Roberta Fagandini 
wrote:

> Hi all!
> I'm Roberta Fagandini and this is the final report of my GSoC project.
>
> The title of the project is "GRASS GIS module for Sentinel-2 cloud and
> shadow detection". It adds new tools for the processing of Sentinel 2
> images to GRASS GIS software (Organization: OSGeo).
>
> *Abstract:*
> Optical sensors are unable to penetrate clouds leading to related
> anomalous reflectance values. Unlike Landsat images, Sentinel 2 datasets do
> not include thermal and Quality Assessment bands that simplify the
> detection of clouds avoiding erroneous classification. At the same time,
> also clouds shadows on the ground lead to anomalous reflectance values
> which have to be taken into account during the image processing.
> The project creates a specific module for GRASS GIS application
> (i.sentinel.mask) which implements an automatic procedure for clouds and
> shadows detection for Sentinel 2 images. The procedure is based on an
> algorithm, developed within my PhD research, which allows to automatically
> identify clouds and their shadows applying some rules on reflectance values
> (values thresholds, comparisons between bands, etc.). These have been
> defined starting from rules found in literature and conveniently refined.
> In order to increase the accuracy of the final results, a control check is
> implemented. Clouds and shadows are spatially intersected in order to
> remove misclassified areas. The final outputs are two different vector maps
> (OGR standard formats), one for clouds and one for shadows.
> To run i.sentinel.mask, the bands of the desired Sentinel 2 images have to
> be imported and the atmospheric correction has to be applied.
> In order to make the data preparation easier, another GRASS GIS addon module
> has been developed within the GSoC project.
> i.sentinel.preproc is a module for the preprocessing of Sentinel 2 images
> (Level-1C Single Tile product) which wraps the import and the atmospheric
> correction using respectively two existing GRASS GIS modules,
> i.sentinel.import and i.atcorr.
>
> *The state of the art before the project:*
> Before this GSoC 2018 project, no modules for the detection of clouds and
> shadows were available for Sentinel 2 images. Only a specific module for
> Landsat automatic cloud coverage assessment was available within GRASS GIS
> (i.landsat.acca) while regarding shadows, no specific module was available.
> Moreover, performing the atmospheric correction was a bit complicated
> especially for unexperienced users who have to process one band at a time
> and provide all input parameters manually.
>
> *The added value that the project brought to GRASS GIS:*
> Now a specific module for clouds and shadows detection, i.sentinel.mask, is
> available in GRASS GIS.
> Moreover, i.sentinel.preproc provides a simplified module which allows
> importing images and performing the atmospheric correction avoiding users
> to supply all the required input parameters manually. The module should
> help users in preparing data to use as input for i.sentinel.mask. In fact,
> it makes especially the atmospheric correction procedure easier and faster
> because it allows performing atmospheric correction of all bands of a
> Sentinel 2 scene with a single process and it retrieves most of the
> required input parameters from the image itself. Moreover, one of the
> possible output of i.sentinel.preproc is a text file to be used as input
> for i.sentinel.mask.
>
> *Follow up:*
> Both i.sentinel.mask and i.sentinel.preproc are complete and working
> modules which can be easily installed with g.extension from the official
> GRASS GIS SVN repository.
> Obviously, they can be improved therefore the next steps could be:
>
>- Implementation of other existing algorithms of clouds and shadows
>detection (i.sentinel.mask)
>- Implementation of a new download procedure avoiding dependencies
>(i.sentinel.preproc)
>- Integration of the Topographic Correction (i.sentinel.preproc)
>
> NOTE: Implementation of other existing algorithms of clouds and shadows
> detection was one of the possible goals of the GSoC project but the coding
>  and debugging of some parts of the two addons required more time than
> expected.
>
> *Permanent links:*
>
> *Code developed during the GSoC coding period: *
> https://github.com/RobiFag/GRASS_clouds_and_shadows
>
> *Codes on the official GRASS GIS SVN repository:*
>
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.sentinel.mask
>
> https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.sentinel.preproc
>
> *Documentation:*
> https://grass.osgeo.org/grass74/manuals/addons/i.sentinel.mask.html
> https://grass.osgeo.org/grass74/manuals/addons/i.sentinel.preproc.html
>
> *Weekly reports: *
> https://trac.osgeo.org/grass/wiki/GSoC/2018/CloudsAndShadowsDetection
>
> *Images to showcase the project:*
> i.sentinel.mask
>

Re: [GRASS-dev] [QGIS-Developer] Running grass algorithms in threads

[Rudi von Staden]

Thanks all for your thoughts on this. I suspect the problem may be race
conditions being set up between
https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/grass7/Grass7Utils.py
and
https://github.com/qgis/QGIS/blob/master/python/plugins/processing/algs/grass7/Grass7Algorithm.py.
It attempts to use an existing session and shared mapset which could be
problematic from the sounds of things. It might be an idea to add an
optional parameter to grass algorithms to not use shared mapsets and create
a new mapset for each process?

The Grass7 log output from my script is below. The pairs of input rasters
for the parallel algorithms should be:
hm1.tif, LS90.tif
hm2.tif, LS14.tif
hm3.tif, HS90.tif
hm4.tif, LS90.tif

If could be that not all of the grass commands were logged, but it looks
like HS90.tif and LS14.tif were never passed as parameters to grass. I also
added a typical log when the algorithms are executed sequentially (3800-25.tif
is reused instead of hm1-4).

Kind regards,
Rudi


~~~ [ executed in parallel ] ~~~

2018-08-15T10:09:00 INFOprocessInputs end. Commands: ['g.proj -c
proj4="+proj=aea +lat_1=-24 +lat_2=-32 +lat_0=0 +lon_0=24 +x_0=0 +y_0=0
+datum=WGS84 +units=m +no_defs"', 'r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\cb4c839fbd6f4419873dbc29ffa2b973\\hm4.tif"
band=1 output="rast_5b73df9c811b22" --overwrite -o', 'r.external
input="C:\\Users\\rudi\\GIS\\Projects\\LizeSuitableHabitatModel\\Analysis\\Data\\HS14.tif"
band=1 output="rast_5b73df9c811b23" --overwrite -o', 'g.region
n=-2332657.7474449594 s=-3803200.178477332 e=984393.5909882308
w=-834195.3236270341 res=92.0757893076434']


2018-08-15T10:09:00 INFOprocessCommands end. Commands: ['g.proj -c
proj4="+proj=aea +lat_1=-24 +lat_2=-32 +lat_0=0 +lon_0=24 +x_0=0 +y_0=0
+datum=WGS84 +units=m +no_defs"', 'r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\cb4c839fbd6f4419873dbc29ffa2b973\\hm4.tif"
band=1 output="rast_5b73df9c811b22" --overwrite -o', 'r.external
input="C:\\Users\\rudi\\GIS\\Projects\\LizeSuitableHabitatModel\\Analysis\\Data\\HS14.tif"
band=1 output="rast_5b73df9c811b23" --overwrite -o', 'g.region
n=-2332657.7474449594 s=-3803200.178477332 e=984393.5909882308
w=-834195.3236270341 res=92.0757893076434', 'r.stats.zonal
base=rast_5b73df9c811b22 cover=rast_5b73df9c811b23 method="average"
output=output87eadbc7948b4b5990773af2261f0db8 --overwrite']


2018-08-15T10:09:00 INFOprocessInputs end. Commands: ['r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\79ea35021d26468c8220582c6397dd18\\hm1.tif"
band=1 output="rast_5b73df9c865914" --overwrite -o', 'r.external
input="C:\\Users\\rudi\\GIS\\Projects\\LizeSuitableHabitatModel\\Analysis\\Data\\LS90.tif"
band=1 output="rast_5b73df9c865915" --overwrite -o', 'g.region
n=-2332657.7474449594 s=-3803200.178477332 e=984393.5909882308
w=-834195.3236270341 res=92.0757893076434']


2018-08-15T10:09:00 INFOprocessCommands end. Commands: ['r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\79ea35021d26468c8220582c6397dd18\\hm1.tif"
band=1 output="rast_5b73df9c865914" --overwrite -o', 'r.external
input="C:\\Users\\rudi\\GIS\\Projects\\LizeSuitableHabitatModel\\Analysis\\Data\\LS90.tif"
band=1 output="rast_5b73df9c865915" --overwrite -o', 'g.region
n=-2332657.7474449594 s=-3803200.178477332 e=984393.5909882308
w=-834195.3236270341 res=92.0757893076434', 'r.stats.zonal
base=rast_5b73df9c865914 cover=rast_5b73df9c865915 method="average"
output=output8a381f2f7cc44b4eae194a33b1839457 --overwrite']


2018-08-15T10:09:00 INFOprocessInputs end. Commands: ['r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\1d1e057229aa4ee88a8abcb009b78f41\\hm3.tif"
band=1 output="rast_5b73df9c8ac116" --overwrite -o', 'g.region
n=-3680453.4502051217 s=-3763597.8879499235 e=-293721.76789138245
w=-478149.5738745922 res=92.0757893076434']


2018-08-15T10:09:00 INFOprocessCommands end. Commands: ['r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\1d1e057229aa4ee88a8abcb009b78f41\\hm3.tif"
band=1 output="rast_5b73df9c8ac116" --overwrite -o', 'g.region
n=-3680453.4502051217 s=-3763597.8879499235 e=-293721.76789138245
w=-478149.5738745922 res=92.0757893076434', 'r.stats.zonal
base=rast_5b73df9c8ac116 cover=rast_5b73df9c865915 method="average"
output=outputbef87f5fb08c43cd9e1cdd7ad44e5f1b --overwrite']


2018-08-15T10:09:00 INFOprocessInputs end. Commands: ['r.external
input="C:\\Users\\rudi\\AppData\\Local\\Temp\\processing_74832af828bb48a9bb0e64fd59d74f24\\4895f2e889a84f01aa852e7ca0035294\\hm2.tif"
band=1 output="rast_5b73df9c8defa7" --overwrite -o', 'g.region
n=-3680453.4502051217 s=-3763597.8879499235 e=-293721.76789138245
w=-478149.5738745922 

Re: [GRASS-dev] i.sentinel.import does not list downloaded data

[Martin Landa]

Hi,

út 14. 8. 2018 v 14:58 odesílatel Markus Neteler  napsal:
> which downloaded as desired (since i.sentinel.download does not have a
> uuid parameter I used sentinelsat):

now possible :-)

$ i.sentinel.download settings=sentinel.txt
uuid=98f00e12-3b62-4c54-8554-c4439bb1ebef output=data

> # print
> i.sentinel.import -p input=data
> --> nothing, should list the channels?

This seems to work for me.

$ i.sentinel.import -p input=/tmp/sentinel
/tmp/sentinel/S2A_MSIL2A_20180628T115401_N0208_R023_T29UMU_20180628T125228.SAFE/GRANULE/L2A_T29UMU_A015750_20180628T115400/IMG_DATA/R10m/T29UMU_20180628T115401_B03_10m.jp2
0 (EPSG: 32629)
/tmp/sentinel/S2A_MSIL2A_20180628T115401_N0208_R023_T29UMU_20180628T125228.SAFE/GRANULE/L2A_T29UMU_A015750_20180628T115400/IMG_DATA/R10m/T29UMU_20180628T115401_B08_10m.jp2
0 (EPSG: 32629)

Can you try on empty data directory with fresh addons?

Martin

-- 
Martin Landa
http://geo.fsv.cvut.cz/gwiki/Landa
http://gismentors.cz/mentors/landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Trunk fails at configure phase on Ubuntu 18.04

[Maris Nartiss]

Hello folks,
I just updated my Ubuntu box from previous LTS to new LTS (18.04).
When I try to configure a fresh trunk checkout, it fails to pass
configure phase:

./configure --enable-debug --with-nls --with-cxx --with-bzlib
--with-liblas --with-freetype
--with-freetype-includes=/usr/include/freetype2 --with-X11
--with-wxwidgets --with-python --with-sqlite --with-geos --with-blas
--with-openmp --with-lapack --with-pdal --with-zstd
configure: error: freetype: invalid package name

Here's the last part of sh -x:
+ test -n
+ ac_optarg=
+ echo --with-freetype
+ sed -e s/-*with-// -e s/=.*//
+ ac_package=freetype
+ echo freetype
+ sed s/[-_a-zA-Z0-9]//g
+ test -n y
+ echo configure: error: freetype: invalid package name
configure: error: freetype: invalid package name
+ exit 1

Any hints how to work around this issue?

It is not a freetype issue, as removing freetype from configure call
then results in failure with Python, or "freetype-includes: invalid
package name". Thus it is something bad in the configure "magic".

Thanks,
Māris.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [QGIS-Developer] Running grass algorithms in threads

[Stefan Blumentrath]

Dear Rudi, Nyall,

GRASS is being used on HPC systems for heavily parallelisation. So, in 
principle, the answer is yes, you can for sure run GRASS algorithms in parallel.
On Linux, I often run several commands in parallel using xargs. So it works 
just fine in many cases. GRASS also has some specific python functions for 
parallel processing. See also:
https://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs
https://grasswiki.osgeo.org/wiki/Parallelizing_Scripts

However, if GRASS algorithms can be run in parallel in this particular case 
depends.

E.g., if the algorithm in question temporarily modifies the computational 
region, parallel processes can get in the way for each other.
Also, with SQLite as DB backend writing several vector maps (and attribute 
tables) in parallel will be a problem (due to SQLite locks).

In addition, if GRASS commands can be executed in parallel in the QGIS 
Processing framework is probably yet another question, depending on how e.g. 
QGIS handles data management (locations and mapsets) esp. in more complex 
workflows / models...

CCing also grass-dev list for more qualified answers...

Cheers
Stefan


-Original Message-
From: QGIS-Developer  On Behalf Of 
Nyall Dawson
Sent: onsdag 15. august 2018 01:10
To: Rudi von Staden 
Cc: qgis-developer 
Subject: Re: [QGIS-Developer] Running grass algorithms in threads

On Tue, 14 Aug 2018 at 21:43, Rudi von Staden  wrote:
>
> Hi all,
>
> The bottleneck in my script at the moment is the calculation of zonal stats 
> using 'grass7:r.stats.zonal'. I thought I might speed things up by using 
> QgsTask.fromFunction() or QgsProcessingAlgRunnerTask() to run these 
> calculations in parallel. In my tests of both approaches the tasks seem to 
> complete (task.status() == QgsTask.Complete), but the output file is only 
> generated for 1 of 4 parallel tasks (the task that finishes first).
>
> I'm assuming this is because grass algorithms are not thread safe? Or am I 
> missing something in my implementation that could make this work?

I strongly suspect that grass algorithms cannot be run in parallel.
This is why they cannot run in the background in QGIS like the native/GDAL 
algorithms can. But I'd love for confirmation about this and whether there's 
any way to make GRASS multi-thread safe.

Because this is grass related (and not QGIS specific) I'd suggest asking on the 
grass mailing list, and relaying any responses back here.

Nyall

>
> Thanks,
> Rudi
>
>
>
> My code for the QgsTask approach is as below:
>
> def getZonal(task, habitatModelFile, cover):
> tempFile = QgsProcessingUtils.generateTempFilename("output.tif")
> processing.run("grass7:r.stats.zonal", {
> 'base':habitatModelFile,
> 'cover':cover,
> 'method':5,
> '-c':False,
> '-r':False,
> 'output':tempFile,
> 'GRASS_REGION_PARAMETER':None,
> 'GRASS_REGION_CELLSIZE_PARAMETER':0,
> 'GRASS_RASTER_FORMAT_OPT':'',
> 
> 'GRASS_RASTER_FORMAT_META':''},context=context,feedback=algFeedback)
>
> if task.isCanceled():
> deleteFile(tempFile)
> return
>
> return tempFile
>
> ls90Task = QgsTask.fromFunction('LS90', getZonal, 
> habitatModelFile=hm1, cover=ls90Layer)
> QgsApplication.taskManager().addTask(ls90Task)
> feedback.pushInfo("Calculating LS14 mean...") ls14Task = 
> QgsTask.fromFunction('LS14 ', getZonal, habitatModelFile=hm2, 
> cover=ls14Layer)
> QgsApplication.taskManager().addTask(ls14Task)
> hs90Task = QgsTask.fromFunction('HS90 ', getZonal, 
> habitatModelFile=hm3, cover=hs90Layer)
> QgsApplication.taskManager().addTask(hs90Task)
> hs14Task = QgsTask.fromFunction('HS14 ', getZonal, 
> habitatModelFile=hm4, cover=hs14Layer)
> QgsApplication.taskManager().addTask(hs14Task)
>
> while (len([t for t in [ls90Task.status(), ls14Task.status(), 
> hs90Task.status(),
> hs14Task.status()] if t in [QgsTask.Running, QgsTask.Queued]]) > 
> 0)
> and not feedback.isCanceled():
> sleep(1)
>
> if feedback.isCanceled():
> # some cleanup code (send task.cancel() and wait for tasks to terminate)
> break
>
> ls90Result = ls90Task.returned_values
> ls14Result = ls14Task.returned_values
> hs90Result = hs90Task.returned_values   # only this file exists
> hs14Result = hs14Task.returned_values
>
>
> ___
> QGIS-Developer mailing list
> qgis-develo...@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
QGIS-Developer mailing list
qgis-develo...@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev