Re: [GRASS-dev] [GRASS GIS] #2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce

[GRASS GIS]

#2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce
-+-
  Reporter:  dylan   |  Owner:  grass-dev@…
  Type:  defect  | Status:  new
  Priority:  normal  |  Milestone:  7.2.3
 Component:  Raster  |Version:  unspecified
Resolution:  |   Keywords:
   CPU:  x86-64  |   Platform:  Linux
-+-

Comment (by dylan):

 Replying to [comment:32 mmetz]:

 > Markus Neteler in particular spent a lot of time to fix various systems
 for parallel execution of GRASS commands. GRASS itself was never the
 problem, instead the main problem was that the multiple outputs to be
 written to a single storage device were too much for that storage device.

 OK. Good to know. Are there any other diagnostics for these kind of
 problems, other than looking through the output from `dmesg` or kernel
 messages? I typically run `dstat` while developing parallel processing
 scripts, but I haven't noticed any disk-thrashing in this case. I see that
 the SSD is at about 5% of its "write bandwidth" as reported by `dstat`.

 Each instance of `r.sun` is writing out ~5000 rows of data over a period
 of about 8 minutes, so that is 5000 rows * 8 processes * 1/8 proc per
 minute * 1/60 minutes per second = ~80 write operations per second
 (assuming rows are written as processed). Are rows written 1-by-1, or in
 batches?


 > > I know this is a lot to ask, but did you try testing using ZLIB
 compression and running it multiple times? It took a couple of tiles
 before I noticed the error.
 >
 > I did use ZLIB compression when running the test with the data and
 scripts provided. Do you mean I should run the test several times with the
 same data?

 Yes (please). It wasn't until I had ran a couple of tiles that I
 encountered errors.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce

[GRASS GIS]

#2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce
-+-
  Reporter:  dylan   |  Owner:  grass-dev@…
  Type:  defect  | Status:  new
  Priority:  normal  |  Milestone:  7.2.3
 Component:  Raster  |Version:  unspecified
Resolution:  |   Keywords:
   CPU:  x86-64  |   Platform:  Linux
-+-

Comment (by mmetz):

 Replying to [comment:31 dylan]:
 > Replying to [comment:30 mmetz]:
 > > Replying to [comment:29 dylan]:
 > > >
 > > > [...] Note that I don't have any issues with any other GRASS
 commands, or (as far as I can tell) general usage on this machine. I only
 see these errors when working with GRASS commands that:
 > > >
 > > >   * take a long time to run: `r.sun` or `t.rast.mapcalc` ([http
 ://osgeo-org.1560.x6.nabble.com/Error-reading-raster-data-for-row-xxx-
 only-when-using-r-series-and-t-rast-series-td5229569.html e.g. a couple of
 years ago])
 > > >   * operate on moderately large, floating-point maps
 > > >   * are done in parallel, either via GNU `parallel` or as
 implemented in the temporal suite of modules
 > > >
 > > > ...hence the extreme difficulty in recreating the errors or further
 debugging.
 > >
 > > Unfortunately, I was not able to recreate these errors with the
 provided test data and scripts.
 > >
 > > I still think this is some obscure disk IO error. You could try to use
 `nice`, e.g. `nice r.sun ...` and `nice r.mapcalc ...` in `daily-rad.sh`.
 At least this helps when running many GRASS modules in parallel on HPC
 systems where results are written out to one single storage device.
 >
 > Well thank you very much for all of your patience, patches, and testing.
 I'll try the `nice` option. For now, I think that I can tolerate the much
 lower frequency of errors after switching to LZ4 compression. Perhaps the
 faster speed of LZ4 lowers the probability of concurrent write operations.

 I don't think so because because with LZ4, more data need to be written,
 which takes longer.

 >
 > It is still quite puzzling that this kind of error has come up on
 several different machines while tracking GRASS trunk over a 10 year
 period. Maybe this is a subtle hint that it is time to build a new
 workstation...

 Markus Neteler in particular spent a lot of time to fix various systems
 for parallel execution of GRASS commands. GRASS itself was never the
 problem, instead the main problem was that the multiple outputs to be
 written to a single storage device were too much for that storage device.
 >
 > I know this is a lot to ask, but did you try testing using ZLIB
 compression and running it multiple times? It took a couple of tiles
 before I noticed the error.

 I did use ZLIB compression when running the test with the data and scripts
 provided. Do you mean I should run the test several times with the same
 data?
 >
 > There will be an opportunity to test these same scripts in an HPC
 environment over the next two months. I'll be sure and report back any
 findings from those tests.

 An HPC environment basically consists of a job scheduler (e.g slurm) and
 any number of execution hosts. You can set up a minimal HPC environment on
 a single standard workstation. More execution hosts mean more (hardware)
 trouble.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce

[GRASS GIS]

#2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce
-+-
  Reporter:  dylan   |  Owner:  grass-dev@…
  Type:  defect  | Status:  new
  Priority:  normal  |  Milestone:  7.2.3
 Component:  Raster  |Version:  unspecified
Resolution:  |   Keywords:
   CPU:  x86-64  |   Platform:  Linux
-+-

Comment (by dylan):

 Replying to [comment:30 mmetz]:
 > Replying to [comment:29 dylan]:
 > >
 > > [...] Note that I don't have any issues with any other GRASS commands,
 or (as far as I can tell) general usage on this machine. I only see these
 errors when working with GRASS commands that:
 > >
 > >   * take a long time to run: `r.sun` or `t.rast.mapcalc` ([http
 ://osgeo-org.1560.x6.nabble.com/Error-reading-raster-data-for-row-xxx-
 only-when-using-r-series-and-t-rast-series-td5229569.html e.g. a couple of
 years ago])
 > >   * operate on moderately large, floating-point maps
 > >   * are done in parallel, either via GNU `parallel` or as implemented
 in the temporal suite of modules
 > >
 > > ...hence the extreme difficulty in recreating the errors or further
 debugging.
 >
 > Unfortunately, I was not able to recreate these errors with the provided
 test data and scripts.
 >
 > I still think this is some obscure disk IO error. You could try to use
 `nice`, e.g. `nice r.sun ...` and `nice r.mapcalc ...` in `daily-rad.sh`.
 At least this helps when running many GRASS modules in parallel on HPC
 systems where results are written out to one single storage device.

 Well thank you very much for all of your patience, patches, and testing.
 I'll try the `nice` option. For now, I think that I can tolerate the much
 lower frequency of errors after switching to LZ4 compression. Perhaps the
 faster speed of LZ4 lowers the probability of concurrent write operations.

 It is still quite puzzling that this kind of error has come up on several
 different machines while tracking GRASS trunk over a 10 year period. Maybe
 this is a subtle hint that it is time to build a new workstation...

 I know this is a lot to ask, but did you try testing using ZLIB
 compression and running it multiple times? It took a couple of tiles
 before I noticed the error.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce

[GRASS GIS]

#2764: corrupt data written to FCELL and DCELL rasters, hard to re-produce
-+-
  Reporter:  dylan   |  Owner:  grass-dev@…
  Type:  defect  | Status:  new
  Priority:  normal  |  Milestone:  7.2.3
 Component:  Raster  |Version:  unspecified
Resolution:  |   Keywords:
   CPU:  x86-64  |   Platform:  Linux
-+-

Comment (by mmetz):

 Replying to [comment:29 dylan]:
 >
 > [...] Note that I don't have any issues with any other GRASS commands,
 or (as far as I can tell) general usage on this machine. I only see these
 errors when working with GRASS commands that:
 >
 >   * take a long time to run: `r.sun` or `t.rast.mapcalc` ([http://osgeo-
 org.1560.x6.nabble.com/Error-reading-raster-data-for-row-xxx-only-when-
 using-r-series-and-t-rast-series-td5229569.html e.g. a couple of years
 ago])
 >   * operate on moderately large, floating-point maps
 >   * are done in parallel, either via GNU `parallel` or as implemented in
 the temporal suite of modules
 >
 > ...hence the extreme difficulty in recreating the errors or further
 debugging.

 Unfortunately, I was not able to recreate these errors with the provided
 test data and scripts.

 I still think this is some obscure disk IO error. You could try to use
 `nice`, e.g. `nice r.sun ...` and `nice r.mapcalc ...` in `daily-rad.sh`.
 At least this helps when running many GRASS modules in parallel on HPC
 systems where results are written out to one single storage device.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.0
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by madi):

 Can't check this now, as I'm currently not behind a proxy. Operating
 system at the time of reporting was CentOS.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3179: g.extension not installing addons behind proxy

[GRASS GIS]

#3179: g.extension not installing addons behind proxy
--+
  Reporter:  madi |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:  7.4.0
 Component:  Addons   |Version:  unspecified
Resolution:   |   Keywords:  g.extension, proxy, addons
   CPU:  Unspecified  |   Platform:  Unspecified
--+

Comment (by neteler):

 Please indicate the operating system, the output of `g.version -reg` and
 the Python version.

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3476: Patch to fix various spelling errors

[GRASS GIS]

#3476: Patch to fix various spelling errors
-+-
  Reporter:  Bas Couwenberg  |  Owner:  grass-dev@…
  Type:  enhancement | Status:  closed
  Priority:  normal  |  Milestone:  7.4.0
 Component:  Default |Version:  svn-releasebranch74
Resolution:  fixed   |   Keywords:
   CPU:  Unspecified |   Platform:  All
-+-

Comment (by neteler):

 In [changeset:"72063" 72063]:
 {{{
 #!CommitTicketReference repository="" revision="72063"
 correct typos (thanks to Bas Couwenberg), fixes #3476
 }}}

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3476: Patch to fix various spelling errors

[GRASS GIS]

#3476: Patch to fix various spelling errors
-+-
  Reporter:  Bas Couwenberg  |  Owner:  grass-dev@…
  Type:  enhancement | Status:  closed
  Priority:  normal  |  Milestone:  7.4.0
 Component:  Default |Version:  svn-releasebranch74
Resolution:  fixed   |   Keywords:
   CPU:  Unspecified |   Platform:  All
-+-
Changes (by neteler):

 * status:  new => closed
 * resolution:   => fixed


Comment:

 In [changeset:"72062" 72062]:
 {{{
 #!CommitTicketReference repository="" revision="72062"
 correct typos (thanks to Bas Couwenberg), fixes #3476
 }}}

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #2755: Commandline switch "/S" is not working anymore

[GRASS GIS]

#2755: Commandline switch "/S" is not working anymore
---+-
  Reporter:  derseppel |  Owner:  grass-dev@…
  Type:  defect| Status:  new
  Priority:  normal|  Milestone:  7.0.6
 Component:  Installation  |Version:  7.0.1
Resolution:|   Keywords:  wingrass
   CPU:  Unspecified   |   Platform:  MSWindows 7
---+-

Comment (by aaronsmiley):

 Hey there, thank you guys for making the software and creating a /S option
 for software packaging!

 I'm currently trying to package this for a University lab image, and I
 can't figure out how to suppress the prompt for VC++.  I have no problem
 making sure the dependencies are there in advance.  Is there some way I
 can suppress this message so the software will install silently?

 Thank you again!

 Aaron

--
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: [Action required] Please prepare the ideas page for GSoC 2018

[Moritz Lennert]

https://trac.osgeo.org/grass/wiki/GSoC/2018

Am 9. Januar 2018 16:02:16 MEZ schrieb Margherita Di Leo :
>-- Forwarded message --
>From: Margherita Di Leo 
>Date: Tue, Jan 9, 2018 at 3:57 PM
>Subject: [Action required] Please prepare the ideas page for GSoC 2018
>To: OSGeo Discussions , OSGeo-SoC <
>s...@lists.osgeo.org>
>Cc: "gsoc-adminosgeo.org" 
>
>
>Dear members of the Community,
>
>this is a reminder that to participate in GSoC also this year we need
>you
>to send us (admins:  ) the URL for your project's
>ideas page.
>
>Every idea should indicate a title, a description, 2 mentors and a test
>for
>the students to submit to your evaluation. The test aims at evaluating
>if
>the student is capable for the project, so please design the test
>having in
>mind the skills required to complete the project.
>
>Time is short, we need to collect all URLs by Jan 20.
>
>Thanks and please forward to your project community.
>
>Cheers,
>
>-- 
>Margherita Di Leo
>
>
>
>-- 
>Margherita Di Leo
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Fwd: [Action required] Please prepare the ideas page for GSoC 2018

[Margherita Di Leo]

-- Forwarded message --
From: Margherita Di Leo 
Date: Tue, Jan 9, 2018 at 3:57 PM
Subject: [Action required] Please prepare the ideas page for GSoC 2018
To: OSGeo Discussions , OSGeo-SoC <
s...@lists.osgeo.org>
Cc: "gsoc-adminosgeo.org" 


Dear members of the Community,

this is a reminder that to participate in GSoC also this year we need you
to send us (admins:  ) the URL for your project's
ideas page.

Every idea should indicate a title, a description, 2 mentors and a test for
the students to submit to your evaluation. The test aims at evaluating if
the student is capable for the project, so please design the test having in
mind the skills required to complete the project.

Time is short, we need to collect all URLs by Jan 20.

Thanks and please forward to your project community.

Cheers,

-- 
Margherita Di Leo



-- 
Margherita Di Leo
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev