Re: [GRASS-dev] RFC: variations of statistics in r.neighbors (and the stats lib)

[Markus Metz]

Hi Francesco,

the proposed change to r.neighbors is interesting, but maybe too specific:
you have introduced two new functions, and many more functions would be
needed to e.g. get the filtered standard deviation or median.

Therefore I suggest adding some filtering option to r.neighbors consisting
of a filtering function and a comparison operator. The filtering function
would be any of the currently supported neighborhood functions returning
some value. The comparison operator would be one of gt, ge, le, lt (>, >=,
<=, <). r.neighbors would then first get the result value of the filtering
function, then set all values in the neighborhood to NULL that do not
fulfil the condition "value  , then
call the actual neighborhood operation with the filtered values. This would
be more flexible, because the user can freely combine a neighborhood filter
function with a neighborhood operation.

Makes sense?


On Wed, Jan 25, 2023 at 10:55 AM Francesco P. Lovergine 
wrote:

> Hi,
>
> for some specific needs of a research project, I had to make a little
> change to r.neighbors (the target version was 7.8.5 but that's not
> essential).
>
> Essentially, the idea behind is computing first order statistics on partial
> populations as identified by selected quantiles (samples >= or <= of a
> threshold value of quantile).
>
> For that, I introduced average_ge_quantile and average_le_quantile
> operators modes.
>
>
> https://github.com/fpl/grass/commit/6b83795b037c6645a32d6a525cfdee3cc65d521c
>
> (the html file is still not updated)
>
> I'm not persuaded this is the most elegant way of doing this kind of
> things,
> maybe it would be better using an option (as in the case of -w for weighted
> operations) to compute average and possibly other statistics. Even, one
> could
> think in general to other multiple ways of selecting population on the
> base of
> quantiles/percentiles of population in a window.
>
> Any hint/opinion/alternative/critic about that?
>
> All this in the remote hypothesis that a pull request could have sense
> for such a kind of features.
>
> Thanks
>
>
> --
> Francesco P. Lovergine
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [gdal-dev] Moving GDAL GRASS driver in a dedicated repository ?

[Markus Metz]

Hi Even,

IMHO it has been a bit of a luxury that the GDAL GRASS driver was allowed
to exist as a regular GDAL supported format within frmts/grass. With every
new release of GDAL, you (the GDAL maintainers) also released a separate
new GDAL GRASS driver which was really nice of you!

Considering the workaround for this circular dependency, I agree that a
dedicated repository makes sense.

I personally don't use the GDAL GRASS driver at all (I just try to maintain
it), but I am aware that a number of projects use the GDAL GRASS driver.
Feedback from any affected projects would be helpful.

Markus M

On Thu, Nov 18, 2021 at 7:13 PM Even Rouault 
wrote:

> Hi,
>
> (writing to both GDAL and GRASS lists)
>
> Working on the transition to CMake as the GDAL build system, the
> particular status of the GRASS driver in GDAL raised my attention.
>
> (The following is based on my understanding. It has been ages since I
> didn't try this...)
>
> This driver is a bit odd in the sense that there's a cyclic dependency
> to work around, as GRASS links to GDAL , but the GDAL GRASS driver needs
> to be linked against GRASS.
>
> So the usual procedure is:
>
> - build GDAL without the GRASS driver
>
> - build GRASS against GDAL
>
> - build the GDAL GRASS driver from the separate gdal-grass tarball that
> GDAL distributes along its main tarball.
>
> With the current GDAL autoconf build system, there's also the
> possibility to rebuild GDAL with the GRASS driver builtin in libgdal,
> but that's a bit odd, since you need to make sure that this new libgdal
> is the one that GRASS will link against at runtime, otherwise chaos will
> ensure. I'm not sure if that's used. This is typically something I would
> *not* want to support in the new GDAL cmake build.
>
> All in all, given the particular nature of that driver, I believe it
> would be better in a dedicated repository, with its standalone build
> scripts, whose initial version could be just the ones of
> https://github.com/OSGeo/gdal/tree/master/frmts/grass/pkg, or evolve to
> CMake or whatever the maintainers of that driver would prefer. I believe
> this would make the situation clearer.
>
> Opinions ? and people interested in setting up that dedicated repository ?
>
> Even
>
> --
> http://www.spatialys.com
> My software is free, but my time generally not.
>
> ___
> gdal-dev mailing list
> gdal-...@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Renaming master branch to main branch

[Markus Metz]

On Fri, Sep 10, 2021 at 8:46 PM Vaclav Petras  wrote:
>
> On Fri, Sep 10, 2021 at 2:30 PM Markus Metz 
wrote:
>>
>> The reason for the rebase was that some tests were failing, apparently
because of a combination of renaming master to main and changes to GHA, and
a PR can only be squashed and merged if all tests are passed.
>
> Ah, that makes sense. You need to update in one way or another in this
case.

After `git rebase main` + `git push -f ...`, all the previously timed-out
tests for the affected PRs are passed, great!

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Renaming master branch to main branch

[Markus Metz]

On Fri, Sep 10, 2021 at 7:51 PM Vaclav Petras  wrote:
>
>
>
> On Fri, Sep 10, 2021 at 1:16 PM Markus Metz 
wrote:
>>
>> ...
>> $ git checkout raster_tempdir
>> $ git rebase main
>>
>> Successfully rebased and updated refs/heads/raster_tempdir.
>>
>> $ git status
>> On branch raster_tempdir
>> Your branch and 'metzm/raster_tempdir' have diverged,
>> and have 48 and 7 different commits each, respectively.
>>   (use "git pull" to merge the remote branch into yours)
>>
>> $ git push metzm raster_tempdir
>> ! [rejected]raster_tempdir -> raster_tempdir
(non-fast-forward)
>> error: failed to push some refs to 'github.com:metzm/grass.git'
>> hint: Updates were rejected because the tip of your current branch is
behind
>> hint: its remote counterpart. Integrate the remote changes (e.g.
>> hint: 'git pull ...') before pushing again.
>> hint: See the 'Note about fast-forwards' in 'git push --help' for
details.
>
>
> This is how it is supposed to behave when you do `git rebase`. Rebasing
re-applies the changes you made in raster_tempdir one by one on top of your
local main. This creates different commits in the sense that the commit
hashes are different. This causes the "have diverged" part. 48 on the local
raster_tmpdir is your 7 commits plus 41 commits which happened in the main
branch since you created raster_tempdir. 7 commits on metzm/raster_tempdir
is the original commits you made which have the original, different hash,
so Git considers them to be different.
>
> Force push with `git push -f metzm raster_tempdir` is appropriate here.
You will replace whatever is in the remote branch by your newly updated
(rebased) local branch content.

A force push helped, thanks! The commit history of the PR has not been
messed up, only the relevant commits are shown.

The reason for the rebase was that some tests were failing, apparently
because of a combination of renaming master to main and changes to GHA, and
a PR can only be squashed and merged if all tests are passed.

Thanks for the fast response!

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Renaming master branch to main branch

[Markus Metz]

Probably a silly question:

I'm having problems with rebasing my local feature branches for PRs, e.g.
for https://github.com/OSGeo/grass/pull/1786
My local main branch is up to date with the official main branch. In my
case the grass repo is the remote named origin, my fork is the remote named
metzm

$ git checkout raster_tempdir
$ git rebase main

Successfully rebased and updated refs/heads/raster_tempdir.

$ git status
On branch raster_tempdir
Your branch and 'metzm/raster_tempdir' have diverged,
and have 48 and 7 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

$ git push metzm raster_tempdir
! [rejected]raster_tempdir -> raster_tempdir (non-fast-forward)
error: failed to push some refs to 'github.com:metzm/grass.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Any hints on how to proceed from here without messing up the history of my
feature branch? Thanks!

Markus M


On Sun, Aug 22, 2021 at 9:26 PM Vaclav Petras  wrote:

> Dear all,
>
> The master branch was renamed to main. Please update your local
> repositories and workflows accordingly.
>
> ## 1. Local repo update and branch rename
>
> If you are contributing code or compiling source code from Git, you need
> to modify your local repository as well. Go to your fork's website on
> GitHub. It will warn you about the rename in the OSGeo repo. Go to settings
> and to the rename. Go to the main page of your fork again. Now it will warn
> you about the rename in your repo (your fork). It will give you
> instructions on renaming the local branch and making it point to the
> renamed branch. (The assumption is that your fork repo is the remote named
> origin.)
>
> Additionally, there still will be upstream/master and you can remove it
> using (assuming upstream remote is the OSGeo repo. You can remove
> upstream/master and update info about the default branch using:
>
> git fetch upstream --prune
> git remote set-head upstream -a
>
> Alternatively, if you are not keeping your fork's master/main branch up to
> date, you can follow instructions from my initial email which allow you to
> skip the fork part resulting in a slightly different local setup.
>
> ## 2. Updating your contributing steps
>
> To update your local main branch (base branch) and to rebase or merge with
> the latest source code, use upstream/main instead of upstream/master.
>
> ## 3. Updating workflows using source code from Git
>
> If you have a workflow which is using the source code obtained from Git ,
> you will need to update it if you explicitly specify the branch name. If
> you do --branch, checkout, or switch, and at the same time using the master
> branch, you will need to change "master" to "main". If you use only a
> simple `git clone`, no changes are needed because `git clone` takes the
> default branch which will continue to work.
>
> ## 4. Other issues
>
> The base branch for PRs was changed automatically. The PRs updating most
> of the source code for core and addons were merged. Some CI-related changes
> will still be needed for core. Additionally, I assume the workflows outside
> of the source code will be updated as part of the updates to addons repo
> structure and changes for the release branch for 8.0. Help in this area is
> welcome.
>
> https://github.com/OSGeo/grass/pull/1806
> https://github.com/OSGeo/grass-addons/pull/598
>
> Let me know if you face any questions.
>
> Best,
> Vaclav
>
>
> On Wed, Aug 18, 2021 at 10:41 PM Vaclav Petras 
> wrote:
>
>> Dear all,
>>
>> I will rename our default branch currently called master to main in the
>> following days. We already discussed it here and there, but the rename
>> should be relatively smooth. Much simpler than the addons repo
>> reorganization. Users not compiling code themselves should not be affected.
>> Contributors will be, but GitHub warns you about the rename when you come
>> to the repo web page. Those compiling from source may need to make changes.
>>
>> The update of a local clone can be done with instructions similar to the
>> following (I will post them again once I confirm it is the best fit):
>>
>> git branch -m master main
>> git fetch upstream
>> git branch -u upstream/main main
>> git remote set-head upstream -a
>>
>> Additionally, to get rid of upstream/master, do:
>>
>> git fetch --prune
>>
>> You don't need to update your fork unless you want to.
>>
>> The changes needed in the repo are already in a PR (feel free to review):
>>
>> https://github.com/OSGeo/grass/pull/1806
>>
>> Those compiling from source code obtained from Git will need to make
>> changes when they explicitly specify the branch. A simple `git clone` takes
>> the default branch which will continue to work. However, if you do
>> --branch, checkout, or switch, 

Re: [GRASS-dev] lib/gis/band_reference.c ?

[Markus Metz]

The easiest way to assign custom band names to raster maps in a strds is to
create a corresponding file for t.register.

Note that band names in a strds do not need to match band names assigned
with i.bands. The whole concept of band references in GRASS is IMHO still
experimental and unfortunately far away from the STAC eo bands extension.

Markus M


On Tue, Aug 24, 2021 at 11:09 AM Maris Nartiss  wrote:

> Band reference rules were totally relaxed 7 days a go:
>
> https://github.com/OSGeo/grass/commit/abe194dce78adf5f63885a6a09c452fc7ae4f735
>
> New relaxed rule changes haven't propagated to the documentation yet
> (PR's welcome).
>
> Band reference editing via r.support was added on 4th of July:
>
> https://github.com/OSGeo/grass/commit/ca1551206eaa0fc462f4849a06ebb035808470da
>
> Still – keep in mind – there are no changes in band metadata as
> managed by g.bands. Changes only deal with ability to have a band
> reference without extended metadata in json files. Thus limitation of
> not being able to define your own band reference is still in place,
> now only you can assign a band reference without having a definition
> (and thus any extended metadata apart from its name).
>
> Māris.
>
> 2021-08-24 11:18 GMT+03:00, Stefan Blumentrath  >:
> > Thanks Maris for the reply.
> >
> > I could not find that option in r.support.
> >
> > i.band allows to add e.g. S2_12 as band reference, but nothing custom.
> And
> > the g.bands manual says that user-defined band references are not yet
> > supported.
> >
> > Has that been added recently? Or would I have to edit the bands json
> file of
> > the system first?
> >
> > My system is:
> > g.version -ger
> > version=8.0.dev
> > date=2021
> > revision=dd9d36830
> > build_date=2021-07-08
> > build_platform=x86_64-pc-linux-gnu
> > build_off_t_size=8
> > libgis_revision=d2a5e8e8f
> > libgis_date=2021-06-16T04:05:21+00:00
> > proj=7.0.0
> > gdal=3.0.4
> > geos=3.8.0
> > sqlite=3.22.0
> >
> > Cheers
> > Stefan
> >
> > -Original Message-
> > From: Maris Nartiss 
> > Sent: tirsdag 24. august 2021 09:03
> > To: Stefan Blumentrath 
> > Cc: Martin Landa ; Markus Metz
> > ; GRASS developers list
> > 
> > Subject: Re: [GRASS-dev] lib/gis/band_reference.c ?
> >
> > GRASS supports arbitrary band reference names. Just make them unique
> enough
> > to not mix together apples with oranges by accident (e.g. "min"
> > is a bad idea, "min_t_c" is better; "NDVI" would work; "elevation" – bad,
> > "elevation_m" – better).
> > You can set band references after import with r.support module.
> >
> > Have fun,
> > Māris.
> >
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GRASS module version strictness (libgrass_gis)

[Markus Metz]

On Fri, Mar 5, 2021 at 8:30 PM Benjamin Ducke  wrote:
>
> Dear Devs,
>
> I am trying to find a way to inject GRASS modules (C code)
> compiled  in a GRASS 7 source tree into an already installed
> version of GRASS. Even if I compile in a GRASS 7.8.5 tree and
> then copy into a GRASS 7.8.5 (i.e. exact version match)
> installation, I get this:
>
> ERROR: Module built against version 2021-03-04T21:51:47+00:00 but trying
to
> use version 2021-01-01T10:33:43+00:00. You need to rebuild GRASS
GIS
> or untangle multiple installations.

GRASS uses the git revision to check if separately compiled modules are
compatible with the current version. If the git revision is not available,
the date of compilation is used instead. This is a safety measure to ensure
binary compatibility. The idea of using the date of compilation as
surrogate is targeted at branches under development and might not be
appropriate for released versions that do not change any more. Apparently
we need to distinguish between released versions and versions under
development and adjust the check for binary compatibility accordingly.

Markus M

>
> This is coming from 'libgrass_gis'.
>
> Apparently, GRASS keeps exact timestamps of the binaries and
> won't allow any mixing.
>
> Is it really necessary to be this strict about build versions?
> Would it be possible to relax this, so that a module compiled with
> GRASS 7.8.a can be run under GRASS 7.8.b?
>
> I worked around this issue with some LD_LIBRARY_PATH "magic",
> and even running 7.8.1 binaries in a 7.8.5 installation seems
> to work flawlessly.
>
> Background: I am trying to find a way to provide sets of
> additional modules, that are not part of the GRASS base
> distribution, for injection into an existing GRASS installation.
> This is exceedingly hard if all dependencies have to be
> exact matches; especially if the modules are to be injected
> into a version of GRASS bundled with QGIS.
>
> Best,
>
> Ben
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] lib/gis/band_reference.c ?

[Markus Metz]

The functions defined in lib/gis/band_reference.c [0] are used for raster
imagery and should thus be removed from lib/gis. There is also
lib/raster/band_reference.c [1] which should be moved to lib/imagery and
the functions need to be renamed.

Martin, can you explain the reason for the existence of
lib/gis/band_reference.c and lib/raster/band_reference.c and the
non-existence of lib/imagery/band_reference.c ?

Markus M

[0] https://github.com/OSGeo/grass/blob/master/lib/gis/band_reference.c
[1] https://github.com/OSGeo/grass/blob/master/lib/raster/band_reference.c
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GRASS GIS ready for PROJ 7?

[Markus Metz]

On Fri, Mar 5, 2021 at 8:11 PM Helmut Kudrnovsky  wrote:
>
> Hi devs,
>
> in OSGeo4W actually:
>
> C:\>proj --version
> Rel. 6.3.2, May 1st, 2020
>
> OSGeo4W next generation is in a testing phase at the moment.
>
> one of the biggest steps forward is:
>
> >PROJ 7.2 <=
>
> is GRASS GIS ready for PROJ 7?

Yes, insofar that GRASS 7.8+9 compile and run fine with PROJ 7, but...

>
> in PROJ7, there is a complete change how PROJ related/needed data is
> handled:
>
> (1) dynamically via https://cdn.proj.org/
> (2) via a revamped proj-data with file size of more than 500 MB
>
> * will GRASS be ready to handle it dynamically via CDN?
> * including more than 500MB data into our standalone installer may
explode our used NSIS system.

this new functionality of PROJ 7+ is not yet used in GRASS.

We need to introduce a mechanism where users can switch on and off dynamic
download of PROJ-related data. Depending on the available internet
connection (none, too slow, too expensive), automated download might need
to be disabled, in which case we need to introduce warnings that
PROJ-related data are missing for certain coordinate operations.

Markus M

>
> kind regards
> Helmut
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Thu, Feb 11, 2021 at 1:29 PM Nicklas Larsson  wrote:
>
> Moritz,
>
> I'd be honoured!
> I will put it on GRASS Wiki [1] if you don't have another suggestion and
notify here when done.
>
Thanks a lot Nicklas for raising this discussion!

I am sure GRASS will benefit if we allow new C standard features, e.g.
handling of integer types, math constants, nan, inf, etc, which will also
improve portability.

Markus M

> Best,
> Nicklas
>
>
> [1] https://trac.osgeo.org/grass/wiki/RFC
>
>
>
> On Thursday, 11 February 2021, 12:54:30 CET, Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
>
> On 10/02/21 13:16, Nicklas Larsson wrote:
> > It would be most favourable for all contributors and the project if the
> > community could come to an agreement on this topic. I see no reason to
> > postpone a decision on this much longer.
> >
> > The final word on this need to be that of the PSC's. Whether through
> > simple vote or a RFC. However, a sounding of the opinion of the
> > dev-community on this matter is of equal importance and can be of help
> > for the PSC.
>
> Thanks a lot, Nicklas, for this very comprehensive summary !
>
> A suggestion made at the first meeting of the new PSC was to use this
> discussion as a use case for a more extensive usage of RFC's to put
> important decisions into more permanent documents than mailing list
> archives and to provoke a formal decision as you suggest. Would you be
> willing to write a first draft of such an RFC ?
>
>
> Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

Just for clarification,

C code written for an older C standard works fine with newer C standards.

This is different with Python: code written for a previous Python version
might no longer work with a newer Python version.

Increasing the C standard for existing GRASS C code is safe, it will still
compile and work.

Increasing the Python version is not safe because existing GRASS Python
code might not be compatible with newer Python versions, thus the need for
a minimum required Python version.

Markus M

On Sun, Feb 7, 2021 at 10:36 PM Markus Metz 
wrote:
>
>
>
> On Sun, Feb 7, 2021 at 4:07 PM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
> >
> >
> >
> > Am 7. Februar 2021 05:01:38 MEZ schrieb "Anna Petrášová" <
kratocha...@gmail.com>:
> > >On Wed, Feb 3, 2021 at 2:47 PM Anna Petrášová 
wrote:
> > >
> > >>
> > >>
> > >> On Tue, Feb 2, 2021 at 11:20 AM Nicklas Larsson 
> > >> wrote:
> > >>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Friday, 29 January 2021, 18:50:34 CET, Anna Petrášová <
> > >>> kratocha...@gmail.com> wrote:
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> On Thu, Jan 28, 2021 at 4:28 AM Nicklas Larsson via grass-dev <
> > >>> grass-dev@lists.osgeo.org> wrote:
> > >>> > Dear Devs!
> > >>> >
> > >>> > As a relatively new member of the GRASS GIS dev community, I have
had
> > >>> to search for information on mailing lists, old trac comments etc.
> > >>> regarding coding practice and in particular minimum programming
language
> > >>> standard support. Ending up in not entirely conclusive
understanding. Up
> > >>> until now, I have been mostly involved in Python development and
I’m still
> > >>> not absolutely certain, although I assume 3.5 is minimum version.
And I’m
> > >>> not alone, see e.g. [1].
> > >>> >
> > >>> > Now, I’ve encountered a similar dilemma with C standard support,
> > >>> attempting to address compiler warnings [2], in particular with the
PR
> > >>> #1256 [3].
> > >>> >
> > >>> > I would be great if there were a (one) place where the min
support of
> > >>> Python version, C (C89, C99, C11, C17…) and C++ (C++03, C++11,
C++14 …)
> > >>> standard is stated -- loud and clear. Obviously, there has to be a
> > >>> consensus in the community on these matters for that to happen.
Such a
> > >>> statement will also have to be revised now and then. (A related
question is
> > >>> also whether or not to support 32 bit, which I know have been raised
> > >>> recently).
> > >>> >
> > >>> > I’d appreciate your opinion is on this issue!
> > >>> > Let me put up a a suggestion for min. req. for coming GRASS GIS 8
as a
> > >>> starting point of discussion:
> > >>> > - Python 3.7
> > >>> > - C11
> > >>> > - C++11
> > >>> >
> > >>> >
> > >>> > Best regards,
> > >>> > Nicklas
> > >>> >
> > >>>
> > >>> Regarding Python, not sure if we shouldn't set 3.6 as minimum for
G8, it
> > >>> is still used e.g. in Ubuntu 18. Any reason to set 3.7 as minimum,
some
> > >>> specific features we would want to use?
> > >>>
> > >>> Anna
> > >>>
> > >>> >
> > >>> >
> > >>> > [1] https://github.com/OSGeo/grass/issues/1241
> > >>> > [2] https://github.com/OSGeo/grass/issues/1247
> > >>> > [3] https://github.com/OSGeo/grass/pull/1256
> > >>> >
> > >>> > ___
> > >>> > grass-dev mailing list
> > >>> > grass-dev@lists.osgeo.org
> > >>> > https://lists.osgeo.org/mailman/listinfo/grass-dev
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>>
> > >>> Well, I don’t have a very strong opinion regarding 3.7, but
personally
> > >>> I’d say 3.6 is an absolute minimum. I presume, for example, most of
us
> > >>> would prefer to use f-strings for

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Sun, Feb 7, 2021 at 4:07 PM Moritz Lennert 
wrote:
>
>
>
> Am 7. Februar 2021 05:01:38 MEZ schrieb "Anna Petrášová" <
kratocha...@gmail.com>:
> >On Wed, Feb 3, 2021 at 2:47 PM Anna Petrášová 
wrote:
> >
> >>
> >>
> >> On Tue, Feb 2, 2021 at 11:20 AM Nicklas Larsson 
> >> wrote:
> >>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Friday, 29 January 2021, 18:50:34 CET, Anna Petrášová <
> >>> kratocha...@gmail.com> wrote:
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Jan 28, 2021 at 4:28 AM Nicklas Larsson via grass-dev <
> >>> grass-dev@lists.osgeo.org> wrote:
> >>> > Dear Devs!
> >>> >
> >>> > As a relatively new member of the GRASS GIS dev community, I have
had
> >>> to search for information on mailing lists, old trac comments etc.
> >>> regarding coding practice and in particular minimum programming
language
> >>> standard support. Ending up in not entirely conclusive understanding.
Up
> >>> until now, I have been mostly involved in Python development and I’m
still
> >>> not absolutely certain, although I assume 3.5 is minimum version. And
I’m
> >>> not alone, see e.g. [1].
> >>> >
> >>> > Now, I’ve encountered a similar dilemma with C standard support,
> >>> attempting to address compiler warnings [2], in particular with the PR
> >>> #1256 [3].
> >>> >
> >>> > I would be great if there were a (one) place where the min support
of
> >>> Python version, C (C89, C99, C11, C17…) and C++ (C++03, C++11, C++14
…)
> >>> standard is stated -- loud and clear. Obviously, there has to be a
> >>> consensus in the community on these matters for that to happen. Such a
> >>> statement will also have to be revised now and then. (A related
question is
> >>> also whether or not to support 32 bit, which I know have been raised
> >>> recently).
> >>> >
> >>> > I’d appreciate your opinion is on this issue!
> >>> > Let me put up a a suggestion for min. req. for coming GRASS GIS 8
as a
> >>> starting point of discussion:
> >>> > - Python 3.7
> >>> > - C11
> >>> > - C++11
> >>> >
> >>> >
> >>> > Best regards,
> >>> > Nicklas
> >>> >
> >>>
> >>> Regarding Python, not sure if we shouldn't set 3.6 as minimum for G8,
it
> >>> is still used e.g. in Ubuntu 18. Any reason to set 3.7 as minimum,
some
> >>> specific features we would want to use?
> >>>
> >>> Anna
> >>>
> >>> >
> >>> >
> >>> > [1] https://github.com/OSGeo/grass/issues/1241
> >>> > [2] https://github.com/OSGeo/grass/issues/1247
> >>> > [3] https://github.com/OSGeo/grass/pull/1256
> >>> >
> >>> > ___
> >>> > grass-dev mailing list
> >>> > grass-dev@lists.osgeo.org
> >>> > https://lists.osgeo.org/mailman/listinfo/grass-dev
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> Well, I don’t have a very strong opinion regarding 3.7, but personally
> >>> I’d say 3.6 is an absolute minimum. I presume, for example, most of us
> >>> would prefer to use f-strings for string formatting.
> >>>
> >>
> >> yes, f-strings are nice although they have limitations for using with
> >> translatable strings (Vashek can expand on that)
> >>
> >>>
> >>>
> >>> On the other hand, 3.6 will reach end-of-support at the end of this
year
> >>> right after its 5th birthday party and the support for data classes
in 3.7
> >>> may potentially offer intriguing applications in G8.
> >>>
> >>
> >> I noticed the data classes as well. Given 3.6 is reaching
end-of-support
> >> soon, I agree with 3.7 for G8. I assume grass would be compatible with
3.6
> >> for a while anyway.
> >>
> >>
> >>> Ubuntu 18 has Python 3.6 and Debian 9 has Python 3.5! What will make
the
> >>> lowest common denominator? Debian 10 and Ubuntu 20 actually supports
Python
> >>> 3.7 and 3.8 respectively. Forgive me if I’m ignorant, but isn’t it
possible
> >>> to upgrade Python version on Ubuntu? Or is it just a pain with package
> >>> dependencies? Relying on default Python has never/rarely been a
luxury for
> >>> other platforms.
> >>>
> >>> That being said, I think the most important part of this is that the
> >>> community make a clear decision on min. supported Python version.
> >>>
> >>
> >This GDAL's RFC [1] is helpful in summarizing the issue with Python.
> >Looking more into this, I suggest to go have a longer term strategy for
> >dropping support for Python versions, which would be relatively simple.
> >Basically we would keep the lowest Python version that wouldn't reach end
> >of life at the time of a major release of GRASS. E.g. when we release G8
> >this year, 3.6 will be minimum maintained version. Since 3.6 ends Dec
2021,
> >we could drop 3.6 support next year. I am not saying we need to be strict
> >about that, but might be helpful as a guidance, and it is independent on
> >distributions (which is probably both advantage and disadvantage). I am
> >unsure how this decision impacts packaging of grass, i.e. once we set 3.7
> >as minimum, would maintainers need to make that Python a dependency of
> >GRASS? Anyway, to summarize, I am for Python 3.6 at this point, but we
need
> >to reevaluate that 

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Fri, Feb 5, 2021 at 8:29 PM Markus Metz 
wrote:
>
>
> On Thu, Feb 4, 2021 at 5:35 AM Vaclav Petras  wrote:
>
> > Similarly to the GDAL and PROJ issue where I don't think it is
necessary to have GRASS C89/C++98 compliant when you need C++11 for GDAL
anyway.
>
> The C code of GRASS master is currently compatible with a lot of GDAL and
PROJ versions, also GDAL 1.xand PROJ 4.8.x which most likely do not require
C++11 or C99. If we want to keep this compatibility, we need to stick with
the lower C and C++ standards. Related to the question which currently
supported production environments we want to support.

My PR 1283 [0] is related to this discussion, it does not work with stock
Centos 7, and the github test with Centos 7 is thus failing. Of course it
is still possible to use GRASS master on Centos 7 as long as there is a
more recent PROJ version (as in my setup).

I suggest removing the github test for Centos 7 and using Centos 8 instead,
if possible. This does not mean that GRASS master is no longer running on
Centos 7, or similar old, but still supported OS's. This means that people
who want to use GRASS master with those OS's need to update certain GRASS
dependencies themselves. As long as those OS's support the minimum C and
C++ standards required by GRASS.

Markus M

[0] https://github.com/OSGeo/grass/pull/1283
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Use # %, not #% in Python scripts

[Markus Metz]

On Fri, Feb 5, 2021 at 5:14 AM Vaclav Petras  wrote:
>
> Dear all,
>
> I prepared a change to g.parser and related code which changes the option
definitions in scripts (aka script header) from:
>
> #%option
> #% key
> #%end
>
> to:
>
> # %option
> # % key
> # %end
>
> The reason is PEP8 compliance where "each line of a block comment starts
with a # and a single space" [2].
>
> The PR is removing all use of #% from the code, but it is still allowed,
so addons and existing user code should continue to work. However, I don't
see a need to depreciate support of #% at this point.
>
> This change moves us a little closer to PEP8 or Flake8 compliance. We can
now enable the check in CI once the normal comments are fixed. (The file
header comment is ignored, so no need to change that, but we can consider
that anyway.)
>
> The vision is that users and contributors can write scripts which just
work with more or less default Flake8 settings. This is a big step towards
that because it removes tens of warnings for a standard module.

These warnings for #% are a real annoyance, distracting from meaningful
warnings. The suggested changes will make code quality checking and code
improvement much easier.

+1

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Thu, Feb 4, 2021 at 5:35 AM Vaclav Petras  wrote:

> Similarly to the GDAL and PROJ issue where I don't think it is necessary
to have GRASS C89/C++98 compliant when you need C++11 for GDAL anyway.

The C code of GRASS master is currently compatible with a lot of GDAL and
PROJ versions, also GDAL 1.xand PROJ 4.8.x which most likely do not require
C++11 or C99. If we want to keep this compatibility, we need to stick with
the lower C and C++ standards. Related to the question which currently
supported production environments we want to support.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Wed, Feb 3, 2021 at 6:30 AM Huidae Cho  wrote:
>
> Thanks Nicklas for the great summary! That helped a lot.
>
> I was wondering if we have a list of all supported platforms somewhere?
We have official downloads for Linux, Mac, and Windows. Are these three
only "officially" supported platforms? I know GRASS is compilable on
FreeBSD [1] and maybe (Open|Net)BSD. Is that it? Minix3 supports GDAL
1.11.3, PROJ 4.9.2, GEOS 3.5.0, and GCC 6.2.0 [2]. I know... they are a
little behind.

I think it is a bit more complicated, e.g. there is not one single Linux
version. We need to make a decision not only about platforms, but also
about platform versions. Traditionally, GRASS aims to be compatible with
all currently supported platform versions. For RHEL, this would mean
support for RHEL 7 (ignoring the extended support for RHEL 6 until 2024).
For Debian, this would mean support for Stretch (Debian 9). For FreeBSD,
this would mean support for FreeBSD 11.x. For Solaris, this would mean
support for Solaris 11 (ignoring the extended support for Solaris 10 until
2024). I have previously tested GRASS on all these platforms, but not on
all supported versions of these platforms. IMHO, it is ok if the current
GRASS version is running on the current version of these OSs (RHEL, Debian,
FreeBSD, Solaris) which are often used in production environments.

The derived and often more up-to-date OSs like Fedora, Ubuntu, etc. do not
seem to be a problem.

Markus M

>
> Since we cannot (or will be difficult to) go back once we move to a newer
C standard, I think we need to discuss what platforms we want to support
officially or unofficially (?) first. Is [3] or [4] (GRASS 6.3) still
valid? Has anyone tried all or some of those platforms recently (Sun
Solaris (SPARC/Intel), Silicon Graphics Irix, HP-UX, DEC-Alpha, AIX, BSD,
iPAQ/Linux and other UNIX compliant platforms)? I think at some point this
list was removed from a release announcement. Maybe, we are being more
realistic because many of these platforms are now irrelevant or we just
don't have enough resources or interest to maintain such a list of
supported platforms anymore?
>
> Best,
> Huidae
>
> [1] https://www.freebsd.org/cgi/man.cgi?query=index=3
> [2] http://www.minix3.org/pkgsrc/distfiles/local/3.4.0-2016Q3/
> [3] https://old.grass.osgeo.org/screenshots/platforms/
> [4] https://grass.osgeo.org/news/2008_04_23_announce_grass630/
>
>
> On Mon, Feb 1, 2021 at 4:25 PM Markus Metz 
wrote:
>>
>> A pity that Nicklas did not answer in this thread, see his answer in
https://lists.osgeo.org/pipermail/grass-dev/2021-February/094913.html
>>
>> I have not studied the different C standards and the state of their
implementation in different compilers and their different versions in
depth, and thus appreciate very much the summary of Nicklas!
>>
>> IIUC, Nicklas recommends to allow C11 standard features in GRASS C code,
with no need to change the current code base, and all compilers in all
supported platforms apparently support C11.
>>
>> +1 from me, as long as all stock compilers on all supported platforms
support C11
>>
>> Markus M
>>
>>
>> On Mon, Feb 1, 2021 at 8:56 AM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>> >
>> >
>> >
>> > Am 31. Januar 2021 22:15:53 MEZ schrieb Markus Metz <
markus.metz.gisw...@gmail.com>:
>> > >On Fri, Jan 29, 2021 at 11:19 PM Moritz Lennert <
>> > >mlenn...@club.worldonline.be> wrote:
>> > >>
>> > >>
>> > >>
>> > >> Am 29. Januar 2021 20:54:06 GMT+00:00 schrieb Markus Metz <
>> > >markus.metz.gisw...@gmail.com>:
>> > >> >Hi Huidae,
>> > >> >
>> > >> >On Thu, Jan 28, 2021 at 6:30 PM Huidae Cho 
wrote:
>> > >> >>
>> > >> >> Markus,
>> > >> >>
>> > >> >> I think we have to think about what benefits it would bring to
us by
>> > >> >modernizing C code. Probably, not much at all. Personally, I would
keep
>> > >it
>> > >> >as is because the minimum set of anything (e.g., ANSI C with no new
>> > >> >features) would probably be more portable, I believe. In other
words,
>> > >what
>> > >> >are we missing from C99?
>> > >> >
>> > >> >as I mentioned, there is no need to modernize the GRASS C code. The
>> > >> >question is if we officially allow C99 features.
>> > >> >
>> > >> >For example a number of useful math-related functions and macros
are only
>> > >> >available with C99. See /usr/include/math.h on your system and

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

A pity that Nicklas did not answer in this thread, see his answer in
https://lists.osgeo.org/pipermail/grass-dev/2021-February/094913.html

I have not studied the different C standards and the state of their
implementation in different compilers and their different versions in
depth, and thus appreciate very much the summary of Nicklas!

IIUC, Nicklas recommends to allow C11 standard features in GRASS C code,
with no need to change the current code base, and all compilers in all
supported platforms apparently support C11.

+1 from me, as long as all stock compilers on all supported platforms
support C11

Markus M


On Mon, Feb 1, 2021 at 8:56 AM Moritz Lennert 
wrote:
>
>
>
> Am 31. Januar 2021 22:15:53 MEZ schrieb Markus Metz <
markus.metz.gisw...@gmail.com>:
> >On Fri, Jan 29, 2021 at 11:19 PM Moritz Lennert <
> >mlenn...@club.worldonline.be> wrote:
> >>
> >>
> >>
> >> Am 29. Januar 2021 20:54:06 GMT+00:00 schrieb Markus Metz <
> >markus.metz.gisw...@gmail.com>:
> >> >Hi Huidae,
> >> >
> >> >On Thu, Jan 28, 2021 at 6:30 PM Huidae Cho  wrote:
> >> >>
> >> >> Markus,
> >> >>
> >> >> I think we have to think about what benefits it would bring to us by
> >> >modernizing C code. Probably, not much at all. Personally, I would
keep
> >it
> >> >as is because the minimum set of anything (e.g., ANSI C with no new
> >> >features) would probably be more portable, I believe. In other words,
> >what
> >> >are we missing from C99?
> >> >
> >> >as I mentioned, there is no need to modernize the GRASS C code. The
> >> >question is if we officially allow C99 features.
> >> >
> >> >For example a number of useful math-related functions and macros are
only
> >> >available with C99. See /usr/include/math.h on your system and search
for
> >> >C99. Also a number of features related to data types, particularly for
> >> >various int datatypes (stdint.h), become available with C99. And the
> >> >geographic lib in PROJ with src/geodesic.c wants C99. For new PROJ
> >> >versions, C99 is a requirement.
> >>
> >>
> >> If proj requires it, doesn't it automatically become a requirement for
> >GRASS as well ?
> >
> >No, because the code base of other libs might have completely different
> >compile requirements. A software can use functions and libs of other
> >software packages, but does not need to follow the compile standards of
> >those other software packages, because they are compiled independently.
> >
>
> Thanks for the clarification.
>
> In light of that I agree that we should choose the oldest standard
possible, unless we _really_ need something only present in a more recent
version.
>
> @those who want to use more recent standards: what are your reasons for
that ?
>
> Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Sat, Jan 30, 2021 at 5:13 AM Vaclav Petras  wrote:
>
>
>
> On Fri, Jan 29, 2021 at 5:20 PM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>>
>>
>>
>> Am 29. Januar 2021 20:54:06 GMT+00:00 schrieb Markus Metz <
markus.metz.gisw...@gmail.com>:
>> >
>> >For new PROJ
>> >versions, C99 is a requirement.
>>
>>
>> If proj requires it, doesn't it automatically become a requirement for
GRASS as well ?
>
>
> That's a good point. We have the same situation with GDAL. It requires
C++11 [1]. Although this may not set the requirement absolutely, it sets it
practically.

For GDAL, not for GRASS. GRASS is compiled independently of GDAL, it just
makes use of GDAL fns that are compiled with their own set of requirements.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

On Fri, Jan 29, 2021 at 11:19 PM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
>
>
> Am 29. Januar 2021 20:54:06 GMT+00:00 schrieb Markus Metz <
markus.metz.gisw...@gmail.com>:
> >Hi Huidae,
> >
> >On Thu, Jan 28, 2021 at 6:30 PM Huidae Cho  wrote:
> >>
> >> Markus,
> >>
> >> I think we have to think about what benefits it would bring to us by
> >modernizing C code. Probably, not much at all. Personally, I would keep
it
> >as is because the minimum set of anything (e.g., ANSI C with no new
> >features) would probably be more portable, I believe. In other words,
what
> >are we missing from C99?
> >
> >as I mentioned, there is no need to modernize the GRASS C code. The
> >question is if we officially allow C99 features.
> >
> >For example a number of useful math-related functions and macros are only
> >available with C99. See /usr/include/math.h on your system and search for
> >C99. Also a number of features related to data types, particularly for
> >various int datatypes (stdint.h), become available with C99. And the
> >geographic lib in PROJ with src/geodesic.c wants C99. For new PROJ
> >versions, C99 is a requirement.
>
>
> If proj requires it, doesn't it automatically become a requirement for
GRASS as well ?

No, because the code base of other libs might have completely different
compile requirements. A software can use functions and libs of other
software packages, but does not need to follow the compile standards of
those other software packages, because they are compiled independently.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

Hi Huidae,

On Thu, Jan 28, 2021 at 6:30 PM Huidae Cho  wrote:
>
> Markus,
>
> I think we have to think about what benefits it would bring to us by
modernizing C code. Probably, not much at all. Personally, I would keep it
as is because the minimum set of anything (e.g., ANSI C with no new
features) would probably be more portable, I believe. In other words, what
are we missing from C99?

as I mentioned, there is no need to modernize the GRASS C code. The
question is if we officially allow C99 features.

For example a number of useful math-related functions and macros are only
available with C99. See /usr/include/math.h on your system and search for
C99. Also a number of features related to data types, particularly for
various int datatypes (stdint.h), become available with C99. And the
geographic lib in PROJ with src/geodesic.c wants C99. For new PROJ
versions, C99 is a requirement.

Markus M

>
> Regards,
> Huidae
>
> On Thu, Jan 28, 2021 at 12:19 PM Markus Metz <
markus.metz.gisw...@gmail.com> wrote:
>>
>> Hi all,
>>
>> regarding C, there is no need to modernize the code base because the
current C code written for C89 compiles just fine with newer standards
which are backwards compatible it seems. The question is if we allow
features from a newer C standard, say C99, to be included in the code base.
In fact, a few C99 features (supported by gnu89) have already sneaked into
the GRASS C code base. I am opting to allow C99 features.
>>
>> Regarding C++, it's a bit more difficult because apparently C++
standards are not fully backwards compatible. We had corresponding problems
with C++ code in GRASS previously and fixed these problems when they arose.
>>
>> Markus M
>>
>>
>> On Thu, Jan 28, 2021 at 3:33 PM Huidae Cho  wrote:
>>>
>>> Nicklas,
>>>
>>> Thanks for your suggestions. As far as I know, C code "is written in
portable ANSI-C and is fully POSIX compliant" [1] (C89 or C90?, rather old
standards, I know, but GRASS itself is old and predates both standards) and
the minimum "recommended" version of Python going forward is 3.5 [2]. I
don't know about C++, but there are not many modules written in it.
Modernizing the current code base to a newer C standard would be great
though.
>>>
>>> Best,
>>> Huidae
>>>
>>> [1] https://old.grass.osgeo.org/screenshots/platforms/
>>> [2] https://github.com/OSGeo/grass/blob/master/REQUIREMENTS.html
>>>
>>>
>>> On Thu, Jan 28, 2021 at 4:28 AM Nicklas Larsson via grass-dev <
grass-dev@lists.osgeo.org> wrote:
>>>>
>>>> Dear Devs!
>>>>
>>>> As a relatively new member of the GRASS GIS dev community, I have had
to search for information on mailing lists, old trac comments etc.
regarding coding practice and in particular minimum programming language
standard support. Ending up in not entirely conclusive understanding. Up
until now, I have been mostly involved in Python development and I’m still
not absolutely certain, although I assume 3.5 is minimum version. And I’m
not alone, see e.g. [1].
>>>>
>>>> Now, I’ve encountered a similar dilemma with C standard support,
attempting to address compiler warnings [2], in particular with the PR
#1256 [3].
>>>>
>>>> I would be great if there were a (one) place where the min support of
Python version, C (C89, C99, C11, C17…) and C++ (C++03, C++11, C++14 …)
standard is stated -- loud and clear. Obviously, there has to be a
consensus in the community on these matters for that to happen. Such a
statement will also have to be revised now and then. (A related question is
also whether or not to support 32 bit, which I know have been raised
recently).
>>>>
>>>> I’d appreciate your opinion is on this issue!
>>>> Let me put up a a suggestion for min. req. for coming GRASS GIS 8 as a
starting point of discussion:
>>>> - Python 3.7
>>>> - C11
>>>> - C++11
>>>>
>>>>
>>>> Best regards,
>>>> Nicklas
>>>>
>>>>
>>>>
>>>> [1] https://github.com/OSGeo/grass/issues/1241
>>>> [2] https://github.com/OSGeo/grass/issues/1247
>>>> [3] https://github.com/OSGeo/grass/pull/1256
>>>> ___
>>>> grass-dev mailing list
>>>> grass-dev@lists.osgeo.org
>>>> https://lists.osgeo.org/mailman/listinfo/grass-dev
>>>
>>>
>>>
>>> --
>>> Huidae Cho, Ph.D., GISP, /hidɛ t͡ɕo/, 조희대, 曺喜大
>>> GRASS GIS Developer
>>> https://idea.isnew.info
>>> ___
>>> grass-dev mailing list
>>> grass-dev@lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/grass-dev
>
>
>
> --
> Huidae Cho, Ph.D., GISP, /hidɛ t͡ɕo/, 조희대, 曺喜大
> GRASS GIS Developer
> https://idea.isnew.info
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Min. req. of programming language standard support, GRASS GIS 8

[Markus Metz]

Hi all,

regarding C, there is no need to modernize the code base because the
current C code written for C89 compiles just fine with newer standards
which are backwards compatible it seems. The question is if we allow
features from a newer C standard, say C99, to be included in the code base.
In fact, a few C99 features (supported by gnu89) have already sneaked into
the GRASS C code base. I am opting to allow C99 features.

Regarding C++, it's a bit more difficult because apparently C++ standards
are not fully backwards compatible. We had corresponding problems with C++
code in GRASS previously and fixed these problems when they arose.

Markus M


On Thu, Jan 28, 2021 at 3:33 PM Huidae Cho  wrote:

> Nicklas,
>
> Thanks for your suggestions. As far as I know, C code "is written in
> portable ANSI-C and is fully POSIX compliant" [1] (C89 or C90?, rather old
> standards, I know, but GRASS itself is old and predates both standards) and
> the minimum "recommended" version of Python going forward is 3.5 [2]. I
> don't know about C++, but there are not many modules written in it.
> Modernizing the current code base to a newer C standard would be great
> though.
>
> Best,
> Huidae
>
> [1] https://old.grass.osgeo.org/screenshots/platforms/
> [2] https://github.com/OSGeo/grass/blob/master/REQUIREMENTS.html
>
>
> On Thu, Jan 28, 2021 at 4:28 AM Nicklas Larsson via grass-dev <
> grass-dev@lists.osgeo.org> wrote:
>
>> Dear Devs!
>>
>> As a relatively new member of the GRASS GIS dev community, I have had to
>> search for information on mailing lists, old trac comments etc. regarding
>> coding practice and in particular minimum programming language standard
>> support. Ending up in not entirely conclusive understanding. Up until now,
>> I have been mostly involved in Python development and I’m still not
>> absolutely certain, although I assume 3.5 is minimum version. And I’m not
>> alone, see e.g. [1].
>>
>> Now, I’ve encountered a similar dilemma with C standard support,
>> attempting to address compiler warnings [2], in particular with the PR
>> #1256 [3].
>>
>> I would be great if there were a (one) place where the min support of
>> Python version, C (C89, C99, C11, C17…) and C++ (C++03, C++11, C++14 …)
>> standard is stated -- loud and clear. Obviously, there has to be a
>> consensus in the community on these matters for that to happen. Such a
>> statement will also have to be revised now and then. (A related question is
>> also whether or not to support 32 bit, which I know have been raised
>> recently).
>>
>> I’d appreciate your opinion is on this issue!
>> Let me put up a a suggestion for min. req. for coming GRASS GIS 8 as a
>> starting point of discussion:
>> - Python 3.7
>> - C11
>> - C++11
>>
>>
>> Best regards,
>> Nicklas
>>
>>
>>
>> [1] https://github.com/OSGeo/grass/issues/1241
>> [2] https://github.com/OSGeo/grass/issues/1247
>> [3] https://github.com/OSGeo/grass/pull/1256
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/grass-dev
>>
>
>
> --
> Huidae Cho, Ph.D., GISP, /hidɛ t͡ɕo/, 조희대, 曺喜大
> GRASS GIS Developer
> https://idea.isnew.info
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] PROJ6+ / WKT2 support

[Markus Metz]

Dear all,

I have prepared a rather large PR [0] to include proper support for PROJ6+
and WKT2.

The reason is that PROJ6+ uses a new mechanism based on spatial reference
id (authority name and code) and WKT2 for coordinate transformations. The
good old GRASS projection definition, a derivate of PROJ4-style definition,
is not able to capture details of all currently available coordinate
reference systems. With the recent substantial enhancements in PROJ, it is
time for GRASS to use these new enhancements. Current support for PROJ6+ in
G78 is not complete and still contains some bugs.

This PR (together with previous commits) would fix bugs also present in G78:
CRS mismatch [1] between input and output when creating a location from
some GDAL/OGR recognized input, importing to this location and exporting
again. The CRS should be exactly preserved. This bug happens when GRASS is
compiled with PROJ6+ and GDAL3.

A bug in PROJ can cause PROJ to select a wrong coordinate operation with a
datum transformation grid that does not cover the whole area of interest.
In these cases, reprojection fails. This PR takes care of this bug.

Fixing these bugs in G78 requires that all changes related to the new SRID
and WKT2 handling including all affected import and export modules need to
be backported.
Therefore I want to backport all relevant changes from master to G78 in
order to fix these bugs. These changes do not modify existing function
definitions, but add new functions to the GRASS headers.

Please let me know if you have objections regarding backporting to G78!

Markus M

[0] https://github.com/OSGeo/grass/pull/1240
[1] https://github.com/qgis/QGIS/issues/18596#issuecomment-753481210
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-user] compare a DCELL and FCELL question

[Markus Metz]

A note about the usage of GRASS_EPSILON = 1.0e-15:

The range of single precision floating point is about 1.4012984643e−45 to
3.4028234664e38
That means for large numbers, using GRASS_EPSILON will also detect floating
point precision limits, not only meaningful differences. For small numbers,
GRASS_EPSILON will not detect meaningful differences. GRASS_EPSILON could
instead be modified with something like "max(abs(map_A), abs(map_B)) *
1.0e-15" to test for meaningful differences.

Markus M


On Sun, Jan 24, 2021 at 5:32 PM Markus Metz 
wrote:
>
>
>
> On Sun, Jan 24, 2021 at 5:06 PM ming han  wrote:
> >
> > Hi Everyone
> >
> >Many thanks for your help. Is if(fabs(map_A - map_B) <= 1.0e-15, ...
) approach may increase the runtime compare to '==' way?
>
> The formulas are very simple, I don't think that differences in runtime
can be reliably measured. (De-)compression of the data and the operating
system's file cache have a much stronger influence on the runtime.
>
> Markus M
> >
> > Thanks
> > Ming
> >
> > Markus Metz  于2021年1月24日周日 上午10:57写道：
> >>
> >> Trying to answer the original question: with a DCELL map
"cat1_acc_riv" and a FCELL map "cat1_minacc", why is "float(cat1_acc_riv)
== float(cat1_minacc)" not equal to "int(cat1_acc_riv) == int(cat1_minacc)"
?
> >>
> >> int truncates to integer while float converts to single precision
floating point. E.g. with cat1_acc_riv = 1.1 and cat1_minacc = 1.9,
"float(cat1_acc_riv) == float(cat1_minacc)" becomes "1.1 == 1.9" whereas
"int(cat1_acc_riv) == int(cat1_minacc)" becomes "1 == 1", thus the results
are different.
> >>
> >> Another reason for possible differences is that float can only
represent max 7 decimal digits. E.g. float(194320567) becomes 194320560 but
int(194320567) preserves the value 194320567.
> >>
> >> Thus the safest is to cast everything to the type with the highest
precision. In this case with FCELL and DCELL, use "double(cat1_acc_riv) ==
double(cat1_minacc)" or even better the suggestion of Markus N.
> >>
> >> Markus M
> >>
> >>
> >> On Sun, Jan 24, 2021 at 3:51 PM ming han  wrote:
> >> >
> >> > Hi Markus and Micha
> >> >
> >> >  I am just trying to find grids have the same values in these
two rasters, I will try the threshold approach.
> >> >
> >> > Thanks
> >> > Ming
> >> >
> >> > Markus Neteler  于2021年1月24日周日 上午6:58写道：
> >> >>
> >> >> Hi Ming,
> >> >>
> >> >> On Sun, Jan 24, 2021 at 10:49 AM ming han 
wrote:
> >> >> >
> >> >> > Hi Micha
> >> >> >
> >> >> >  Many thanks for your reply.
> >> >> >  Here is the command I am using:
> >> >> >
> >> >> >  if(float(cat1_acc_riv) == float(cat1_minacc), str_r, null())
> >> >> >
> >> >> >   The str_r is a CELL raster. the result is different when I
change it to:
> >> >> >if(int(cat1_acc_riv) == int(cat1_minacc), str_r, null())
> >> >>
> >> >> Note that numerical "equality" is better tested with a threshold
test
> >> >> against the map pixel difference.
> >> >> As the threshold, we use GRASS_EPSILON which is defined as 1.0e-15.
> >> >>
> >> >> Hence the test needs to be implemented in a different way, i.e. by
> >> >> using an epsilon.
> >> >> Essentially something like this:
> >> >>
> >> >> if(fabs(map_A - map_B) <= 1.0e-15, ... )
> >> >>
> >> >> In your case (untested):
> >> >> r.mapcalc diffepsilon = if( abs( map_A - map_B) <= 1.0e-15, str_r ,
null())
> >> >>
> >> >> See related discussions here: [1], [2] and elsewhere.
> >> >>
> >> >> [1] Comment by Glynn:
https://trac.osgeo.org/grass/ticket/2854#comment:9
> >> >> [2] Comment by Glynn:
> >> >>
https://lists.osgeo.org/pipermail/grass-user/2015-October/073200.html
> >> >>
> >> >> Best,
> >> >> Markus
> >> >
> >> > ___
> >> > grass-dev mailing list
> >> > grass-dev@lists.osgeo.org
> >> > https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-user] compare a DCELL and FCELL question

[Markus Metz]

On Sun, Jan 24, 2021 at 5:06 PM ming han  wrote:
>
> Hi Everyone
>
>Many thanks for your help. Is if(fabs(map_A - map_B) <= 1.0e-15, ... )
approach may increase the runtime compare to '==' way?

The formulas are very simple, I don't think that differences in runtime can
be reliably measured. (De-)compression of the data and the operating
system's file cache have a much stronger influence on the runtime.

Markus M
>
> Thanks
> Ming
>
> Markus Metz  于2021年1月24日周日 上午10:57写道：
>>
>> Trying to answer the original question: with a DCELL map "cat1_acc_riv"
and a FCELL map "cat1_minacc", why is "float(cat1_acc_riv) ==
float(cat1_minacc)" not equal to "int(cat1_acc_riv) == int(cat1_minacc)" ?
>>
>> int truncates to integer while float converts to single precision
floating point. E.g. with cat1_acc_riv = 1.1 and cat1_minacc = 1.9,
"float(cat1_acc_riv) == float(cat1_minacc)" becomes "1.1 == 1.9" whereas
"int(cat1_acc_riv) == int(cat1_minacc)" becomes "1 == 1", thus the results
are different.
>>
>> Another reason for possible differences is that float can only represent
max 7 decimal digits. E.g. float(194320567) becomes 194320560 but
int(194320567) preserves the value 194320567.
>>
>> Thus the safest is to cast everything to the type with the highest
precision. In this case with FCELL and DCELL, use "double(cat1_acc_riv) ==
double(cat1_minacc)" or even better the suggestion of Markus N.
>>
>> Markus M
>>
>>
>> On Sun, Jan 24, 2021 at 3:51 PM ming han  wrote:
>> >
>> > Hi Markus and Micha
>> >
>> >  I am just trying to find grids have the same values in these two
rasters, I will try the threshold approach.
>> >
>> > Thanks
>> > Ming
>> >
>> > Markus Neteler  于2021年1月24日周日 上午6:58写道：
>> >>
>> >> Hi Ming,
>> >>
>> >> On Sun, Jan 24, 2021 at 10:49 AM ming han  wrote:
>> >> >
>> >> > Hi Micha
>> >> >
>> >> >  Many thanks for your reply.
>> >> >  Here is the command I am using:
>> >> >
>> >> >  if(float(cat1_acc_riv) == float(cat1_minacc), str_r, null())
>> >> >
>> >> >   The str_r is a CELL raster. the result is different when I
change it to:
>> >> >if(int(cat1_acc_riv) == int(cat1_minacc), str_r, null())
>> >>
>> >> Note that numerical "equality" is better tested with a threshold test
>> >> against the map pixel difference.
>> >> As the threshold, we use GRASS_EPSILON which is defined as 1.0e-15.
>> >>
>> >> Hence the test needs to be implemented in a different way, i.e. by
>> >> using an epsilon.
>> >> Essentially something like this:
>> >>
>> >> if(fabs(map_A - map_B) <= 1.0e-15, ... )
>> >>
>> >> In your case (untested):
>> >> r.mapcalc diffepsilon = if( abs( map_A - map_B) <= 1.0e-15, str_r ,
null())
>> >>
>> >> See related discussions here: [1], [2] and elsewhere.
>> >>
>> >> [1] Comment by Glynn:
https://trac.osgeo.org/grass/ticket/2854#comment:9
>> >> [2] Comment by Glynn:
>> >> https://lists.osgeo.org/pipermail/grass-user/2015-October/073200.html
>> >>
>> >> Best,
>> >> Markus
>> >
>> > ___
>> > grass-dev mailing list
>> > grass-dev@lists.osgeo.org
>> > https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-user] compare a DCELL and FCELL question

[Markus Metz]

Trying to answer the original question: with a DCELL map "cat1_acc_riv" and
a FCELL map "cat1_minacc", why is "float(cat1_acc_riv) ==
float(cat1_minacc)" not equal to "int(cat1_acc_riv) == int(cat1_minacc)" ?

int truncates to integer while float converts to single precision floating
point. E.g. with cat1_acc_riv = 1.1 and cat1_minacc = 1.9,
"float(cat1_acc_riv) == float(cat1_minacc)" becomes "1.1 == 1.9" whereas
"int(cat1_acc_riv) == int(cat1_minacc)" becomes "1 == 1", thus the results
are different.

Another reason for possible differences is that float can only represent
max 7 decimal digits. E.g. float(194320567) becomes 194320560 but
int(194320567) preserves the value 194320567.

Thus the safest is to cast everything to the type with the highest
precision. In this case with FCELL and DCELL, use "double(cat1_acc_riv) ==
double(cat1_minacc)" or even better the suggestion of Markus N.

Markus M


On Sun, Jan 24, 2021 at 3:51 PM ming han  wrote:
>
> Hi Markus and Micha
>
>  I am just trying to find grids have the same values in these two
rasters, I will try the threshold approach.
>
> Thanks
> Ming
>
> Markus Neteler  于2021年1月24日周日 上午6:58写道：
>>
>> Hi Ming,
>>
>> On Sun, Jan 24, 2021 at 10:49 AM ming han  wrote:
>> >
>> > Hi Micha
>> >
>> >  Many thanks for your reply.
>> >  Here is the command I am using:
>> >
>> >  if(float(cat1_acc_riv) == float(cat1_minacc), str_r, null())
>> >
>> >   The str_r is a CELL raster. the result is different when I
change it to:
>> >if(int(cat1_acc_riv) == int(cat1_minacc), str_r, null())
>>
>> Note that numerical "equality" is better tested with a threshold test
>> against the map pixel difference.
>> As the threshold, we use GRASS_EPSILON which is defined as 1.0e-15.
>>
>> Hence the test needs to be implemented in a different way, i.e. by
>> using an epsilon.
>> Essentially something like this:
>>
>> if(fabs(map_A - map_B) <= 1.0e-15, ... )
>>
>> In your case (untested):
>> r.mapcalc diffepsilon = if( abs( map_A - map_B) <= 1.0e-15, str_r ,
null())
>>
>> See related discussions here: [1], [2] and elsewhere.
>>
>> [1] Comment by Glynn: https://trac.osgeo.org/grass/ticket/2854#comment:9
>> [2] Comment by Glynn:
>> https://lists.osgeo.org/pipermail/grass-user/2015-October/073200.html
>>
>> Best,
>> Markus
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.stats.quantile: does the 50th percentile equal to the median?

[Markus Metz]

On Fri, Jan 22, 2021 at 2:51 PM  wrote:
>
> Hi devs,
>
> does the default `percentiles=50` in `r.stats.quantile` equal to the
> median?
> Is the definition exclusive or inclusive?

The median is commonly defined as the 50% percentile, the "middle" item of
a sorted dataset. See
https://en.wikipedia.org/wiki/Median

In GRASS, calculations of the median and the 50% percentile are identical.

HTH,

Markus M


>
>
> Thank you for any insight, Nikos
> --
>
> See also
> - https://en.wikipedia.org/wiki/Percentile
> - https://math.stackexchange.com/q/2048470/445399
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] db.copy gives error

[Markus Metz]

On Sat, Mar 14, 2020 at 3:27 PM Paulo van Breugel 
wrote:
>
> Dear devs,
>
> I have a grassgis database with in the mapset 'Permanent' a table
> 'vegetation_lookup'. I want top copy this table to the current mapset.
> The following used to work I think, but now gives me some errors:
>
> db.copy from_driver=sqlite
> from_database=/media/grassdb/biodiv/PERMANENT/sqlite/sqlite.db
> from_table=vegetation_lookup to_driver=sqlite
> to_database=$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite/sqlite.db
> to_table=vegetation_lookup

Looking at the code of db.copy, it is not possible that this ever worked,
you need to replace the variables with their actual values.

Markus M
>
> ERROR: Unable to create directory '///sqlite' for sqlite database
> dbmi: Protocol error
> WARNING: Unable to open database  by driver 
>
> Any idea what I am doing wrong here?
>
> With kind regards,
>
> Paulo
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Automatic/CI compilation on macOS

[Markus Metz]

On Fri, Mar 6, 2020 at 8:50 AM Rainer M Krug  wrote:
>
> OK. Looking at the parameters GRASS is compiled with (grass78 —config), I
get the following proj related parameter (the complete output at the end of
the email):
>
> --with-proj-includes=/usr/local/opt/osgeo-proj/include
> --with-proj-libs=/usr/local/opt/osgeo-proj/lib
> --with-proj-share=/usr/local/opt/osgeo-proj/share/proj
>
> Which seem to be correct, and proj.db is in the proj-share directory.
>
>
> 08:37 $ ls -la /usr/local/opt/osgeo-proj/share/proj
> total 12408
> drwxr-xr-x  15 rainerkrug  staff  480 Feb 10 11:16 .
> drwxr-xr-x   4 rainerkrug  staff  128 Feb 10 11:16 ..
> -rw-r--r--   1 rainerkrug  staff 1183 Feb 10 11:16 CH
> -rw-r--r--   1 rainerkrug  staff  728 Feb 10 11:16 GL27
> -rw-r--r--   1 rainerkrug  staff 2099 Feb 10 11:16 ITRF2000
> -rw-r--r--   1 rainerkrug  staff 3660 Feb 10 11:16 ITRF2008
> -rw-r--r--   1 rainerkrug  staff 3468 Feb 10 11:16 ITRF2014
> -rw-r--r--   1 rainerkrug  staff 6385 Feb 10 11:16 nad.lst
> -rw-r--r--   1 rainerkrug  staff19535 Feb 10 11:16 nad27
> -rw-r--r--   1 rainerkrug  staff16593 Feb 10 11:16 nad83
> -rw-r--r--   1 rainerkrug  staff  232 Feb 10 11:16 null
> -rw-r--r--   1 rainerkrug  staff 3915 Feb 10 11:16 other.extra
> -rw-r--r--   1 rainerkrug  staff  6234112 Feb 10 11:16 proj.db
> -rw-r--r--   1 rainerkrug  staff32060 Feb 10 11:16
projjson.schema.json
> -rw-r--r--   1 rainerkrug  staff 7079 Feb 10 11:16 world
>
>
> So it looks fine, but I, even locally, get the following error when
running the simple test:
>
> 08:40 $ grass78 --tmp-location EPSG:4326 --exec g.region res=0.1 -p
> Starting GRASS GIS...
> Creating new GRASS GIS location ...
> ERROR: b'proj_get_authorities_from_database: Cannot find proj.db

This error comes directly from PROJ.
Try
export PROJ_LIB="usr/local/opt/osgeo-proj/share/proj"

before starting GRASS. This will tell PROJ where its own share data are.

Markus M

>
>
>
> Here is the complete output from --config:
>
> 08:35 $ grass78 --config
> x86_64-apple-darwin17.7.0
> ./configure  --prefix=/usr/local/Cellar/osgeo-grass/7.8.2_3 --with-cxx
--enable-shared --enable-largefile --with-nls
--with-includes=/usr/local/include --with-libs=/usr/local/LIB
--with-python=/usr/local/Cellar/osgeo-grass/7.8.2_3/libexec/vendor/bin/python-config
--with-tcltk --with-netcdf=/usr/local/opt/osgeo-netcdf/bin/nc-config
--with-zstd --with-zstd-includes=/usr/local/opt/zstd/include
--with-zstd-libs=/usr/local/opt/zstd/lib --with-readline
--with-readline-includes=/usr/local/opt/readline/include
--with-readline-libs=/usr/local/opt/readline/lib --with-blas
--with-blas-includes=/usr/local/opt/openblas/include
--with-blas-libs=/usr/local/opt/openblas/lib --with-lapack
--with-lapack-includes=/usr/local/opt/lapack/include
--with-lapack-libs=/usr/local/opt/lapack/lib
--with-geos=/usr/local/opt/geos/bin/geos-config
--with-geos-includes=/usr/local/opt/geos/include
--with-geos-libs=/usr/local/opt/geos/lib --with-odbc
--with-odbc-includes=/usr/local/opt/unixodbc/include
--with-odbc-libs=/usr/local/opt/unixodbc/lib
--with-gdal=/usr/local/opt/osgeo-gdal/bin/gdal-config
--with-zlib-includes=/usr/local/opt/zlib/include
--with-zlib-libs=/usr/local/opt/zlib/lib --with-bzlib
--with-bzlib-includes=/usr/local/opt/bzip2/include
--with-bzlib-libs=/usr/local/opt/bzip2/lib --with-cairo
--with-cairo-includes=/usr/local/opt/cairo/include/cairo
--with-cairo-libs=/usr/local/opt/cairo/lib
--with-cairo-ldflags=-lfontconfig --with-freetype
--with-freetype-includes=/usr/local/opt/freetype/include/freetype2
--with-freetype-libs=/usr/local/opt/freetype/lib
--with-proj-includes=/usr/local/opt/osgeo-proj/include
--with-proj-libs=/usr/local/opt/osgeo-proj/lib
--with-proj-share=/usr/local/opt/osgeo-proj/share/proj --with-tiff
--with-tiff-includes=/usr/local/opt/libtiff/include
--with-tiff-libs=/usr/local/opt/libtiff/lib --with-png
--with-png-includes=/usr/local/opt/libpng/include
--with-png-libs=/usr/local/opt/libpng/lib --with-regex --with-fftw
--with-fftw-includes=/usr/local/opt/fftw/include
--with-fftw-libs=/usr/local/opt/fftw/lib --with-sqlite
--with-sqlite-includes=/usr/local/opt/sqlite/include
--with-sqlite-libs=/usr/local/opt/sqlite/lib
--with-liblas=/usr/local/opt/osgeo-liblas/bin/liblas-config --with-postgres
--with-postgres-includes=/usr/local/opt/osgeo-postgresql/include
--with-postgres-libs=/usr/local/opt/osgeo-postgresql/lib --with-mysql
--with-mysql-includes=/usr/local/opt/mysql/include/mysql
--with-mysql-libs=/usr/local/opt/mysql/lib --with-pthread
--with-pthread-includes=/usr/local/opt/boost/include/boost/thread
--with-pthread-libs=/usr/local/opt/boost/lib
--with-macosx-sdk=/Applications/Xcode-10.1.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
--with-macosx-archs=x86_64

Re: [GRASS-dev] Adding path iconv library to ./configure

[Markus Metz]

On Wed, Feb 26, 2020 at 5:35 PM Vaclav Petras  wrote:
>
> Dear all,
>
> In order to compile in a conda environment (where iconv is installed
through conda, not in the system), I'm trying to add a path to icon into
./configure(.in). The issue is that although GRASS happens to configure
properly, the compilation fails because libiconv.* is not found.

it seems that GRASS configure uses iconv from libc if present, but at
compile time this is not always true: sometimes iconv from libiconv is
attempted to be used and not from libc.

See https://lists.osgeo.org/pipermail/grass-dev/2014-December/072690.html

Most importantly, regarding only iconv:

"In the configure script, the header checks and the library checks are
disconnected."

This needs to be fixed.If the configure checks pass, the settings used by
configure (included headers, linkied libraries) need to be used as compile
time.

Markus M

>
> I have added code which is used for other libraries, although I'm not
sure what exactly should I do for a mandatory dependency. The part which
seems to be failing is AC_CHECK_FUNC.
>
> ac_save_ldflags="$LDFLAGS"
> LDFLAGS="$LDFLAGS $ICONVLIBPATH"
> # LDFLAGS now contains the path set using --with-iconv-libs
> AC_CHECK_FUNC(iconv, ICONVLIB=, [
> AC_CHECK_LIB(iconv, iconv, ICONVLIB=-liconv, [
> AC_CHECK_LIB(giconv, iconv, ICONVLIB=-lgiconv, [
> AC_CHECK_FUNC(libiconv, ICONVLIB=, [
> AC_CHECK_LIB(iconv, libiconv, ICONVLIB=-liconv, [
> AC_CHECK_LIB(giconv, libiconv, ICONVLIB=-lgiconv, [
> AC_MSG_WARN([*** Unable to locate iconv() function.])
> ICONVLIB=
> ])])])])])])
> AC_SUBST(ICONVLIB)
> LDFLAGS="${ac_save_ldflags}"
> # The AC_CHECK_FUNC step above seems to succeed
> # The output is "checking for iconv... yes"
> # The step below fails because $ICONVLIB is empty
> # With "checking for iconv in -l... no"
> LOC_CHECK_LIBS($ICONVLIB,iconv,iconv,$ICONVLIBPATH,ICONV,,,)
>
> I'm including a full diff too. Note that the iconv library is different
from the others because our ./configure allows for different names of the
library.
>
> Alternative would be to use LDFLAGS. That would be actually ideal,
because all libraries are just at one common prefix. However, providing it
as a parameter to ./configure (which seems to be the recommended way even
over the above) results in "unknown platform" and providing it as an
environmental variable does not have any effect.
>
> Please, let me know if you have any advice,
> Vaclav
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [release planning] GRASS GIS 7.8.3

[Markus Metz]

On Wed, Feb 12, 2020 at 6:08 PM Markus Neteler  wrote:
>
> Hi devs,
>
> We have accumulated a series of important fixes and would suggest that
> we consider a release in the very near future.
> There is a related 7.8.3 milestone in GH:
>
> https://github.com/OSGeo/grass/milestone/1
>
> along with the "classical" one in trac:
> https://trac.osgeo.org/grass/milestone/7.8.3
>
> Two blockers are there, please check:
>
https://trac.osgeo.org/grass/query?status=new=assigned=reopened=blocker=critical=7.8.3=type=priority

The blocker https://trac.osgeo.org/grass/ticket/3880 is addressed by PR
#325 (https://github.com/OSGeo/grass/pull/325).

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] view trac tickets disabled?

[Markus Metz]

Hi Markus,

On Sat, Feb 8, 2020 at 11:21 AM Markus Neteler  wrote:
>
> Hi again,
>
> The problem of the missing button seems to be solved (by actually
> re-running the permission assignment in the trac admin section after
> having edited trac.ini).
>
> Now there are two buttons at https://trac.osgeo.org/grass
> - "View Tickets" --> pointing to GitHub issues
> (https://github.com/OSGeo/grass/issues)
> - "Old tickets" --> pointing to the old ones in trac.

Perfect, works for me, thanks!

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] view trac tickets disabled?

[Markus Metz]

On Fri, Feb 7, 2020 at 10:06 PM Vaclav Petras  wrote:
>
> Hi Markus,
>
> On Fri, Feb 7, 2020 at 3:40 PM Markus Metz 
wrote:
>>
>>
>> now and then I want to view existing GRASS trac tickets, but going to
https://trac.osgeo.org/grass and clicking on "View Tickets" leads me to
https://github.com/OSGeo/grass/issues. Can this redirection be reverted,
please? I need to be able to search GRASS trac tickets.
>
>
> The links in in the Bug Tracking section work:
>
> https://trac.osgeo.org/grass#BugTracking
>
> The section provides context, so it should be clear that person is
opening a list of old issues. There is no "all tickets" link though - it is
just whatever it was before.

What is the reason why going to https://trac.osgeo.org/grass and clicking
on "View Tickets" is leading me to https://github.com/OSGeo/grass/issues?

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] view trac tickets disabled?

[Markus Metz]

Hi all,

now and then I want to view existing GRASS trac tickets, but going to
https://trac.osgeo.org/grass and clicking on "View Tickets" leads me to
https://github.com/OSGeo/grass/issues. Can this redirection be reverted,
please? I need to be able to search GRASS trac tickets.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.slope.aspect with -n flag giving aspect of flat areas as -9998

[Markus Metz]

On Fri, Jan 31, 2020 at 9:35 PM Markus Neteler  wrote:
>
> On Fri, Jan 31, 2020 at 10:39 AM Pedro Venâncio
>  wrote:
> >
> > Hi Anna and Markus,
> >
> > Thank you very much!
> >
> > A filled a ticket: https://github.com/OSGeo/grass/issues/319
>
> It was fixed in master f449ea5 and relbr78 fdd079c by Markus M.
>
> Please test and update/close the issue.

there is a new PR #320 by Anna (https://github.com/OSGeo/grass/pull/320)
fixing also the title and the color table. Please test!

Markus M

>
> Best
> markusN
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.slope.aspect with -n flag giving aspect of flat areas as -9998

[Markus Metz]

On Tue, Jan 28, 2020 at 12:50 PM Pedro Venâncio 
wrote:
>
> Hi all,
>
> I was testing r.slope.aspect with the "new" -n flag, also to include -e
and -n flags in QGIS Processing, but -n flag is giving aspect values of
flat areas as -9998, instead of -, when the precision choosed is CELL,
and the output data type is CELL.

that seems to be a rounding error, even though float or double should be
able to represent - exactly. Should be easy to fix.

Markus M
>
> https://grass.osgeo.org/grass78/manuals/r.slope.aspect.html
>
> I've tested with GRASS 7.6 (on Linux) and GRASS 7.8.2 (on Windows), both
in the GUI and CLI.
>
> (Tue Jan 28 11:19:40 2020)

> r.slope.aspect -e -n --verbose elevation=SRTM_PT_25m@PERMANENT
slope=SRTM_PT_25m_Slope_Deg_GRASS aspect=SRTM_PT_25m_Aspect_Std_GRASS
precision=CELL
> Percent complete...
> Elevation products for mapset  in 
> Min computed aspect 0., max computed aspect 360.
> Aspect raster map  complete
> Min computed slope 0., max computed slope 75.2970
> Slope raster map  complete
> (Tue Jan 28 11:20:24 2020) Comando terminado (43 sec)
>
> r.info map=SRTM_PT_25m_Aspect_Std_GRASS@PERMANENT

>
 ++
>  | Map:  SRTM_PT_25m_Aspect_Std_GRASS@  Date: Tue Jan 28 11:20:24
2020|
>  | Mapset:   PERMANENT  Login of Creator:
PedroVenancio   |
>  | Location: SRTM_PT_25m
 |
>  | DataBase: D:\ICNF\Modelos_Combustivel_2018\Landscape_File\grass78
 |
>  | Title:Aspect counterclockwise in degrees from east
|
>  | Timestamp: none
 |
>
 ||
>  |
 |
>  |   Type of Map:  raster   Number of Categories: 360
|
>  |   Data Type:CELL
|
>  |   Rows: 23200
 |
>  |   Columns:  11360
 |
>  |   Total Cells:  263552000
 |
>  |Projection: ETRS89 / Portugal TM06
 |
>  |N: 278000S:-302000   Res:25
|
>  |E: 164000W:-12   Res:25
|
>  |   Range of data:min = -9998  max = 360
|
>  |
 |
>  |   Data Source:
|
>  |raster elevation file SRTM_PT_25m@PERMANENT
|
>  |
 |
>  |
 |
>  |   Data Description:
 |
>  |generated by r.slope.aspect
|
>  |
 |
>  |   Comments:
 |
>  |aspect map elev = SRTM_PT_25m@PERMANENT
|
>  |zfactor = 1.00
 |
>  |min_slope = 0.00
 |
>  |
 |
>  |r.slope.aspect --verbose -e -n elevation="SRTM_PT_25m@PERMANENT"
slo\   |
>  |pe="SRTM_PT_25m_Slope_Deg_GRASS"
aspect="SRTM_PT_25m_Aspect_Std_GRAS\   |
>  |S" format="degrees" precision="CELL" zscale=1.0 min_slope=0.0
|
>  |
 |
>
 ++
> (Tue Jan 28 11:26:09 2020) Comando terminado (0 sec)

>
>
> Using the default precision as FCELL, all goes ok:
>
> (Tue Jan 28 11:33:08 2020)

> r.slope.aspect -e -n --verbose elevation=SRTM_PT_25m@PERMANENT
slope=SRTM_PT_25m_Slope_Deg_GRASS_2 aspect=SRTM_PT_25m_Aspect_Std_GRASS_2
> Percent complete...
> Elevation products for mapset  in 
> Min computed aspect 0., max computed aspect 359.8096
> Aspect raster map  complete
> Min computed slope 0., max computed slope 75.2970
> Slope raster map  complete
> (Tue Jan 28 11:34:01 2020) Comando terminado (53 sec)
>
> (Tue Jan 28 11:34:27 2020)

> r.info map=SRTM_PT_25m_Aspect_Std_GRASS_2@PERMANENT

>
 ++
>  | Map:  SRTM_PT_25m_Aspect_Std_GRASS_  Date: Tue Jan 28 11:34:01
2020|
>  | Mapset:   PERMANENT  Login of Creator:
PedroVenancio   |
>  | Location: SRTM_PT_25m
 |
>  | DataBase: D:\ICNF\Modelos_Combustivel_2018\Landscape_File\grass78
 |
>  | Title:Aspect counterclockwise in degrees from east
|
>  | Timestamp: none
 |
>
 ||
>  |
 |
>  |   Type of Map:  raster   Number of Categories: 360
|
>  |   Data Type:FCELL
 |
>  |   Rows: 23200
 |
>  |   Columns:  11360
 |
>  |   Total Cells:  263552000
 |
>  |Projection: ETRS89 / Portugal TM06
 |
>  |N: 278000S:-302000   Res:25
|
>  |E: 164000W:-12   Res:25
|
>  |   Range of data:min = -  max = 359.8097
 |
>  |
 |
>  |   Data Source:
|
>  |raster elevation file SRTM_PT_25m@PERMANENT
|
>  |
 |
>  |
 |
>  |   Data Description:
 |
>  |generated by r.slope.aspect
|
>  |
 |
>  |   Comments:
 |
>  |aspect map elev = SRTM_PT_25m@PERMANENT
|
>  |zfactor = 1.00
 |
>  |min_slope = 0.00
 |
>  |
 |
>  |r.slope.aspect --verbose -e -n elevation="SRTM_PT_25m@PERMANENT"
slo\   |
>  |pe="SRTM_PT_25m_Slope_Deg_GRASS_2"

Re: [GRASS-dev] WARNING: GRASS GIS libgis version and date number not available

[Markus Metz]

On Mon, Jan 20, 2020 at 9:26 PM Helmut Kudrnovsky  wrote:
>
> while starting
>
> --
> System Info

> GRASS version: 7.9.dev

> Code revision: 3a24c8c74

> Build date: 2020-01-18

> Build platform: x86_64-w64-mingw32

> GDAL: 3.0.2

> PROJ: 6.2.1

> GEOS: 3.8.0

> SQLite: 3.29.0

> Python: 3.7.0

> wxPython: 4.0.7

> Platform: Windows-10-10.0.18362-SP0 (OSGeo4W)
> --
>
> I get on the console:
>
> WARNING: GRASS GIS libgis version and date number not available

This warning comes from g.version

> maybe related to recent: 12839f: Update gisinit.c   etc.?

Yes. The reason is that GRASS GIS libgis version and date number are indeed
not available because they were taken from svn, but we are now on git.

Previously, libgis version (GIS_H_VERSION) and date ((GIS_H_DATE)
corresponded to the last change (in svn) to include/gis.h. With git, we
could get the last commit hash and date to include/ (last change to any of
the GRASS headers) with e.g. git log -1 -- include/

Proposals on how to add this to the git commit / GRASS configure / GRASS
make system are welcome.

>
> no addons are installed.

The reason for having this ibgis version and date of the last change is to
warn users if GRASS has been updated but addons have not been updated.
Without the check in gisinit.c, running addons might fail with obscure
errors, the reason being that they have been compiled against another
version of GRASS.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] any special reason why choice of raster compression method is done via environmental variable ?

[Markus Metz]

On Thu, Jan 16, 2020 at 10:56 AM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
> Coming back to this to continue the discussion:
>
> On 15/12/19 21:41, Markus Metz wrote:
> >
> >
> > On Tue, Dec 10, 2019 at 6:47 AM Moritz Lennert
> > mailto:mlenn...@club.worldonline.be>>
wrote:
> >  >
> >  > On 9/12/19 22:50, Markus Metz wrote:
> >  > >   Hi Moritz,
> >  > >
> >  > > On Wed, Dec 4, 2019 at 10:28 AM Moritz Lennert
> >  > > mailto:mlenn...@club.worldonline.be>
> > <mailto:mlenn...@club.worldonline.be
> > <mailto:mlenn...@club.worldonline.be>>> wrote:
> >  > >  >
> >  > >  > Hi Markus,
> >  > >  >
> >  > >  > In recent days, I've been confronted several times with the
issue of
> >  > >  > people trying to share data among themselves, but using
different
> >  > >  > versions of GRASS, and so raster data compressed in a more
recent
> >  > >  > version of GRASS was not usable in an older version of GRASS.
> >  > >  >
> >  > >  > Now, I agree that generally the solution is to tell people to
> > use the
> >  > >  > latest and greatest, but this is not always possible / it is not
> >  > >  > necessarily highest on the list of priorities of people to see
> > how they
> >  > >  > can install the latest version of GRASS within their particular
> >  > > environment.
> >  > >  >
> >  > >  > Obviously, those with the latest version of GRASS can simple
> > recompress
> >  > >  > using ZLIB. However, compression method is defined as an
environment
> >  > >  > variable. This is somewhat daunting to many MS Windows users out
> > there.
> >  > >  > Is there any specific reason that lead to the choice of not
using a
> >  > >  > parameter to allow the choice of compression method (possibly to
> >  > >  > override a default that is still defined by an environment
> > variable) ?
> >  > >
> >  > > Such a parameter would need to be added to every module creating
raster
> >  > > output.
> >  >
> >  > My request is more linked to use cases where one would like to share
> >  > data (e.g. with r.pack) with other GRASS GIS users who do not
> >  > necessarily have access to the same compression method, not
necessarily
> >  > to changing the default compression method. I was just wondering
whether
> >  > it might be easily possible to just implement r.compress method= as a
> >  > quick way to recompress a specific map with a chosen method,
overriding
> >  > the default method. Currently, to do that, you have to change the
> >  > default method by changing the env variable, run r.compress, then
change
> >  > the variable back to the value one wishes generally to use as
default.
> >
> > recompressing an existing raster map seems like a waste of time. Why not
> > compress it in the first place with the appropriate method?
>
> Because what you consider the appropriate method at the time creation of
> the data (and still the appropriate method for your actual work) might
> not be the appropriate method for sharing with someone using a different
> system.

True, for sharing the highest (smallest size) compression method is usually
preferred, while for actual work the fastest compression method is usually
preferred.
> >
> > About the use cases:
> > People want to share a mapset: in this case all GRASS raster maps should
> > be compressed with the most compatible compression method (ZLIB).
> > Currently, this can be done with an env var. Alternatively, a new
> > mechanism via g.gisenv could be established.
>
> This would already make life easier for Windows users who (for the
> majority) are not used to env variables (and this is becoming more and
> more true of GNU/Linux users as well...).
>
> > Another option could be to set the default compression method per
> > mapset, to be stored in in the mapset's VAR file, to be set with e.g.
> > r.compress method=.
>
> That would probably be a nice solution. One could then create an
> "Export" mapset which can then be easily shared.

You could do the same by creating some "Export" mapset, then setting the
compression method and creating raster maps to be exported in this "Export"
mapset.

Implementation of a compression method set with g.gisenv is relatively easy.
Implementation of a compression method defined in a mapset's VAR file is a
bit more work and requir

Re: [GRASS-dev] Import PostGIS CURVEPOLYGON fails

[Markus Metz]

On Sat, Dec 28, 2019 at 11:46 PM Paulo van Breugel 
wrote:
>
> On Sat, Dec 28, 2019 at 10:45 PM Markus Metz
>  wrote:
> >
> >
> >
> > On Fri, Dec 27, 2019 at 4:49 PM Paulo van Breugel <
p.vanbreu...@gmail.com> wrote:
> > >
> > > Hi devs,
> > >
> > > I am trying to import a vector layer from a PostGIS database in GRASS
> > > GIS (7.8.3dev, GDAL 3.0.2, Ubuntu 18.04) , but a large number of the
> > > polygons are skipped with the following warnings:
> > >
> > > WARNING: Skipping unsupported geometry type 'CURVEPOLYGON'
> > > ...
> > >
> > > I though, but am not sure, that these polygons were imported without
> > > problems before I last updated grass and gdal (running grass gis
> > > 7.8.0, gdal 2.4).
> >
> > These CURVEPOLYGON polygons should be imported without problems in
GRASS GIS 7.6+ compiled with GDAL 2+.
>
> Indeed, last time I worked with this data was before I upgraded to
> gdal 3.02 (using the ppa https://qgis.org/ubuntugis/) and grass
> 7.8.3dev.
>
> > Can you provide an extract created with ogr2ogr in GeoPackage format
that contains such polygons for testing? That would help reproduce the
problem.
>
> See attached a small sample (the original data is the BGT postgis dump
> from https://nlextract.nl/downloads/), as well as a screenshot of the
> data opened in QGIS and the same data imported in grass with v.in.ogr.

Thanks for the test data, I could reproduce the problem, fixed in master
264f753 and relbr78 7da40bf

Markus M
>
> >
> > Markus M
> > >
> > > Is there any way I can import those problematic curvepolygons in
grass gis?
> > >
> > > Mvg.
> > >
> > > Paulo
> > > ___
> > > grass-dev mailing list
> > > grass-dev@lists.osgeo.org
> > > https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Import PostGIS CURVEPOLYGON fails

[Markus Metz]

On Fri, Dec 27, 2019 at 4:49 PM Paulo van Breugel 
wrote:
>
> Hi devs,
>
> I am trying to import a vector layer from a PostGIS database in GRASS
> GIS (7.8.3dev, GDAL 3.0.2, Ubuntu 18.04) , but a large number of the
> polygons are skipped with the following warnings:
>
> WARNING: Skipping unsupported geometry type 'CURVEPOLYGON'
> ...
>
> I though, but am not sure, that these polygons were imported without
> problems before I last updated grass and gdal (running grass gis
> 7.8.0, gdal 2.4).

These CURVEPOLYGON polygons should be imported without problems in GRASS
GIS 7.6+ compiled with GDAL 2+. Can you provide an extract created with
ogr2ogr in GeoPackage format that contains such polygons for testing? That
would help reproduce the problem.

Markus M
>
> Is there any way I can import those problematic curvepolygons in grass
gis?
>
> Mvg.
>
> Paulo
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] any special reason why choice of raster compression method is done via environmental variable ?

[Markus Metz]

On Tue, Dec 10, 2019 at 6:47 AM Moritz Lennert 
wrote:
>
> On 9/12/19 22:50, Markus Metz wrote:
> >   Hi Moritz,
> >
> > On Wed, Dec 4, 2019 at 10:28 AM Moritz Lennert
> > mailto:mlenn...@club.worldonline.be>>
wrote:
> >  >
> >  > Hi Markus,
> >  >
> >  > In recent days, I've been confronted several times with the issue of
> >  > people trying to share data among themselves, but using different
> >  > versions of GRASS, and so raster data compressed in a more recent
> >  > version of GRASS was not usable in an older version of GRASS.
> >  >
> >  > Now, I agree that generally the solution is to tell people to use the
> >  > latest and greatest, but this is not always possible / it is not
> >  > necessarily highest on the list of priorities of people to see how
they
> >  > can install the latest version of GRASS within their particular
> > environment.
> >  >
> >  > Obviously, those with the latest version of GRASS can simple
recompress
> >  > using ZLIB. However, compression method is defined as an environment
> >  > variable. This is somewhat daunting to many MS Windows users out
there.
> >  > Is there any specific reason that lead to the choice of not using a
> >  > parameter to allow the choice of compression method (possibly to
> >  > override a default that is still defined by an environment variable)
?
> >
> > Such a parameter would need to be added to every module creating raster
> > output.
>
> My request is more linked to use cases where one would like to share
> data (e.g. with r.pack) with other GRASS GIS users who do not
> necessarily have access to the same compression method, not necessarily
> to changing the default compression method. I was just wondering whether
> it might be easily possible to just implement r.compress method= as a
> quick way to recompress a specific map with a chosen method, overriding
> the default method. Currently, to do that, you have to change the
> default method by changing the env variable, run r.compress, then change
> the variable back to the value one wishes generally to use as default.

recompressing an existing raster map seems like a waste of time. Why not
compress it in the first place with the appropriate method?

Recompressing an existing map with a chosen method via r.compress method=XY
means that a new raster map should not be compressed with the default
method but another supported method. This would require a new function in
lib/raster like Rast_open_new_compression_method() that overrides the
default compression method. Of course this is possible, but it requires
substantial changes to lib/raster.

About the use cases:
People want to share a mapset: in this case all GRASS raster maps should be
compressed with the most compatible compression method (ZLIB). Currently,
this can be done with an env var. Alternatively, a new mechanism via
g.gisenv could be established.
Another option could be to set the default compression method per mapset,
to be stored in in the mapset's VAR file, to be set with e.g. r.compress
method=.

The point is that it is fairly easy to set a suitable compression method
before the raster lib is initialized, and there are different possible
mechanisms apart from the current method using an env var that are easy to
implement.

> Obviously, you can always just export as tiff and share that, but that
> just feels less elegant. Anyway, this is probably somewhat of a luxury
> problem :-)

This only a luxury problem if you don't need to care about processing speed
and disk usage. The different compression methods by now available in GRASS
allow for optimization for speed (LZ4) or for optimization for disk usage
(BZIP2 for larger CELL maps). An easy (easier) way to set the compression
method seems reasonable.

Markus M

>
> Moritz
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [OSGeo/grass] libproj (#252)

[Markus Metz]

On Tue, Dec 10, 2019 at 9:27 AM Markus Neteler  wrote:
>
> Hi,
>
> On Mon, Dec 9, 2019 at 11:32 PM Helmut Kudrnovsky <
notificati...@github.com> wrote:
>>
>> After backporting, another RC for 7.8.2 needed?
>>
>> —
>> You are receiving this because you are subscribed to this thread.
>> Reply to this email directly, view it on GitHub, or unsubscribe.
>
>
> ... I would suggest to leave it for 7.8.3 and get out 7.8.2 now.

this fixes a bug that has been known for a long time, and the fix is quite
important for correct datum transformations with all versions of PROJ.
Therefore I have backported the fix to both relbr78 and relbr76. Please
include this fix in 7.8.2.

Markus M

>
> my 0.02 cents
>
> markusN
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] any special reason why choice of raster compression method is done via environmental variable ?

[Markus Metz]

 Hi Moritz,

On Wed, Dec 4, 2019 at 10:28 AM Moritz Lennert 
wrote:
>
> Hi Markus,
>
> In recent days, I've been confronted several times with the issue of
> people trying to share data among themselves, but using different
> versions of GRASS, and so raster data compressed in a more recent
> version of GRASS was not usable in an older version of GRASS.
>
> Now, I agree that generally the solution is to tell people to use the
> latest and greatest, but this is not always possible / it is not
> necessarily highest on the list of priorities of people to see how they
> can install the latest version of GRASS within their particular
environment.
>
> Obviously, those with the latest version of GRASS can simple recompress
> using ZLIB. However, compression method is defined as an environment
> variable. This is somewhat daunting to many MS Windows users out there.
> Is there any specific reason that lead to the choice of not using a
> parameter to allow the choice of compression method (possibly to
> override a default that is still defined by an environment variable) ?

Such a parameter would need to be added to every module creating raster
output.

The reason for an environment variable is historical: previously, CELL maps
were by default compressed with RLE, this could be changed with the env var
GRASS_INT_ZLIB. This mechanism of using an env var for raster compression
has been extended.

Other solutions than an env var would be to use g.gisenv/$GISRC to set the
default compression method per user or a mapset-specific compression method
(to be set with a modified r.compress?) stored in the mapset's VAR file.

Regarding MS Windows users + raster compression: I think it is reasonable
to assume at least GRASS 7.6 which has support all compression methods
available in GRASS.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [EXTERNAL] ZSTD error

[Markus Metz]

On Wed, Dec 4, 2019 at 2:30 PM Anna Petrášová  wrote:
>
> That might be it! Thanks for the answers.
>
> On Wed, Dec 4, 2019 at 1:51 PM Newcomb, Doug  wrote:
>>
>> It may be an older version of the zstd library compiled on 7.6.0.

yes, there were some reports of zstd minor version (1.x vs 1.y)
incompatibilities regarding GRASS ZSTD compression. Unfortunately, I could
not find corresponding notes in the ZSTD documentation.

Markus M
>>
>> Doug
>>
>> On Wed, Dec 4, 2019 at 4:01 AM Anna Petrášová 
wrote:
>>>
>>> Hi,
>>>
>>> I can't open colleague's data, I am getting:
>>>
>>> ZSTD compression error -2: Unknown frame descriptor
>>> Error uncompressing fp raster data for row 24 of :
error code -1
>>>
>>> r.compress -p:
>>>  is compressed (method 5: ZSTD). Data type: FCELL
>>>  has a compressed NULL file
>>>
>>> Any idea what could be going on? He said it works ok on his Mac and
Windows? I tried latest master and 78 with ZSTD compression on my side.
>>>
>>> Thank you,
>>> Anna
>>
>>
>>
>> --
>> Doug Newcomb - Cartographer
>> USFWS
>> 551F Pylon Dr
>> Raleigh, NC
>> 919-856-4520 ext. 14 doug_newc...@fws.gov
>>
-
>>
>> NOTE: This email correspondence and any attachments to and from this
sender is subject to the Freedom of Information Act (FOIA) and may be
disclosed to third parties.
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] v.in.ogr: important bug fix when importing from PostgreSQL

[Markus Metz]

Hi all,

we found a serious bug when importing polygons from a PostgreSQL/PostGIS
db: resultant areas might have wrong attributes because of a mix-up of the
connection from geometries to attributes. This bug has been fixed in master
and relbr78 (a new release of GRASS 7.8 should probably come out soon).

Apparently this bug only affects those OGR datasources with a client/server
database system, e.g. PostgreSQL or MySQL.

This bug does not affect common file-based formats like shapefile,
geopackage, etc.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-user] WinGRASS build up (WIP)

[Markus Metz]

On Wed, Nov 13, 2019 at 3:39 PM Markus Neteler  wrote:
>
> On Wed, Nov 13, 2019 at 3:27 PM Pedro Venâncio
>  wrote:
> >
> > Hi all again,
> >
> > I've tested GRASS standalone and I got the same error with
WinGRASS-7.8.1-1-Setup-x86_64.
> >
> > Then I uninstalled this version and installed
WinGRASS-7.8.0-2-Setup-x86_64 and everything worked fine.
> >
> > So, it seems that there is something wrong with GRASS-7.8.1 in my
machine. Also when running algorithms in CLI, GRASS throws the message
about gdal300.dll, but that dll is present in the path specified
(screenshot attached).
> >
> > Any hint about what can be wrong?
>
> Probably this recent fix is needed?
>   https://github.com/OSGeo/grass/pull/191

No, because GRASS is actually looking for gdal300.dll.

Looks like a packaging error in WinGRASS-7.8.1-1-Setup-x86_64, if others
can confirm this.

Markus M
>
> It will be part of 7.8.2.
>
> Best regards,
> Markus
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-user] GRASS GIS 7.8.1 released with PROJ 6 and GDAL 3 support

[Markus Metz]

Regarding GDAL 3 support, we still need to add the new GDAL library
versions to lib/raster/gdal.c for r.external to work.

Markus M

On Tue, Nov 12, 2019 at 8:56 AM Markus Neteler  wrote:

> What's new in a nutshell
>
> As a follow-up to the recent GRASS GIS 7.8.0 release we have pusblished
> the new stable release GRASS GIS 7.8.1. Besides improving the Python 3
> compatibility efforts have concentrated on implementing PROJ 6 and GDAL 3
> support.
>
> An overview of the new features in the 7.8 release series is available at new
> features in GRASS GIS 7.8
> .
> Binaries/Installer download:
>
>- winGRASS 7.8.1: 32bit standalone installer
>
> 
>| 64bit standalone installer
>
> 
>- winGRASS 7.8.1 OSGeo4W: 32bit OSGeo4W installer
> | 64bit
>OSGeo4W installer
>
>- Debian 
>- Fedora and EPEL7
> (CentOS7,
>RHEL7, ... - included in Fedora 31+)
>- Linux Mint: see Ubuntu
>- Ubuntu
>
>(ubuntugis-unstable)
>-
> *... further binary packages for other Linux distributions will follow
>shortly, please check at software downloads
>. *
>
> Source code download:
>
>- https://grass.osgeo.org/grass78/source/
>- https://grass.osgeo.org/grass78/source/grass-7.8.1.tar.gz
>- To get the GRASS GIS 7.8.1 source code directly from GitHub, see here
>.
>
> More details:
>
> See also our detailed announcement:
> https://trac.osgeo.org/grass/wiki/Release/7.8.1-News
> https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures78 (overview of new
> 7.8 stable release series)
> https://grass.osgeo.org/grass7/manuals/addons/ (list of available addons)
>
> Thanks to all contributors!
> ___
> grass-user mailing list
> grass-u...@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

On Fri, Nov 8, 2019 at 10:15 AM Martin Landa  wrote:
>
> Hi,
>
> pá 8. 11. 2019 v 8:22 odesílatel Markus Metz
>  napsal:
> > > I will try to reword my question. Are you aware of any related
> > > blockers which should avoid us packaging upcoming GRASS 7.8.1 with
> > > GDAL3/Proj6? Thanks for clarification! Martin
> >
> > No, that's the only bug I found so far.
>
> thanks for clarification! In that case let's release 7.8.1 ASAP in
> order to allow OSGeo4W switch to GDAL3/PROJ6.

the EPSG:3857 (Web Mercator) bug has been fixed in master 4167542. Is there
still time to backport to relbr78?

It is pure chance that the previous method worked, this is a fix for the
GRASS hack to deal with "the hacky way of encodign webmerc in GDAL WKT1" to
cite PROJ 6.

All this trouble because +proj=webmerc exists only since PROJ 5.1.0, and
because even GDAL 3 and PROJ 6 (at least PROJ 6 knows +proj=webmerc) still
report "+proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0
+k=1 +units=m +nadgrids=@null +wktext +no_defs" as a workaround.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

On Thu, Nov 7, 2019 at 9:39 PM Martin Landa  wrote:
>
> Hi Markus,
>
> čt 7. 11. 2019 v 19:50 odesílatel Markus Metz
>  napsal:
> > yes, EPSG:3857 is causing trouble. This is the WGS 84 / Pseudo-Mercator
projection on a sphere with radius 6378137 that needs special treatment
which no longer works with GRASS + PROJ 6 + GDAL 3.
>
> I will try to reword my question. Are you aware of any related
> blockers which should avoid us packaging upcoming GRASS 7.8.1 with
> GDAL3/Proj6? Thanks for clarification! Martin

No, that's the only bug I found so far.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

On Thu, Nov 7, 2019 at 8:04 PM Even Rouault 
wrote:
>
> On jeudi 7 novembre 2019 19:50:33 CET Markus Metz wrote:
> > On Thu, Nov 7, 2019 at 6:47 PM Martin Landa 
wrote:
> > > Hi,
> > >
> > > čt 7. 11. 2019 v 15:17 odesílatel Markus Metz
> > >
> > >  napsal:
> > > > PROJ 6 support is not yet finished in GRASS. Currently it sort of
works
> >
> > in master and relbr78, but I expect troubles because GRASS is heavily
> > relying on deprecated proj strings as CRS definitions. This is becoming
> > problematic e.g. when creating locations from GDAL/OGR datasets because
> > GRASS goes from OGR spatialreference through proj string to GRASS
> > definition.
> >
> > > OSGeo4W packages are planned to be switched to GDAL3/PROJ6 soon, are
> > > you aware of any known bugs?
> >
> > yes, EPSG:3857 is causing trouble. This is the WGS 84 / Pseudo-Mercator
> > projection on a sphere with radius 6378137 that needs special treatment
> > which no longer works with GRASS + PROJ 6 + GDAL 3.
> >
> > g.proj epsg=3857 -p gives me
> > -PROJ_INFO-
> > name   : WGS 84 / Pseudo-Mercator
> > datum  : wgs84
> > ellps  : wgs84
> > proj   : merc
> > lat_ts : 0
> > lon_0  : 0
> > x_0: 0
> > y_0: 0
> > k  : 1
> > wktext : defined
> > no_defs: defined
> > -PROJ_EPSG-
> > epsg   : 3857
> > -PROJ_UNITS
> > unit   : meter
> > units  : meters
> > meters : 1
> >
> > which is wrong because it should be a Mercator projection on a sphere,
not
> > an ellipsoid
>
> Well, this is a messy topic. EPSG:3857 uses the WebMercator projection,
which
> is not the standard Mercator projection since it uses spherical formulas
even
> if the datum and ellipsoid is WGS84

Indeed messy. Look at the WKT2 representation of projinfo -o WKT2_2018
EPSG:3857: projection on the ellipsoid "WGS 84", not on a sphere.
>
> So neither  proj=merc + ellps=wgs84 or proj=merc + ellps=sphere are
> technically correct
>
> Nothing new with PROJ 6 for that case
>
> > projinfo -o PROJ EPSG:3857
> > gives me
> > PROJ.4 string:
> > +proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +k=1
> > +units=m +nadgrids=@null +wktext +no_defs +type=crs
>
> Yes, this is a hack of PROJ.4 still supported by PROJ 6 to more or less
get
> the correct result of WebMercator using the standard Mercator, by lying
on the
> ellipsoid definition (using the Sphere) and applying a nadgrids=null, so
that
> no datum transformation between WGS84 and that Sphere is attempted !

I don't understand "lying on the ellipsoid definition (using the Sphere)",
it should be a sphere with radius 6378137, no?

> >
> > that is correct, but unfortunately not returned by GDAL's function
> > OSRExportToProj4() which is now deprecated.
>
> Hum, I do ot agree with this. This *is* still what is returned by GDAL
> OSRExportToProj4() with GDAL + PROJ master :
>
> $ gdalsrsinfo EPSG:3857
>
> PROJ.4 : +proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0
> +k=1 +units=m +nadgrids=@null +wktext +no_defs

OK, I will figure out what goes wrong in GRASS regarding the representation
of EPSG:3857.

Thanks a lot for your very fast response, Even!

Markus M
>
> And you also actually use (PROJ 6 required here)
>
> $ gdalsrsinfo "+proj=webmerc +datum=WGS84"
> (or $ projinfo "+proj=webmerc +datum=WGS84 +type=crs" -o PROJ )
>
> PROJ.4 : +proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0
> +k=1 +units=m +nadgrids=@null +wktext +no_defs
>
>
> > That means GRASS needs to avoid
> > OSRExportToProj4() with GDAL 3+:
>
> In the general case, I agree with your conclusion, but not for the above
one
> :-)
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

On Thu, Nov 7, 2019 at 6:47 PM Martin Landa  wrote:
>
> Hi,
>
> čt 7. 11. 2019 v 15:17 odesílatel Markus Metz
>  napsal:
> > PROJ 6 support is not yet finished in GRASS. Currently it sort of works
in master and relbr78, but I expect troubles because GRASS is heavily
relying on deprecated proj strings as CRS definitions. This is becoming
problematic e.g. when creating locations from GDAL/OGR datasets because
GRASS goes from OGR spatialreference through proj string to GRASS
definition.
>
> OSGeo4W packages are planned to be switched to GDAL3/PROJ6 soon, are
> you aware of any known bugs?

yes, EPSG:3857 is causing trouble. This is the WGS 84 / Pseudo-Mercator
projection on a sphere with radius 6378137 that needs special treatment
which no longer works with GRASS + PROJ 6 + GDAL 3.

g.proj epsg=3857 -p gives me
-PROJ_INFO-
name   : WGS 84 / Pseudo-Mercator
datum  : wgs84
ellps  : wgs84
proj   : merc
lat_ts : 0
lon_0  : 0
x_0: 0
y_0: 0
k  : 1
wktext : defined
no_defs: defined
-PROJ_EPSG-
epsg   : 3857
-PROJ_UNITS
unit   : meter
units  : meters
meters : 1

which is wrong because it should be a Mercator projection on a sphere, not
an ellipsoid

from https://proj.org/operations/projections/webmerc.html:
"This is a variant of the regular Mercator projection, except that the
computation is done on a sphere, using the semi-major axis of the
ellipsoid."
and
"All parameters for the projection are optional, except the ellipsoid
definition, which is WGS84 for the typical use case of EPSG:3857."
and
"+ellps=

See proj -le for a list of available ellipsoids.

Defaults to “GRS80”."

projinfo -o PROJ EPSG:3857
gives me
PROJ.4 string:
+proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +k=1
+units=m +nadgrids=@null +wktext +no_defs +type=crs

that is correct, but unfortunately not returned by GDAL's function
OSRExportToProj4() which is now deprecated. That means GRASS needs to avoid
OSRExportToProj4() with GDAL 3+:
https://gdal.org/doxygen/ogr__srs__api_8h.html#ae410c0b0bb86b7d7903193828db8a1f5
-->
WarningUse of this function is discouraged. Its behaviour in GDAL >= 3 /
PROJ >= 6 is significantly different from earlier versions. In particular
+datum will only encode WGS84, NAD27 and NAD83, and +towgs84/+nadgrids
terms will be missing most of the time. PROJ strings to encode CRS should
be considered as a legacy solution. Using a AUTHORITY:CODE or WKT
representation is the recommended way.<--
Thus GRASS should use an SRID or WKT instead of proj4 strings.

Markus M

>
> > For GDAL 3 + PROJ 6, GRASS should switch to WKT as main CRS definition
format, ideally also storing a WKT definition next to PROJ_INFO. When
creating locations from GDAL/OGR datasets, GRASS should go from OGR
spatialreference through WKT to GRASS definition, and probably use the new
PROJ6+ methods as much as possible.
>
> Yes, good topic for
> https://grasswiki.osgeo.org/wiki/GRASS_GIS_Community_Sprint_Prague_2019
> :-)
>
> > When getting the projection info of a GRASS location, GRASS should then
check in that order 1) WKT, 2) EPSG or other SRID, 3) GRASS native
definition (which deviated from proj definition some decades ago).
> >
> > That means, g.proj, r.in.gdal, r.external, v.in.ogr, v.external need
updates, and the mechanism of conversion between different formats of CRS
definitions in lib/proj needs to be largely rewritten.
>
> This task for next stable GRASS release (7.10?) I guess.
>
> Ma
>
> --
> Martin Landa
> http://geo.fsv.cvut.cz/gwiki/Landa
> http://gismentors.cz/mentors/landa
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

PROJ 6 support is not yet finished in GRASS. Currently it sort of works in
master and relbr78, but I expect troubles because GRASS is heavily relying
on deprecated proj strings as CRS definitions. This is becoming problematic
e.g. when creating locations from GDAL/OGR datasets because GRASS goes from
OGR spatialreference through proj string to GRASS definition.

For GDAL 3 + PROJ 6, GRASS should switch to WKT as main CRS definition
format, ideally also storing a WKT definition next to PROJ_INFO. When
creating locations from GDAL/OGR datasets, GRASS should go from OGR
spatialreference through WKT to GRASS definition, and probably use the new
PROJ6+ methods as much as possible.

When getting the projection info of a GRASS location, GRASS should then
check in that order 1) WKT, 2) EPSG or other SRID, 3) GRASS native
definition (which deviated from proj definition some decades ago).

That means, g.proj, r.in.gdal, r.external, v.in.ogr, v.external need
updates, and the mechanism of conversion between different formats of CRS
definitions in lib/proj needs to be largely rewritten.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GRASS 7.8.1 ?

[Markus Metz]

On Mon, Oct 28, 2019 at 9:18 PM Helmut Kudrnovsky  wrote:
>
> Helmut Kudrnovsky wrote
> >>I would like to get PROJ 6 support into G 7.8.1, can we get RC1 out >end
> of
> > this week?
> >
> > Will grass support then also GDAL 3?

Yes, GRASS will then work with GDAL 3 + PROJ 6 (just finished some local
tests, looks all fine).

Markus M
>
> reasoning, mentioned in gdal/news
> (https://github.com/OSGeo/gdal/blob/v3.0.1/gdal/NEWS):
>
> == GDAL/OGR 3.0.0 - General Changes ==
> [...]
> All:
>  * PROJ >= 6.0 is a required external dependency
> [...]
>
> mixing old and new proj/gdal while building winGRASS would be a really
> pain
>
>
>
> -
> best regards
> Helmut
> --
> Sent from: http://osgeo-org.1560.x6.nabble.com/Grass-Dev-f3991897.html
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GRASS 7.8.1 ?

[Markus Metz]

I would like to get PROJ 6 support into G 7.8.1, can we get RC1 out end of
this week?

Markus M

On Sun, Oct 20, 2019 at 2:30 PM Markus Neteler  wrote:

> On Sat, Oct 19, 2019 at 8:20 PM Markus Neteler  wrote:
> >
> > Hi,
> >
> > Anna Petrášová  schrieb am Mo., 14. Okt. 2019,
> 21:11:
> >>
> >> That's good news! Thank you. I would like to have some time to fix at
> least some of the windows issues that popped up with Python3, I need to
> review what needs to be done, I lost track of it. But I agree we should
> release fairly soon.
> >>
> >
> > I agree that we should release soon.
>
> BTW: here the currently 43 commits to releasebranch_7_8 since the 7.8.0
> release:
> https://github.com/OSGeo/grass/compare/7.8.0...releasebranch_7_8
>
> > How about RC1 a week?
>
> Markus
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

On Wed, Sep 25, 2019 at 10:55 PM Even Rouault 
wrote:
>
> On mercredi 25 septembre 2019 22:47:21 CEST Markus Metz wrote:
> >
> > PROJ6 will not use the best method if any required datum transformation
> > grid is not available. Users will need to obtain the corresponding grid
> > themselves, which raises another question: where to put this grid? On
> > Linux, this would be e.g. /usr/share/proj, but if the user can not
write to
> > /usr/share/proj, the grids need to be saved somewhere in $HOME, and
PROJ6
> > must look at that place. This problem, i.e. the place where PROJ6 should
> > look for in $HOME is not yet solved AFAICT (proj-6.2.0).
>
> Applications may decide for an appropriate user directory and set it with
> proj_context_set_search_paths() (in that case this overrides PROJ_LIB or
> hardcoded directories, so you have to add in the search paths if this is
the
> desired behaviour)
>
> QGIS has code to do exactly that:
>
https://github.com/qgis/QGIS/blob/65359bc7eafbfe967c669d1428eeedb87c5cd2a1/
> src/core/qgsapplication.cpp#L312
> and
>
https://github.com/qgis/QGIS/blob/d45c3dd4f1e8fe3642d42d84aa978b28dba913aa/
> src/core/qgsprojutils.cpp#L258

The (my) idea is that proj suggests reasonable paths to store grids, search
paths should be available in PJ_INFO. Otherwise applications may decide for
a directory specific to that application, not generally valid for proj used
by different applications. Applications can then check the existing proj
search paths and install grids to a path where the current user has write
access. Thus no need to use proj_context_set_search_paths() and no need to
install the same grid(s) repeatedly.

However, there are no default search paths in PJ_INFO. Can you give a hint
where the default search paths accessible in the C API?

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

Several different methods might be available to reproject from one CRS to
another CRS. PROJ6 can select the most appropriate method if a bounding box
is provided. This means that the selected method depends on the bounding
box and that results of a reprojection can differ depending on the provided
bounding box.

In GRASS, this bounding box can be obtained from the current region. That
means if the current region changes, the method and thus the results of
coordinate reprojection might change. This effect can not be underestimated.

With the current pull request #118 the current region is used to help PROJ6
select the best method, and [r|v].import should work as before. Results can
be different compared to PROJ5 or earlier.

IMHO, there is no way around that users become more familiar with details
of coordinate reprojection and need to read the output of [r|v].proj
carefully regarding different methods known to PROJ6.

PROJ6 will not use the best method if any required datum transformation
grid is not available. Users will need to obtain the corresponding grid
themselves, which raises another question: where to put this grid? On
Linux, this would be e.g. /usr/share/proj, but if the user can not write to
/usr/share/proj, the grids need to be saved somewhere in $HOME, and PROJ6
must look at that place. This problem, i.e. the place where PROJ6 should
look for in $HOME is not yet solved AFAICT (proj-6.2.0).

Markus M

On Wed, Sep 4, 2019 at 9:46 PM Markus Metz 
wrote:

>
>
> On Wed, Sep 4, 2019 at 6:01 PM Markus Neteler  wrote:
> >
> > On Wed, Sep 4, 2019 at 4:31 AM Anna Petrášová 
> wrote:
> > > On Tue, Sep 3, 2019 at 4:22 AM Markus Metz <
> markus.metz.gisw...@gmail.com> wrote:
> > >>
> > >> Hi all,
> > >>
> > >> there is a new PR for PROJ6 support in GRASS
> > >> https://github.com/OSGeo/grass/pull/118
> > >>
> > >> There are two important changes when using PROJ6:
> > >>
> > >> First, reprojection with v.proj and r.proj is no longer always
> possible without the user making informed decisions. The reason is that
> there can be several different operations available to reproject
> coordinates from one CRS to another CRS. These different operations are
> listed and the user has to provide the appropriate operation with the
> pipeline option, taking care of any axisswap.
> > >
> > > first, thanks for all the work. Second, I don't see how most users are
> supposed to know what to pick. Is there perhaps a way to pick a good
> default? I just can't imagine not having r.import/v.import...
> >
> > I see the same problem: users won't know what to select if defaults
> > are gone with PROJ 6.
>
> We can provide information that enables users to make an informed
> decision. The options listed by PROJ6 are ordered by guessed best choice,
> i.e. the first one is, given the provided information, the best choice. But
> users need to review the list of options.
>
> Reprojection from one CRS to another CRS was never easy. For both raster
> and vector data, some preparation was always needed to decide on
> appropriate settings. PROJ6 provides methods to improve the accuracy of
> reprojected coordinates, depending on the actual use case. The user decides
> (must decide).
> >
> > > Anna
> > >>
> > >>
> > >> Second, axis order is no longer always easting, northing, e.g. for
> EPSG:4326 it is northing, easting, and an axisswap might need to be removed
> from operations provided by PROJ.
> > >>
> > >> There are many more changes (see details in the PR), but these are
> the two most important ones.
> > >>
> > >> Feedback welcome!
> >
> > Thanks for your hard work.
> >
> > I talked to Robert Bivand here at the GeoSTAT 2019 in Münster who gave
> > a talk about the topic:
> >
> > "Not just R-spatial: sustaining open source geospatial software stacks"
> > https://github.com/rsbivand/geostat19_talk
> >
> > You can quick-view the talk in rendered HTML like this (maybe there
> > are better ways):
> >
> http://htmlpreview.github.io/?https://raw.githubusercontent.com/rsbivand/geostat19_talk/master/docs/geostat_bivand_19.html
> >
> > It contains a section about PROJ (6).
>
> Not so random citation:
> "...we need to manipulate the CRS read in with the file to insert our
> choice of how to make the transformation..."
>
> I will try to restrict the number of options based on the current region
> in a modified PR.
>
> Markus M
>
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

On Wed, Sep 4, 2019 at 6:01 PM Markus Neteler  wrote:
>
> On Wed, Sep 4, 2019 at 4:31 AM Anna Petrášová 
wrote:
> > On Tue, Sep 3, 2019 at 4:22 AM Markus Metz <
markus.metz.gisw...@gmail.com> wrote:
> >>
> >> Hi all,
> >>
> >> there is a new PR for PROJ6 support in GRASS
> >> https://github.com/OSGeo/grass/pull/118
> >>
> >> There are two important changes when using PROJ6:
> >>
> >> First, reprojection with v.proj and r.proj is no longer always
possible without the user making informed decisions. The reason is that
there can be several different operations available to reproject
coordinates from one CRS to another CRS. These different operations are
listed and the user has to provide the appropriate operation with the
pipeline option, taking care of any axisswap.
> >
> > first, thanks for all the work. Second, I don't see how most users are
supposed to know what to pick. Is there perhaps a way to pick a good
default? I just can't imagine not having r.import/v.import...
>
> I see the same problem: users won't know what to select if defaults
> are gone with PROJ 6.

We can provide information that enables users to make an informed decision.
The options listed by PROJ6 are ordered by guessed best choice, i.e. the
first one is, given the provided information, the best choice. But users
need to review the list of options.

Reprojection from one CRS to another CRS was never easy. For both raster
and vector data, some preparation was always needed to decide on
appropriate settings. PROJ6 provides methods to improve the accuracy of
reprojected coordinates, depending on the actual use case. The user decides
(must decide).
>
> > Anna
> >>
> >>
> >> Second, axis order is no longer always easting, northing, e.g. for
EPSG:4326 it is northing, easting, and an axisswap might need to be removed
from operations provided by PROJ.
> >>
> >> There are many more changes (see details in the PR), but these are the
two most important ones.
> >>
> >> Feedback welcome!
>
> Thanks for your hard work.
>
> I talked to Robert Bivand here at the GeoSTAT 2019 in Münster who gave
> a talk about the topic:
>
> "Not just R-spatial: sustaining open source geospatial software stacks"
> https://github.com/rsbivand/geostat19_talk
>
> You can quick-view the talk in rendered HTML like this (maybe there
> are better ways):
>
http://htmlpreview.github.io/?https://raw.githubusercontent.com/rsbivand/geostat19_talk/master/docs/geostat_bivand_19.html
>
> It contains a section about PROJ (6).

Not so random citation:
"...we need to manipulate the CRS read in with the file to insert our
choice of how to make the transformation..."

I will try to restrict the number of options based on the current region in
a modified PR.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: Re: [OSGeoLive] #2181: GRASS freezes when using r.import on file without projection

[Markus Metz]

On Sun, Sep 1, 2019 at 8:35 AM Markus Neteler  wrote:
>
>
>
> 
> Von: OSGeoLive 
> Gesendet: 31. August 2019 17:38:31 OESZ
> Betreff: Re: [OSGeoLive] #2181: GRASS freezes when using r.import on file
without projection
>
> #2181: GRASS freezes when using r.import on file without projection
> ---+
>  Reporter:  johanvdw   |   Owner:  osgeolive@…
>  Type:  defect |  Status:  new
>  Priority:  major  |   Milestone:  OSGeoLive13.0
> Component:  OSGeoLive  |  Resolution:
>  Keywords:  grass  |
> ---+
> Changes (by kalxas):
>
>  * keywords:   => grass
>  * cc: osgeolive@… (added)
>  * priority:  minor => major
>  * type:  task => defect
>  * milestone:   => OSGeoLive13.0
>
>
> --
> Ticket URL: 
> OSGeoLive 
> self-contained bootable DVD, USB thumb drive or Virtual Machine based on
Lubuntu, that allows you to try a wide variety of open source geospatial
software without installing anything.
>

The reason was that r.import did not change back to the original location +
mapset by resetting GISRC. Fixed in master, relbr78, and relbr76, also for
v.import.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] PROJ6 support in GRASS

[Markus Metz]

Hi all,

there is a new PR for PROJ6 support in GRASS
https://github.com/OSGeo/grass/pull/118

There are two important changes when using PROJ6:

First, reprojection with v.proj and r.proj is no longer always possible
without the user making informed decisions. The reason is that there can be
several different operations available to reproject coordinates from one
CRS to another CRS. These different operations are listed and the user has
to provide the appropriate operation with the pipeline option, taking care
of any axisswap.

Second, axis order is no longer always easting, northing, e.g. for
EPSG:4326 it is northing, easting, and an axisswap might need to be removed
from operations provided by PROJ.

There are many more changes (see details in the PR), but these are the two
most important ones.

Feedback welcome!
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [release planning] GRASS GIS 8.0.0

[Markus Metz]

On Mon, Aug 5, 2019 at 3:55 PM Markus Neteler  wrote:
>
> On Mon, Aug 5, 2019 at 9:11 AM Markus Neteler  wrote:
> > On Mon, Aug 5, 2019 at 8:38 AM Markus Metz
> >  wrote:
> > > On Sun, Aug 4, 2019 at 3:20 PM Markus Neteler 
wrote:
> > > >
> > > > Hi devs,
> > > >
> > > > since we now have the 7.8 release branch, I suggest to get rid of
> > > > Python 2 support in master.
> > > >
> > > > This will greatly simplify the code. Python 2 is EOL anyway in a
few months.
> > >
> > > I guess the first step would be to change the deprecated shebang
> > > #!/usr/bin/env python
> > > to
> > > #!/usr/bin/env python3
> >
> > I have created a related PR:
> > https://github.com/OSGeo/grass/pull/75
>
> FYI: It has been reviewed and merged.
>
> I am tempted to apply it also to relbranch78... opinions?

+1 to apply it also to relbr78

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [release planning] GRASS GIS 7.8.0 with Python3 support

[Markus Metz]

On Sun, Aug 4, 2019 at 12:19 PM Markus Neteler  wrote:
>
> Hi,
>
> On Sun, Aug 4, 2019 at 10:44 AM Markus Neteler  wrote:
> 
> >
> > as there were no objections, I have created the new
> >
> > https://github.com/OSGeo/grass/tree/releasebranch_7_8
>
> The needed grass77 -> grass78 updates are covered in:
>
> https://github.com/OSGeo/grass/pull/74
> (hope I found all needed changes; to go ahead I have merged right away).
>
> I have also created the cronjobs for weekly tarballs and manual pages:
>
> https://grass.osgeo.org/grass78/source/snapshot/
> https://grass.osgeo.org/grass78/binary/linux/snapshot/
> https://grass.osgeo.org/grass78/manuals/
>
> What's next:
>
> - GRASS GIS 7.8.0RC1 to be prepared asap
> - fix winGRASS
> - grass77 in master to be changed to grass79

+1

please let me know if I can help

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [release planning] GRASS GIS 8.0.0

[Markus Metz]

On Sun, Aug 4, 2019 at 3:20 PM Markus Neteler  wrote:
>
> Hi devs,
>
> since we now have the 7.8 release branch, I suggest to get rid of
> Python 2 support in master.
>
> This will greatly simplify the code. Python 2 is EOL anyway in a few
months.

I guess the first step would be to change the deprecated shebang
#!/usr/bin/env python
to
#!/usr/bin/env python3

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [QGIS-Developer] Bump to proj 6/gdal 3

[Markus Metz]

On Tue, Jul 23, 2019 at 10:24 AM Jürgen E. Fischer  wrote:
>
> Hi,
>
> On Tue, 23. Jul 2019 at 10:11:59 +1000, Nyall Dawson wrote:
> > Discussion welcome :)
>
> What's the state of GRASS with PROJ 6?

GRASS compiles with PROJ 6, but does not work properly with PROJ 6 because
PROJ 6 behaves differently compared to PROJ 5. PROJ 6 isupport in GRASS is
WIP.

Markus M
>
>
> Jürgen
>
> --
> Jürgen E. Fischer   norBIT GmbH Tel.
+49-4931-918175-31
> Dipl.-Inf. (FH) Rheinstraße 13  Fax.
+49-4931-918175-50
> Software Engineer   D-26506 Norden
https://www.norbit.de
> QGIS release manager (PSC)  GermanyIRC: jef on
FreeNode
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Merging PRs

[Markus Metz]

On Fri, Jul 12, 2019 at 4:23 PM Huidae Cho  wrote:
>
> Greetings Developers,
>
> In https://trac.osgeo.org/grass/wiki/HowToGit, it is not clear who is
responsible for merging PRs when. Are individual core developers encouraged
(allowed?) or discouraged to merge them? I'm talking about core, not
addons. How does it work?
>
> The main reason why I ask this question is because merging PRs into the
upstream master takes time and I cannot wait until that happens (who,
when?) to compile all my PRs into one build.
>
> If you create multiple branches off of master and create PRs, it wouldn't
be easy to keep track of all those individual branches in one place and
compile the personal "latest" version until the PRs get merged. For example,
>
> 1. master => feature1
> 2. master => feature2
>
> Compiling feature1 and feature2 branches would only give me feature1 and
feature2, respectively, but not both. Should I do something like this?
>
> 1. master => feature1
> 2. feature1 => feature2
>
> Then, how can we make sure that there won't be any conflicts between my
feature2 (<= feature1) branch and the OSGeo master?
>
> Not sure if what I'm doing is "right," but I'm doing this:
>
> 1. create a personal master branch "hcho" (yes, my name)
> 2. master => feature1
> 3. master => feature2
> 4. merge feature1 and feature2 into "hcho"
> 5. compile "hcho"
>
> To be honest, it's a lot of work :-(. Any suggestions or hints would be
very welcome.

Yes, I agree, this is a lot of work. git is convenient for contributors,
but a lot of work (much more than with svn) for maintainers.

If your PRs are incremental, i.e PR for feature 2 requires PR for feature
1, it would be easier to have only one PR with several commits.

I tend to merge your PRs related to MS Windows builds, but I can not test.
OTOH, testing by others occurs more likely if the changes are in master,
not in a PR.

Any opinions on merging PRs related to MS Windows builds to master without
testing by another person?

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] dealing with null values in rasters only within mask

[Markus Metz]

Hi Moritz,

On Fri, Jun 28, 2019 at 11:56 AM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
> On 28/06/19 11:10, Stefan Blumentrath wrote:
> > Hi Moritz,
> >
> > Did you consider r.stats -N for checking for NULL values within the
masked area, like:
> > r.stats -N input=tile,MASK
> > if there is *|1 in the output you have NULL cells in the tile...

the fastest way to get the total number of cells and the number of non-NULL
cells, ignoring any MASK, is r.info -s, e.g.
cells=4917486<-- total number of cells
n=4873149 <-- number of non-NULL cells

these numbers are stored in metadata and independent of the current region
>
> Nice trick, didn't actually know about the -N flag...
>
> >
> > For filling NULLs with a given value you could do:
> > r.mapcalc expression=tile_filled=if(MASK,if(isnull(tile),1,tile),null())

in this case, I recommend to create a custom mask that is not named MASK
but e.g. mymask, and the r.mapcalc expression probably needs some
modification regarding NULL values in mymask:
r.mapcalc expression="tile_filled = if(isnull(mymask) != 0 and mymask != 0,
if(isnull(tile), 1, tile), null())"

The reason is that if(MASK, ...) should return NULL if MASK is NULL which
is not what you want, you want " if MASK is not NULL"

Markus M

>
> Duh, now that I see this, it seems quite obvious.
>
> Thanks Stefan !
>
> (And thanks Ken for your ideas)
>
> Moritz
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] datetime function in v.in.ogr

[Markus Metz]

On Thu, Jun 27, 2019 at 9:27 PM Paulo van Breugel 
wrote:
>
> Hi devs,
>
> I am trying to import a shapefile using v.in.ogr, filtering on date
> using datetime, e.g.,
>
> v.in.ogr input=myshapefile.shp layer=myshapefile output=outputlayer
> where="datm_start >= datetime('2019-01-01')"
>
> I am getting the message:
>
> ERROR 1: Undefined function 'datetime' used.
> ERROR: Error setting attribute filter 'datm_start >=
datetime('2019-01-01')'
>
> Does this mean it is not possible to use functions in v.in.ogr? Or am I
> doing somthing wrong?

The OGR shapefile driver does not support this function. You need to import
the shapefile into GRASS, using anything but the dbf driver as database
driver (default is sqlite), then try to filter with v.extract
where="datm_start >= datetime('2019-01-01')"

HTH,

Markus M

>
> Running grass gis 7.6, Linux.
>
>
> Cheers,
>
> Paulo
>
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.terraflow vs. r.hydrodem

[Markus Metz]

Hi Stefan,

On Fri, Jun 14, 2019 at 10:03 PM Stefan Blumentrath <
stefan.blumentr...@nina.no> wrote:
>
> Hei,
>
> Before investing time in getting r.terraflow to work on larger regions I
actually tried r.fill.dir.
>
> The problem is that the results are significantly different and those of
r.terraflow and r.hydrodem (where no breaching is performed) are more
suitable for my needs. Here, r.terraflow fills most, r.fill.dir least and
r.hydrodem inbetween.

I remember from previous tests that r.fill.dir might need several runs.
Using elevation from nc_spm_08 as in your example, r.fill.dir apparently
needs to be run 5 times to resolve all problem areas. Maybe you get the
desired results by running r.fill.dir multiple times, using the output of
the previous run as input for the next run. r.fill.dir should also use far
less resources (disk space and memory) than r.terraflow or r.hydrodem.

Markus M
>
> Here a little comparison based on NC data:
>
> g.extension extension=r.hydrodem operation=add
> g.region -p raster=elevation
>
> r.fill.dir --overwrite --verbose input=elevation
output=elevation_fill_dir_depressionless
direction=elevation_fill_dir_direction areas=elevation_fill_dir_pas
>
> r.hydrodem -a --overwrite input=elevation memory=5000
output=elevation_hydrodem_depressionless
>
> r.terraflow --overwrite --verbose elevation=elevation
filled=elevation_terraflow_depressionless memory=5000
>
> for m in terraflow fill_dir hydrodem
> do
> r.mapcalc --o
expression="${m}_effect=if((elevation_${m}_depressionless-elevation)>0,elevation_${m}_depressionless-elevation,null())"
> done
>
> r.univar map="terraflow_effect"
> total null and non-null cells: 2025000
> total null cells: 1901290
>
> Of the non-null cells:
> --
> n: 123710
> minimum: 7.62939e-06
> maximum: 12.5168
> range: 12.5168
> mean: 1.49711
> mean of absolute values: 1.49711
> standard deviation: 1.97871
> variance: 3.91528
> variation coefficient: 132.169 %
> sum: 185206.976940155
>
> r.univar map="fill_dir_effect"
> total null and non-null cells: 2025000
> total null cells: 1964852
>
> Of the non-null cells:
> --
> n: 60148
> minimum: 7.62939e-06
> maximum: 10.9003
> range: 10.9003
> mean: 0.499953
> mean of absolute values: 0.499953
> standard deviation: 0.939429
> variance: 0.882526
> variation coefficient: 187.903 %
> sum: 30071.1971092224
>
> r.univar map="hydrodem_effect"
> total null and non-null cells: 2025000
> total null cells: 1020396
>
> Of the non-null cells:
> --
> n: 1004604
> minimum: 3.8147e-06
> maximum: 4.74686
> range: 4.74685
> mean: 0.00587574
> mean of absolute values: 0.00587574
> standard deviation: 0.08715
> variance: 0.00759513
> variation coefficient: 1483.22 %
> sum: 5902.79515457153
>
> Cheers
> Stefan
> 
> Fra: Markus Metz 
> Sendt: fredag 14. juni 2019 14.25
> Til: Stefan Blumentrath
> Kopi: grass-dev@lists.osgeo.org
> Emne: Re: [GRASS-dev] r.terraflow vs. r.hydrodem
>
>
>
> On Fri, Jun 14, 2019 at 1:23 PM Stefan Blumentrath <
stefan.blumentr...@nina.no> wrote:
> >
> > Hi,
> >
> > While trying to make r.terraflow work with larger regions, I was
wondering if it would be doable to add a flag to r.hydrodem that forces the
module to perform only sink filling and no breaching / carving?
> >
> > Sink filling is the function I need from r.terraflow but if it would be
an option to add this function to r.hydrodem, that would be fine as well of
course...
>
> what about r.fill.dir? It should work as is with larger regions.
>
> Markus M
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.terraflow vs. r.hydrodem

[Markus Metz]

On Fri, Jun 14, 2019 at 1:23 PM Stefan Blumentrath <
stefan.blumentr...@nina.no> wrote:
>
> Hi,
>
> While trying to make r.terraflow work with larger regions, I was
wondering if it would be doable to add a flag to r.hydrodem that forces the
module to perform only sink filling and no breaching / carving?
>
> Sink filling is the function I need from r.terraflow but if it would be
an option to add this function to r.hydrodem, that would be fine as well of
course...

what about r.fill.dir? It should work as is with larger regions.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] error in v.overlay: Unable to write spatial index. Wrong node position

[Markus Metz]

On Tue, Jun 11, 2019 at 11:12 AM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
> On 10/06/19 21:39, Markus Metz wrote:
> >
> >
> > On Mon, Jun 10, 2019 at 6:12 PM Moritz Lennert
> > mailto:mlenn...@club.worldonline.be>>
wrote:
> >  >
> >  > Hi,
> >  >
> >  > I just came across this error:
> >  >
> >  > **
> >  > ERROR: Unable to write spatial index. Wrong node position
(950329504) in
> >  > file (should be 950333637).
> >  > ERROR: Clipping steps failed. Check above error messages and see
> > following
> >  > details:
> >  > Module run None ['v.overlay', 'binput=temp_30311',
> > 'operator=and',
> >  > 'olayer=0,1,0', 'ainput=sq_batiments',
> >  > 'output=walous_temp_batiment_tile_18980'] ended with error
> >  > Process ended with non-zero return code 1. See errors in the
> > (error)
> >  > output.
> >  > **
> >  >
> >  > Any hint of what this might mean ? Should I file a bug report ?
> >
> > That means there was an error while writing out a spatial index, the
> > file size of the file holding the partially written out spatial index is
> > smaller than expected. Is this error reproducible on a different
> > partition/file system? Can you provide test data to reproduce?
>
> This was running on one node in an HPC. Up to now I haven't been able to
> reproduce it, so it was probably some glitch on that node.

Looking at the code again, this error can happen if not all data that
should be written out where actually written out, e.g. with no space left
on device. The diglib routines should do some more error (return code)
checking and fail earlier. I don't think it's a bug in the code, but maybe
error messages could be improved.

Markus M

> If I have
> other occurences, I'll try to narrow it down and provide test data.
>
>
> Moritz
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] error in v.overlay: Unable to write spatial index. Wrong node position

[Markus Metz]

On Mon, Jun 10, 2019 at 6:12 PM Moritz Lennert 
wrote:
>
> Hi,
>
> I just came across this error:
>
> **
> ERROR: Unable to write spatial index. Wrong node position (950329504) in
> file (should be 950333637).
> ERROR: Clipping steps failed. Check above error messages and see following
> details:
> Module run None ['v.overlay', 'binput=temp_30311', 'operator=and',
> 'olayer=0,1,0', 'ainput=sq_batiments',
> 'output=walous_temp_batiment_tile_18980'] ended with error
> Process ended with non-zero return code 1. See errors in the
(error)
> output.
> **
>
> Any hint of what this might mean ? Should I file a bug report ?

That means there was an error while writing out a spatial index, the file
size of the file holding the partially written out spatial index is smaller
than expected. Is this error reproducible on a different partition/file
system? Can you provide test data to reproduce?

Markus M
>
> Moritz
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] git howto get a single diff for a PR with multiple commits

[Markus Metz]

On Wed, Jun 5, 2019 at 7:51 AM Peter Petrik <
peter.pet...@lutraconsulting.co.uk> wrote:
>
> Hi,
>
> you can diff the branch agains the branch origin point (see eg
https://stackoverflow.com/questions/29810331/is-there-a-quick-way-to-git-diff-from-the-point-or-branch-origin).
For example if you PR 28 is on local branch pr_28, branched from master you
can do:
>
> git checkout pr_28
> git diff master...

that's exactly what I was looking for, thanks!

Markus M
>
> P.
>
>
>
> On Wed, Jun 5, 2019 at 6:44 AM Ken Mankoff  wrote:
>>
>>
>> On 2019-06-04 at 23:12 +02, Markus Metz 
wrote...
>> > I would like to speed up reviewing of a PR with multiple commits by
>> > reading a single diff file. How is this possible with git? [...] can
>> > git (on CLI on my local copies) also do this?
>>
>> I think git can diff any two commits. Do this by providing two commit
hashes instead of just one (which defaults to that v. the currently checked
out hash + any local changes).
>>
>>   -k.
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] git howto get a single diff for a PR with multiple commits

[Markus Metz]

Hi git gurus,

I would like to speed up reviewing of a PR with multiple commits by reading
a single diff file. How is this possible with git? An example is
https://github.com/OSGeo/grass/pull/28

Is this only possible in github looking at the "Files changed"? If github
can do this, can git (on CLI on my local copies) also do this?

Thanks,

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] instructions to update my local copy of trunk

[Markus Metz]

On Sun, Jun 2, 2019 at 11:12 PM Markus Neteler  wrote:
>
> On Sun, Jun 2, 2019 at 10:52 PM Panagiotis Mavrogiorgos
>  wrote:
> ...
> > Markus wrote:
> >> It there a way (magic flag?) to avoid this out-of-sync in the first
place?
> >> E.g., for the cronjobs I do not want to go there weekly and "git push"
> >> stuff around.
> >
> > Yes, but we need to know the contents of the cron scripts first.
>
> It simply follows the HowToGit on trac which leads to the mentioned
problem.
>
> What I am searching for is that the disk copy is the same as on the
> server.

Which version on the server? The orginal GRASS repo (
https://github.com/OSGeo/grass.git) or a fork? If the aim is to have an
unmodified local copy that is in sync with the original GRASS repo, you
don't need any other remote and it should be enough to pull.

> As it is a cronjob there will be no local changes. Still, some
> push magic seems to me missing from
> https://trac.osgeo.org/grass/wiki/HowToGit#Keepyourlocalsourcecodeuptodate

In this case (have a local copy in sync with
https://github.com/OSGeo/grass.git) push is not needed (no modifications
and no fork added as another remote).

Works ar t least for me.

Markus M
>
> thanks,
> Markus
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] instructions to update my local copy of trunk

[Markus Metz]

On Sun, Jun 2, 2019 at 8:59 PM Veronica Andreo  wrote:
>
> Hi devs
>
> Following instructions here:
https://trac.osgeo.org/grass/wiki/HowToGit#Keepyourlocalsourcecodeuptodate,
I get:
>
> [veroandreo@localhost grass7_trunk]$ git remote -v
> origin g...@github.com:veroandreo/grass.git (fetch)
> origin g...@github.com:veroandreo/grass.git (push)
> upstream g...@github.com:OSGeo/grass.git (fetch)
> upstream g...@github.com:OSGeo/grass.git (push)
>
> [veroandreo@localhost grass7_trunk]$ git fetch upstream
> remote: Enumerating objects: 242, done.
> remote: Counting objects: 100% (242/242), done.
> remote: Compressing objects: 100% (25/25), done.
> remote: Total 296 (delta 223), reused 233 (delta 217), pack-reused 54
> Receiving objects: 100% (296/296), 56.83 KiB | 1.18 MiB/s, done.
> Resolving deltas: 100% (226/226), completed with 105 local objects.
> From github.com:OSGeo/grass
>  * [new branch]  changelog_fix_msg -> upstream/changelog_fix_msg
>3797ccaaf..b300e66fc  master-> upstream/master
>1ca5e4e34..e1e13d782  releasebranch_7_4 -> upstream/releasebranch_7_4
>15f53e803..dcfce280b  releasebranch_7_6 -> upstream/releasebranch_7_6
>
> [veroandreo@localhost grass7_trunk]$ git branch -a
> * master
>   remotes/origin/master
>   remotes/origin/releasebranch_7_0
>   remotes/origin/releasebranch_7_2
>   remotes/origin/releasebranch_7_4
>   remotes/origin/releasebranch_7_6
>   remotes/upstream/changelog_fix_msg
>   remotes/upstream/master
>   remotes/upstream/releasebranch_7_0
>   remotes/upstream/releasebranch_7_2
>   remotes/upstream/releasebranch_7_4
>   remotes/upstream/releasebranch_7_6
>
> [veroandreo@localhost grass7_trunk]$ git rebase upstream/master
> First, rewinding head to replay your work on top of it...
> Fast-forwarded master to upstream/master.
>
> [veroandreo@localhost grass7_trunk]$ git status
> On branch master
> Your branch is ahead of 'origin/master' by 51 commits.
>   (use "git push" to publish your local commits)
>
> Why does my origin/master is now ahead by 51 commits, if I have not a
single local change?

Read carefully: "Your branch is ahead of 'origin/master' by 51 commits."
That means your local branch master is ahead of your fork at github:
"origin g...@github.com:veroandreo/grass.git"
Your local branch master is based on "upstream g...@github.com:OSGeo/grass.git",
i.e. the original repo.
You need to update your fork with something like
git push origin

Don't know if that helps,

Markus M

> I just want to get the new stuff from upstream and update my fork and
local copy of the source code (what I used to do with `svn update`).
>
> Can someone please provide full instructions [ideally in
https://trac.osgeo.org/grass/wiki/HowToGit#Keepyourlocalsourcecodeuptodate]?
>
> Thanks much in advance
> Vero
>
>
>
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Forestfrag WARNING: ZSTD compression error -14: Unsupported frame parameter

[Markus Metz]

On Tue, May 28, 2019 at 6:26 PM Paulo van Breugel 
wrote:
>
>
>
> On Tue, May 28, 2019 at 5:30 PM Vaclav Petras 
wrote:
>>
>>
>>
>> On Mon, May 27, 2019 at 3:25 PM Paulo van Breugel 
wrote:
>>>
>>> Hi Václav / devs
>>>
>>> I am running r.forestfrag in Windows 10, grass 7.6.2 Computation with a
moving window size of 21 (but at another computer same happens at size 9)
gives met the following error:
>>>
>>> C:\Users\brp\AppData\Roaming\GRASS7\addons/scripts/r.forestfrag.py
input=groenreclass@landschap output=AAtest7 size=21
>>> Step 1: Computing Pf values...
>>> Step 2: Computing Pff values...
>>
>>
>> Some computations happened, but then something went wrong.
>
>
> It goes wrong when with the moving window computation using r.mapcalc.
But when (e.g., with what moving window size) differs depending on the
computer and the size of the raster.
>
>>
>>>
>>> WARNING: ZSTD compression error -14: Unsupported frame parameter
>>> ERROR: Error uncompressing raster data for row 328 of

>>
>>
>> Something wrong with the libraries or the files (perhaps not enough
space on disk?).
>
>
> I guess something with the files, as it works with smaller moving windows
and it works in Linux.

This is probably a read error: data are not correctly read from disk. But
this should be reported as a read error.

A ZSTD compression error has been reported previously when a raster map was
compressed with ZSTD 1.3.x and on another system decompressed with ZSTD
1.3.y. Not sure if this applies here as well.

Markus M

>
>>
>>
>>>
>>> WARNING: No data base element files found
>>
>>
>> Probably, g.remove trying to clean something as the the script is
exiting. Likely harmless.
>
>
> Ah, ok
>
>>
>>>
>>> Traceback (most recent call last):
>>>   File "C:\Users\brp\AppData\Roaming\GRASS7\addons/scripts/
r.forestfrag.py", line 420, in  sys.exit(main(*gs.parser()))
>>>   File "C:\Users\brp\AppData\Roaming\GRASS7\addons/scripts/
r.forestfrag.py", line 290, in main ipl=ipl, tmpl4=expr1, tmpl5=expr2,
pff=pff)
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\raster.py", line
111, in mapcalc " with expression: %s") % e)
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\core.py", line 667,
in fatal error(msg)
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\core.py", line 651,
in errormessage(msg, flag='e')
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\core.py", line 584,
in message run_command("g.message", flags=flag, message=msg,
errors='ignore')
>>
>>
>> Here, it is g.message failing, so at this point nothing works.
>>
>>>
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\core.py", line 415,
in run_command ps = start_command(*args, **kwargs)
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\core.py", line 382,
in start_command
>>> return Popen(args, **popts)
>>>   File
"C:\OSGEO4~1\apps\grass\grass76\etc\python\grass\script\core.py", line 76,
in __init__ subprocess.Popen.__init__(self, args, **kwargs)
>>>   File "C:\OSGEO4~1\apps\Python27\lib\subprocess.py", line 390, in
__init__ errread, errwrite)
>>>   File "C:\OSGEO4~1\apps\Python27\lib\subprocess.py", line 640, in
_execute_child startupinfo)
>>> WindowsError: [Error 206] De bestandsnaam of -extensie is te lang
>>
>>
>> Too many open files? Or maybe just the error message happens to be too
long.
>>
>>>
>>> Any idea what may be the problem here?
>>
>>
>> Try smaller region, smaller window. Look into task manager. ...
>
>
> I did try a smaller region, this does seem to make a difference. I will
try if I can do this more systematically.
>
>>
>>
>> Best,
>> Vaclav
>>
>>>
>>>
>>> Greetings,
>>>
>>> Paulo
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] grass-addons on github

[Markus Metz]

The main advantage of having all grass-addons in one repo is that they are
easy to manage (for core devs).

The main disadvantage of having all grass-addons in one repo is that a
contributor concerned about his single addon needs to get the whole repo,
wasting disk space.

There are probably two different scenarios regarding contributions to
grass-addons:
1. a new addon: needs IMHO a PR, also if the contributor has write access
to the addons repo
2. modifications to an existing addon (bug fix, updated manual, new
functionality, etc): if the contributor has properly validated the changes,
the modifications could be pushed directly to the repo, without a PR. But
since even people (more or less) familiar with git have different opinions
on how to use git the correct way (as many correct ways as people having an
opinion on it), it is probably safer to go through a PR.

my0.2cents,

Markus M

On Fri, May 24, 2019 at 4:25 PM Stefan Blumentrath <
stefan.blumentr...@nina.no> wrote:
>
> Hi,
>
> Collecting addons in a central repo seems very valuable to me too, for
all the reasons Vacslav mentioned.
>
> I am no git expert either, but PRs should not be a big issue to do
(unless you are VERY productive). People could merge their own PRs, no?
Creating a PR, does not mean it has to be reviewed by another dev, right?
In my organization colleagues even usr PRs for repos where they are the
only contributor...
>
> I would argue having procedures as equal as possible between addons and
core is just beneficial. Less confusion, fewer guidelines to maimtain,
possibility to have CI before things are merged, and training for new devs
that evolve from addon-dev to core-dev...
>
> Cheers
> Stefan
>
>
> From: Anna Petrášová
> Sent: Friday 24 May 15:57
> Subject: Re: [GRASS-dev] grass-addons on github
> To: Martin Landa
> Cc: Paulo van Breugel, GRASS developers email list
>
>
>
>
> On Fri, May 24, 2019 at 3:55 AM Martin Landa 
wrote:
>
> Hi,
>
> pá 24. 5. 2019 v 8:48 odesílatel Paulo van Breugel
>  napsal:
> > I have read about the procedure for contributors to the main grass
repository. Question is, how are we going to deal with add-ons?
> >
> > Are we working with a central repository (OSGeo/grass-addons) and
follow the same protocol as for OSGEO/grass. If so, who will be responsible
for approving pull requests? An alternative more like the old situation is
that authors will be able to directly commit to the addon repository.
>
> in my opinion requesting PRs for `grass-addons` is maybe overkill. It
>
>
>
> If we don't care about the history and any mess in the grass-addons
repository, then yes, we don't need pull requests.
> But a lot of people who might be contributing there might not be familiar
with the peculiarities of git (since even most core grass devs including me
aren't), so eventually we will end up with a lot of mess, which somebody
will need to clean up. PR is a standard way to work on GitHub, so let's use
it. The same approach as for the main grass repo could be used.
>
>
> must be somehow discussed anyway. If we suggest direct commits it's
> important to avoid not needed 'merge from master' commits [1]. The
> workflow must be clear (rebase always) to avoid such situations. It
>
>
> I don't quite get how to use rebase yet, but that's the issue, it seems
that if you use it incorrectly, it can be dangerous.
>
>
> was not defined yet. Even suggested workflow related to the main
> repository is not clearly defined [2]. This must be improved in a near
> future.
>
> > Or should add-on authors maintain their own repositories, and will
there be a way to provide links to the authors repositories in a central
place?
>
>
> We did this with couple more complicated addons, we do internal
development in our git and then push it to the main repo when we want. I
like the idea of having all addons in one repository, then you can provide
the Windows binaries for them, that is also an incentive for contributers
to put it there (you get windows binary, hosting of manuals, simple
installation). But I get people want the distributed approach too.
>
> Anna
>
>
> Would be nice if g.extension (wingrass builds) supports distributed
> personal repos. I can imagine that it could be driven by a metadata
> file stored in central `grass-addons` repo. But someone need to
> implement it (g.extension, manual pages builds and wingrass builds).
> Would be cool.
>
> > With a central repository for all add-ons I guess it will be easier to
maintain an overview like https://grass.osgeo.org/grass76/manuals/addons/
and to create the windows binaries?
>
> Sure. But see my notes above.
>
> Ma
>
> [1] https://lists.osgeo.org/pipermail/grass-dev/2019-May/092663.html
> [2] https://trac.osgeo.org/grass/wiki/HowToGit
>
> --
> Martin Landa
> http://geo.fsv.cvut.cz/gwiki/Landa
> http://gismentors.cz/mentors/landa
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
>

[GRASS-dev] git: how to avoid bogus merge commits

[Markus Metz]

Hi git gurus,

when I modify my local copy and do git commit + git push, I sometimes get
an error:
"failed to push some refs"
and a hint:
"Updates were rejected because the remote contains work that you do not
have locally. This is usually caused by another repository pushing to the
same ref. You may want to first integrate the remote changes (e.g., 'git
pull ...') before pushing again."

OK, I do "git pull" and get confronted with a merge commit message.

After that, I do "git push" and commit a merge that does nothing.

Of course I could (should) do a "git pull" before a "git push", but I think
it is not uncommon that someone forgets the "git pull" before the "git
push". Therefore I would like to know how I can get rid of this bogus merge
commit that changes nothing but spams the history, and burns Travis CI,
e.g.  [1, 2].

Thanks,

Markus M

[1] https://travis-ci.com/OSGeo/grass/builds/112790153
[2] https://travis-ci.com/OSGeo/grass/builds/112567602
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

[Markus Metz]

On Mon, May 20, 2019 at 8:43 PM Vaclav Petras  wrote:
>
>
>
> On Mon, May 20, 2019 at 1:46 PM Markus Metz 
wrote:
>>
>>
>> On Mon, May 20, 2019 at 7:18 PM Vaclav Petras 
wrote:
>> >
>> > On Mon, May 20, 2019 at 11:57 AM Markus Metz <
markus.metz.gisw...@gmail.com> wrote:
>> >>
>> >> On Mon, May 20, 2019 at 5:39 PM Vaclav Petras 
wrote:
>> >> >
>> >> > Hi MarkusM and all,
>> >> >
>> >> > I'm trying to understand if the Segment_open() [1] function fills
with zeros or not. I don't think it does since it is calling G_malloc
(malloc) or Segment_format_nofill(). However, it is not completely clear to
me what is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].
>> >>
>> >> Segment_open() uses Segment_format_nofill() [1], if it can not use
the all-in-memory cache. The documentation has not been updated accordingly
(yet).
>> >
>> >
>> > Thank you. Just be to be sure: Now, if you want to ensure that it is
zero-filled across platform, you need to do it yourself using
Segment_put(), right?
>>
>> All modules that use the segment lib use Segment_put() anyway to load
data
>
>
> I'm asking about the case when I'm writing a new raster map. Let's say
with point binning (r.in.lidar/r.in.pdal) or with a simulation I want the
values to be initialized as zeros. I can use Segment_put(), but it seems
that what Segment_format() is for. Hence the question about
Segment_open_zero_fill() (as something more costly than Segment_open() but
cheaper than Segment_open()+Segment_put()).

In this case I would rather initialize with NULL values. IIUC,
Segment_get() will return all null bytes if lseek has been used and no data
have been written out yet. Note that the all-in-memory cache uses malloc,
not calloc (which it probably should), thus no initialization with zero
bytes in this case.

>>
>> > The exact role of lseek() here is still unclear to me (the hole and \0
bytes).
>> man lseek:
>>lseek() allows the file offset to be set beyond the end of the
file (but this does not change the size of the file).  If data is later
 written
>>at this point, subsequent reads of the data in the gap (a "hole")
return null bytes ('\0') until data is actually written into the gap.
>>
>> More info: disk space is allocated and file size increases only when
actual data are written into the gap
>
>
> Thanks. I have seen this, but the segment library is not using the null
bytes as zeros, or does it?

The segment lib does not interpret the bytes, it returns the bytes as is
(actually a void pointer). The module then casts it to whatever it is
supposed to be.
>
> > ...
> >> The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().
> > ...
>
> It seems that not all modules are doing this (e.g. r.cost). How big a
problem that is? I guess the result is a truncated raster without a warning
in case of "no space left on device".

Yes, the result would be either a truncated raster or earlier an error on
reading. r.cost as other affected modules need to be updated to check the
return value of Segment_[get|put]() and exit with fatal error on failure.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

[Markus Metz]

On Mon, May 20, 2019 at 7:18 PM Vaclav Petras  wrote:
>
>
>
> On Mon, May 20, 2019 at 11:57 AM Markus Metz <
markus.metz.gisw...@gmail.com> wrote:
>>
>>
>>
>> On Mon, May 20, 2019 at 5:39 PM Vaclav Petras 
wrote:
>> >
>> > Hi MarkusM and all,
>> >
>> > I'm trying to understand if the Segment_open() [1] function fills with
zeros or not. I don't think it does since it is calling G_malloc (malloc)
or Segment_format_nofill(). However, it is not completely clear to me what
is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].
>>
>> Segment_open() uses Segment_format_nofill() [1], if it can not use the
all-in-memory cache. The documentation has not been updated accordingly
(yet).
>
>
> Thank you. Just be to be sure: Now, if you want to ensure that it is
zero-filled across platform, you need to do it yourself using
Segment_put(), right?

All modules that use the segment lib use Segment_put() anyway to load data

> (Assuming you want to use Segment_open().)

or Segment_format_nofill() + Segment_init()

> The exact role of lseek() here is still unclear to me (the hole and \0
bytes).
man lseek:
   lseek() allows the file offset to be set beyond the end of the file
(but this does not change the size of the file).  If data is later  written
   at this point, subsequent reads of the data in the gap (a "hole")
return null bytes ('\0') until data is actually written into the gap.

More info: disk space is allocated and file size increases only when actual
data are written into the gap

>
> Would Segment_open_zero_fill() make sense, i.e. do you know if
Segment_fill() faster then Segment_put()?

Segment_put() is used anyway to load the actual data, therefore the
question if a new Segment_open_zero_fill() is an alternative to
Segment_put() is invalid because you need to use Segment_put() in any case
to populate the Segment structure with meaningful data.

Markus M
>
>>
>> The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().
>
>
> Makes sense. I'll document that as well.
>
>>
>>
>> HTH,
>>
>> Markus M
>>
>> [1] https://github.com/OSGeo/grass/blob/master/lib/segment/open.c#L89
>>
>> >
>> > Markus, can you please clarify that for me? I will then update the
documentation with whatever is needed.
>> >
>> > Thanks,
>> > Vaclav
>> >
>> > [1]
https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
>> > [2]
https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
>> > [3] https://trac.osgeo.org/grass/changeset/73268
>> > [4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

[Markus Metz]

On Mon, May 20, 2019 at 5:39 PM Vaclav Petras  wrote:
>
> Hi MarkusM and all,
>
> I'm trying to understand if the Segment_open() [1] function fills with
zeros or not. I don't think it does since it is calling G_malloc (malloc)
or Segment_format_nofill(). However, it is not completely clear to me what
is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].

Segment_open() uses Segment_format_nofill() [1], if it can not use the
all-in-memory cache. The documentation has not been updated accordingly
(yet). The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().

HTH,

Markus M

[1] https://github.com/OSGeo/grass/blob/master/lib/segment/open.c#L89

>
> Markus, can you please clarify that for me? I will then update the
documentation with whatever is needed.
>
> Thanks,
> Vaclav
>
> [1]
https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
> [2]
https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
> [3] https://trac.osgeo.org/grass/changeset/73268
> [4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] svn/trac -> git/github migration plan

[Markus Metz]

On Sun, May 19, 2019 at 5:24 PM Vaclav Petras  wrote:
>
>
>
> On Sat, May 18, 2019 at 12:11 PM Markus Neteler  wrote:
>>
>>
>> On Sat, May 18, 2019 at 5:59 PM Panagiotis Mavrogiorgos
>>  wrote:
>> >
>> > I would suggest that:
>> >
>> > - even core devs fork the main repo
>> > - "origin" is the personal remote GRASS repository (e.g. in my case
https://github.com/pmav99/grass)
>> > - everyone adds the main GRASS repository as a secondary remote (e.g.
"upstream")
>>
>> yes, it sounds good to me (while my opinion only counts 0.02 cents here).
>>
>> > This way:
>> >
>> > 1. You always push to "origin" and you create a Pull Request from the
Github UI
>> > 2. To get updates you always pull from "upstream"
>> > 3. You always rebase your code to "upstream/master".
>> > 4. You don't need separate instructions for non core-devs.
>>
>> Updated in a similar way to
>>
>> https://trac.osgeo.org/grass/wiki/HowToGit#Workflowforcoredevelopers
>
>
>
> Hi, any opinions on what should be the primary and secondary remote? The
the fork or upstream? As far as I understood, Bas is suggesting adding the
fork, Panos adding the upstream. There does not seem to be that much
difference, but the commands are different, so we need to decide for the
instructions. GitHub help suggests cloning fork and adding upstream.
Syncing to upstream seems more straight forward when cloning upstream and
adding to the fork...

GDAL [0] suggests to clone upstream, then add your own fork. For those who
don't have write access to upstream it does not matter IMHO. For those who
do have write access to upstream, it might be safer to have origin refer to
the personal fork. Ideally, you would set up the local repo such that you
pull from upstream and push to your fork.

my 0.2c (trying to learn fast)

Markus M

[0]
https://github.com/OSGeo/gdal/blob/master/CONTRIBUTING.md#initiate-your-work-repository
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Which is the correct WIND convention for creating a new mapset?

[Markus Metz]

On Thu, May 9, 2019 at 10:01 AM Panagiotis Mavrogiorgos 
wrote:
>
> Thank you markus
>
> On Wed, May 8, 2019 at 10:54 PM Markus Metz 
wrote:
>>
>> It depends. g.mapset changes the current mapset (and location and
database). It might also create a new mapset in an existing location, but
it always switches. If that switch is not desired, GISRC must be adjusted
to the original GRASS session. G_make_mapset() is a C function and should
be used by C modules. Of course you could use it with pygrass, but pygrass
initializes the GRASS C libraries, and you need to take care that
initializations are properly updated when changing mapsets (and locations).
Creating a new mapset in an existing location is not difficult: create the
folder and copy DEFAULT_WIND from PERMANENT to WIND in the new mapset.
Maybe create a new function create_mapset() in lib/python/script/utils.py?
>
>
> Generally speaking, I think it is important that the API comes first.
Once some functionality has been implemented in a library it can then be
exposed to the end users in multiple ways (e.g. CLI, REST, whatever). Among
other things, by implementing the API first, it is much easier to test the
code + you follow DRY.
>
> This is why I mentioned "G_make_mapset()" which seems to already be
implementing this functionality and which, as you mentioned, is exposed to
Python via pygrass. The idea is that if in the future the process of
creating a mapset becomes more complicated, then we would only have to
update "G_make_mapset()". That being said I am not familiar enough with the
initialization process so I can't really argue if a pure python
implementation is needed or not.

In this particular case I think a Python implementation would help because
1) it is really simple to create a new mapset in any existing location, 2)
the overhead of using pygrass is not needed in this case: most important
developer's rule: keep it as simple as possible.
>
> Now to become more specific, as an API user, I think would find it
strange that calling the "create_mapset()" function also changes the
current mapset. I like it when things are explicit + it makes it easier to
document and test. As far as implementing "create_mapset()" in
lib/python/script/utils.py, why not move the relevant functions from
"gui/wxpython/startup/utils.py" and have the GUI code import them from the
"grass" lib?

the suggested new function "create_mapset()" should only create a new
mapset, check if it succeeded in doing so, and not change to the new
mapset. The idea is that any python code that wants to create a new mapset
does not use its own method but instead uses this new function from
lib/python/script/utils.py.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Which is the correct WIND convention for creating a new mapset?

[Markus Metz]

On Wed, May 8, 2019 at 10:22 AM Panagiotis Mavrogiorgos 
wrote:
>
>
>
> On Tue, May 7, 2019 at 11:03 PM Markus Metz 
wrote:
>>
>>
>>
>> On Tue, May 7, 2019 at 10:20 AM Panagiotis Mavrogiorgos 
wrote:
>> >
>> >
>> >
>> > On Mon, May 6, 2019 at 9:38 PM Markus Metz <
markus.metz.gisw...@gmail.com> wrote:
>> >>
>> >>
>> >>
>> >> On Mon, May 6, 2019 at 6:21 PM Panagiotis Mavrogiorgos <
pma...@gmail.com> wrote:
>> >> >
>> >> > Hello all,
>> >> >
>> >> > I am a bit confused WRT what the expected WIND should be when you
create a new mapset.
>> >>
>> >> > [...]
>> >>
>> >> > Is there an established convention?
>> >>
>> >> IMHO, new mapsets should use DEFAULT_WIND from PERMANENT.
>> >
>> >
>> > Thank you Markus, that's what I expected, too.
>> >
>> >>
>> >> But the current region of a new mapset is often adjusted, therefore
it does not really matter if DEFAULT_WIND or WIND is used.
>> >
>> >
>> > I don't disagree, but still, I don't see what's the benefit of not
using the same convention everywhere. Furthermore, I truly don't understand
why 4 different implementations are needed.
>>
>> True, g.mapset -c should be used whenever possible.
>
>
> I would argue that it is G_make_mapset() that should be used whenever
possible :)
> (which of course, is being used by g.mapset -c, too)

It depends. g.mapset changes the current mapset (and location and
database). It might also create a new mapset in an existing location, but
it always switches. If that switch is not desired, GISRC must be adjusted
to the original GRASS session. G_make_mapset() is a C function and should
be used by C modules. Of course you could use it with pygrass, but pygrass
initializes the GRASS C libraries, and you need to take care that
initializations are properly updated when changing mapsets (and locations).
Creating a new mapset in an existing location is not difficult: create the
folder and copy DEFAULT_WIND from PERMANENT to WIND in the new mapset.
Maybe create a new function create_mapset() in lib/python/script/utils.py?

>>> I adjusted the wxGUI startup wizard and the tests in trunk r74472,3
(the tests should use g.mapset -c, not yet implemented)
>
>
> I think that in the nc_spm_full_v2alpha dataset WIND and DEFAULT_WIND
differ, so 74473 might break any tests that don't explicitly set the region
themselves (which of course they should). We will see.

IMHO, tests need to set the region themselves, just as users. The current
region is such a fundamental concept of GRASS (raster processing) that I
regard it as a mistake if the current region is not explicitly set to
actual demands.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] v.to.db: next/prev line

[Markus Metz]

On Wed, May 8, 2019 at 2:34 PM Moritz Lennert 
wrote:
>
> On 8/05/19 11:44, Martin Landa wrote:
> > Hi,
> >
> > st 8. 5. 2019 v 11:31 odesílatel Moritz Lennert
> >  napsal:
> >> I don't think we have a tool for this currently. However, I'm not sure
that right and left are relevant terms for this case. Also: what do you do
in the case when three lines meet at a node. What would be the
'next_left_edge line' ?
> >
> > right, these terms comes from TopoGeo model based described in ISO
13249-3:2006.
> >
> > See eg.
> >
> > http://freegis.fsv.cvut.cz/gwiki/PostGIS_Topology
> >
https://trac.osgeo.org/postgis/attachment/ticket/3046/error_in_getRingedge.png
>
> Right. That does make sens. I didn't think of left and right in that way.
>
> I don't know what use cases are for this feature, compared to the ones I
> listed in my previous mail...

If I understand all this correctly, there is a next_left_edge,
next_right_edge,  previous_left_edge, previous_right_edge. Left would be
equivalent to going CW from the current node, right would be going CCW from
the current node. Next would be the end node, previous would be the start
node.

GRASS example (different from PostGIS terminology): if there are no other
lines connected at the start node of line 1, previous_left_edge =
next_edge_edge = 1, conforming to the GRASS topology model. The
corresponding function to get the next/previous line is
dig_angle_next_line().

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Which is the correct WIND convention for creating a new mapset?

[Markus Metz]

On Tue, May 7, 2019 at 10:20 AM Panagiotis Mavrogiorgos 
wrote:
>
>
>
> On Mon, May 6, 2019 at 9:38 PM Markus Metz 
wrote:
>>
>>
>>
>> On Mon, May 6, 2019 at 6:21 PM Panagiotis Mavrogiorgos 
wrote:
>> >
>> > Hello all,
>> >
>> > I am a bit confused WRT what the expected WIND should be when you
create a new mapset.
>>
>> > [...]
>>
>> > Is there an established convention?
>>
>> IMHO, new mapsets should use DEFAULT_WIND from PERMANENT.
>
>
> Thank you Markus, that's what I expected, too.
>
>>
>> But the current region of a new mapset is often adjusted, therefore it
does not really matter if DEFAULT_WIND or WIND is used.
>
>
> I don't disagree, but still, I don't see what's the benefit of not using
the same convention everywhere. Furthermore, I truly don't understand why 4
different implementations are needed.

True, g.mapset -c should be used whenever possible. I adjusted the wxGUI
startup wizard and the tests in trunk r74472,3 (the tests should use
g.mapset -c, not yet implemented)

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Which is the correct WIND convention for creating a new mapset?

[Markus Metz]

On Mon, May 6, 2019 at 6:21 PM Panagiotis Mavrogiorgos 
wrote:
>
> Hello all,
>
> I am a bit confused WRT what the expected WIND should be when you create
a new mapset.
>
> Some functions copy DEFAULT_WIND from PERMANENT. E.g.:
>
> lib/gis/make_mapset.c
> pygrass.gis.make_mapset()
> the grass executable
>
> While others copy WIND from PERMANENT. E.g.
>
> the tests
> gui.wxpython.startup.create_mapset()
>
> Is there an established convention?

IMHO, new mapsets should use DEFAULT_WIND from PERMANENT. But the current
region of a new mapset is often adjusted, therefore it does not really
matter if DEFAULT_WIND or WIND is used.

my2c

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-SVN] r74459 - grass/trunk/lib/raster - winGRASS77svn compilation fails

[Markus Metz]

On Sun, May 5, 2019 at 5:17 PM Helmut Kudrnovsky  wrote:
>
> Hi,
>
> looking into winGRASS77svn compilation log:
>
> https://wingrass.fsv.cvut.cz/grass77/x86_64/logs/log-r74459-14/error.log
>
> #
> GRASS GIS 7.7.svn r74459M compilation log
> --
> Started compilation: Sat May  4 15:56:01 2019
> --
> Errors in:
> /c/msys64/usr/src/grass_trunk/lib/raster
> /c/msys64/usr/src/grass_trunk/lib/display
> [...]
> #
>
> compilation fails at r74459M

should be fixed in r74466, an alternative to fsync on MS Windows is
suggested but not implemented in r74467.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Benchmark the overhead of calling GRASS modules

[Markus Metz]

Hi Panos,

IMHO the overhead of calling GRASS modules is insignificant because it is
in the range of milliseconds. I am much more concerned whether executing a
GRASS module takes days or hours or minutes.

Also note that the base of GRASS are C modules using the GRASS C library.
GRASS python modules usually call GRASS C modules (or other GRASS python
modules calling GRASS C modules). The first thing a GRASS Python module
does is calling the GRASS C module g.parser, after that it calls (in the
end) some other GRASS C modules. That means it is not straightforward to
test the overhead of calling GRASS Python modules vs calling GRASS C
modules because it is really GRASS Python + C modules vs GRASS C modules
only. And the overhead is insignificant (not measurable) compared to actual
execution time for larger datasets/regions.

Markus M


On Mon, Apr 29, 2019 at 8:49 AM Panagiotis Mavrogiorgos 
wrote:

> Hello all
>
> You might find it easier to read the following text i
> n a gist
> 
>
> ## Introduction
>
> I was trying to write a decorator/contextmanager that would temporary
> change the
> computational region, but while using it I noticed that there was some
> overhead the root
> of which seemed to be the usage of the GRASS modules. So in order to
> quantify this
> I wrote a small benchmark that tries to measure the overhead of calling a
> GRASS Module.
> This is what I found.
>
> ## tl;dr
>
> Calling a GRASS module incurs a constant but measurable overhead which, in
> certain
> cases, e.g. when writing a module that uses a lot of the other modules,
> can quickly
> add up to a significant quantity.
>
> ## Disclaimer
>
> If you try to run the benchmark on your own PC, the actual timings you
> will get will
> probably be different. The differences might be caused by having:
>
> - a stronger/weaker CPU
> - faster/slower hard disk.
> - using different Python version
> - using different compilation flags
>
> Still, I think that some of the findings are reproducible.
>
> For reference, I used:
>
> - OS: Linux 5.0.9
> - CPU: Intel i7 6700HQ
> - Disk type: SSD
> - Python: 3.7
> - `CFLAGS='-O2 -fPIC -march=native -std=gnu99'`
>
> ## Demonstration
>
> The easiest way to demonstrate the performance difference between using a
> GRASS module
> vs using the GRASS API is to run the following snippets.
>
> Both of them do exactly the same thing, i.e. retrieve the current region
> settings 10
> times in a row.  The performance difference is huge though. On my laptop,
> the first one
> needs 0.36 seconds while the second one needs just 0.00038 seconds. That's
> almost
> a 1000x difference...
>
> ``` python
> import time
> import grass.script as gscript
>
> start = time.time()
> for i in range(10):
> region = gscript.parse_command("g.region", flags="g")
> end = time.time()
> total = end - start
>
> print("Total time: %s" % total)
> ```
>
> vs
>
> ``` python
> import time
> from grass.pygrass.gis.region import Region
>
> start = time.time()
> for i in range(10):
> region = Region()
> end = time.time()
> total = end - start
>
> print("Total time: %s" % total)
> ```
>
> ## How much is the overhead exactly?
>
> In order to measure the actual overhead of calling a GRASS module, I
> created two new
> GRASS modules that all they do is parse the command line arguments and
> measured how much
> time is needed for their execution. The first module is [`r.simple`]() and
> is
> implemented in Python while the other one is [`r.simple.c`]() and is
> implemented in C.
> The timings are in msec and the benchmark was executed using Python 3.7
>
> | call method| r.simple | r.simple.c |
> ||::|:--:|
> | pygrass.module.Module  |   85.9   |66.5|
> | pygrass.module.Module.shortcut |   85.5   |66.9|
> | grass.script.run_command   |   41.3   |30.5|
> | subprocess.check_call  |   41.8   |30.3|
>
> As we can see, `gsrcipt.run_command` and `subprocess` give more or less a
> identical
> results, which is to be expected since `run_command` + friends are just a
> thin wrapper
> around `subprocess`.  Similarly `shortcuts` has the same overhead as
> "vanila"
> `pygrass.Module`.  Nevertheless, it is obvious that `pygrass` is roughly
> 2x times slower
> than `grass.script` (but more about that later).
>
> As far as C vs Python goes, on my computer modules implemented in C seem
> to be 25% faster than their
> Python counterparts.  Nevertheless, a 40 msec startup time doesn't seem
> extraordinary
> for a Python script, while 30 msec feels rather large for a CLI
> application implemented
> in C.
>
> ## Where is all that time being spent?
>
> ### C Modules
>
> Unfortunately, I am not familiar enough with the GRASS internals to easily
> check what is
> going on and I didn't have the time to try to profile the 

Re: [GRASS-dev] Addons manuals: logo and CSS file missing

[Markus Metz]

On Thu, Apr 25, 2019 at 7:42 AM Martin Landa  wrote:
>
> Hi,
>
> st 24. 4. 2019 v 16:28 odesílatel Markus Neteler 
napsal:
> > Could you please verify?
>
> yes, the reason is that G76 is not compiled on server due to a new
> required dependency:
>
> configure: error: *** Unable to locate zstd includes.

zstd is not a required dependency, you can disable it with --with-zstd=no

Markus M

>
> I will fix it ASAP. Ma
>
> --
> Martin Landa
> http://geo.fsv.cvut.cz/gwiki/Landa
> http://gismentors.cz/mentors/landa
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Addons manuals: logo and CSS file missing

[Markus Metz]

On Wed, Apr 24, 2019 at 4:28 PM Markus Neteler  wrote:
>
> Hi Martin,
>
> since the switch to use G76 as the basis for the addon manual pages,
> the two files
> grassdocs.css
> grass_logo.png
> are no longer put onto the server.
>
> I suspect that they are not picked up on the build server:
>
> grass-addons/tools/addons/grass-addons-build.sh" line 38 of 59
> cp ${SRC}/grass${1}${2}_release/${DIST}/docs/html/grassdocs.css
$HTMLDIR/
> cp ${SRC}/grass${1}${2}_release/${DIST}/docs/html/grass_logo.png
$HTMLDIR/

The reason is probably that the installation directory has been changed to
include the bugfix release number in r74038, the culprit is probably the
change in configure.in:L219 [0] which needs to be reverted.

Markus M

[0]
https://trac.osgeo.org/grass/changeset/74038/grass/branches/releasebranch_7_6/configure.in

>
> Could you please verify?
>
> thanks
> Markus
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GSoC Proposal: Python package for topology tools

[Markus Metz]

On Mon, Apr 22, 2019 at 4:45 AM Facundo Ferrín 
wrote:
>
> Hello!
>
> I don't know if I'm expressing myself well. The use of GRASS is
command-line, in the sense:
>
> rass run --mapset=/some/directory/grassdata/ncspm/practice1 r.lake
elevation=some/file.tiff
>
>
> What I propose is the use of a package to import from any Python script.
For example, read a TIFF with GDAL or GeoPandas, perform some operations
like removing some data and finally simplify the polygon.

GRASS is essentially a toolset consisting of several hundred modules which
are called on the command line or in scripts. If you want to use this
functionality, you need to call GRASS modules as done in e.g.
https://trac.osgeo.org/grass/browser/grass/trunk/scripts
>
> GRASS is something like GDAL's option to create polygons from images:
>
> https://gdal.org/gdal_polygonize.html

The GRASS equivalent is 'r.to.vect type=area'
>
> My idea, or my need, is to use GRASS functions programmatically, as in
this case.
>
>
https://pcjericks.github.io/py-gdalogr-cookbook/raster_layers.html#polygonize-a-raster-band

Convert an OGR File to a Raster
more generic: convert a vector to a raster
The GRASS equivalent is 'v.to.rast'

Clip a GeoTiff with Shapefile
more generic: clip a raster to vector polygons
In GRASS: convert to vector polygons to raster with 'v.to.rast', then
create a MASK with 'r.mask'

Calculate zonal statistics
more precisely here: get raster statistics for each vector polygon
The GRASS equivalent is 'v.rast.stats'

Raster to vector line
In GRASS: use 'r.reclass' or 'r.mapcalc' to select pixel values that need
to be converted to vector lines, then use r.to.vect type=line. BTW, the
green lines in the example are wrong because they don#t go through the
pixel centers.

Create raster from array
The GRASS equivalent is 'r.in.ascii'

Create least cost path
The GRASS equivalent is 'r.cost' + 'r.path'

Replace No Data Value of Raster with new value
The GRASS equivalent is 'r.mapcalc' using "if(isnull(rastermap),
, rastermap)"

>
> Is that among your ideas for the future? If that's the case and you're
interested, let me know. If so, you could make some prototype use to start
migrating the functionalities.

All this can be easily implemented as a GRASS GIS addon in a Python script
with much less code than in your example. Please have a look at
https://trac.osgeo.org/grass/browser/grass/trunk/scripts
to get started.
Alternatively, a QGIS plugin using GRASS GIS functionality would be an
option (QGIS has a GRASS GIS interface that can be used for inspiration).

my2c,
Markus M

>
> Cheers!
>
>
> On 10/04/2019 16:47, Vaclav Petras wrote:
>
> Hello Facundo,
>
> Using the GRASS GIS algorithms in various settings is certainly something
we are aiming for. However, as you probably understood from the
conversation here, it is not clear what exactly you are trying to achieve
considering the already available ways (APIs if you will). Also, to
evaluate your proposal, we need to test your skills for this particular
project. Therefore, I suggest you 1) implement a prototype of what you have
in mind, 2) compare it with the existing approaches (see e.g. email from
Markus M.), and 3) identify what is missing in the existing approaches and
what would you need to add in order to make you prototype work.
>
> Alternatively, or ideally in addition to the above, you can implement
couple of the test and training tasks from the proposal already linked by
Luca:
>
>
https://trac.osgeo.org/grass/wiki/GSoC/2019#Neweasy-to-useCLIandAPIforGRASSGIS
>
> Best,
> Vaclav
>
> On Thu, Mar 28, 2019 at 4:13 AM Markus Metz 
wrote:
>>
>>
>>
>> On Thu, Mar 28, 2019 at 9:02 AM Maris Nartiss 
wrote:
>> >
>> > Hello Facundo,
>> > the easiest way would be moving functions of v.generalize into a
>> > library (e.g. grass_generalize) and thus make available for calling
>> > via ctypes.
>> > In the past I have had a good success manipulating GRASS vectors via
>> > ctypes. It takes more skill than a plain Python implementation but it
>> > is easier than a full blown C code and faster than pure Python one.
>> >
>> > Māris.
>> >
>> > ceturtd., 2019. g. 28. marts, plkst. 03:13 — lietotājs Facundo Ferrin
>> > () rakstīja:
>> > >
>> > > Hi Luca!
>> > >
>> > > Thanks for replying! In my job, there were things we had to do
programmatically. For example, to manipulate geometries that reach the
backend from a GeoJSON we use tools like these:
>> > >
>> > >
https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt
>> > >
>> > > However, polygon simplification does not work very well because it
does not take topology into account. My idea w

Re: [GRASS-dev] [GRASS-user] Nice opportunity to show off GRASS functionality and topological vector format ?

[Markus Metz]

On Thu, Apr 18, 2019 at 6:38 PM Markus Metz 
wrote:
>
>
>
> On Tue, Apr 16, 2019 at 11:07 AM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
> >
> > Hi,
> >
> > If anyone has some time and wants to show off some GRASS GIS power:
> >
> >
https://www.ecmwf.int/en/learning/workshops/ecmwf-summer-weather-code-2019
> >
> > "The Summer of Weather Code(ESoWC) programme by the European Centre for
> > Medium-Range Weather Forecasts (ECMWF) is a collabrative online
> > programme to promote the development of weather-related open-source
> > software."
> >
> > More specificially:
> >
> > https://github.com/esowc/challenges_2019
> >
> > And even more, I think this challenge is a fairly low hanging fruit
> > using GRASS GIS tools:
> >
> > https://github.com/esowc/challenges_2019/issues/3
>
> I agree, this reads like a low-hanging fruit. I could submit an abstract
until Sunday 21 April with a fairly detailed workflow, but I will most
probably not have the time to write such a tool: apparently they would like
to have a custom QGIS plugin.
>
> Markus M
> >
> >
> > APPLICATION DEADLINE: SUNDAY, 21 APRIL 23:59 GMT

The abstract turned out to be a proposal where you have to follow a
template (much longer than anticipated). I managed to submit a proposal 2
hours before the deadline :-)

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS-user] Nice opportunity to show off GRASS functionality and topological vector format ?

[Markus Metz]

On Tue, Apr 16, 2019 at 11:07 AM Moritz Lennert <
mlenn...@club.worldonline.be> wrote:
>
> Hi,
>
> If anyone has some time and wants to show off some GRASS GIS power:
>
> https://www.ecmwf.int/en/learning/workshops/ecmwf-summer-weather-code-2019
>
> "The Summer of Weather Code(ESoWC) programme by the European Centre for
> Medium-Range Weather Forecasts (ECMWF) is a collabrative online
> programme to promote the development of weather-related open-source
> software."
>
> More specificially:
>
> https://github.com/esowc/challenges_2019
>
> And even more, I think this challenge is a fairly low hanging fruit
> using GRASS GIS tools:
>
> https://github.com/esowc/challenges_2019/issues/3

I agree, this reads like a low-hanging fruit. I could submit an abstract
until Sunday 21 April with a fairly detailed workflow, but I will most
probably not have the time to write such a tool: apparently they would like
to have a custom QGIS plugin.

Markus M
>
>
> APPLICATION DEADLINE: SUNDAY, 21 APRIL 23:59 GMT
>
>
> Moritz
> ___
> grass-user mailing list
> grass-u...@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] FOSS4G 2019: Content for GRASS GIS project talk?

[Markus Metz]

On Mon, Apr 15, 2019 at 8:50 AM Markus Neteler  wrote:
>
> Hi,
>
> Thanks, all collected as bullet list items (so far) here:
> https://yourpart.eu/p/grassgis_at_foss4g2019
>
> The deadline is later today, so pls (all) let flow in your suggestions.

I have added support for PROJ 6 + GDAL 2.5, I think Even will give a talk
on PROJ + GDAL as well which we could refer to.

Markus M
>
> thanks
> Markus
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Fwd: [gdal-dev] shapelib 1.5.0 release

[Markus Metz]

On Sat, Apr 13, 2019 at 7:24 PM Markus Neteler  wrote:
>
> Hi devs,
>
> we still have an older copy of shapelib in
> lib/external/shapelib/
>
> Is that still needed/wanted for the upcoming 7.8.0 release?
> If yes, to be updated?

GRASS is only using the dbf driver of shapelib. Updating is not not needed
but would be nice to keep in sync with upstream. I can do it in the next
few days.

Markus M

>
> Markus
>
> -- Forwarded message -
> From: Even Rouault 
> Date: Thu, Feb 28, 2019 at 5:14 PM
> Subject: [gdal-dev] shapelib 1.5.0 release
> To: 
> Cc: 
>
>
> Hi,
>
> (CC'ing gdal-dev as well, since shape...@lists.osgeo.org has likely lower
> audience)
>
> I've released shapelib 1.5.0
>
> The main change is the removal of the 'shpproj' contrib utility and thus
PROJ
> dependency.
>
> The tarballs are available at:
> http://download.osgeo.org/shapelib/shapelib-1.5.0.tar.gz
> http://download.osgeo.org/shapelib/shapelib-1.5.0.zip
>
> The log of changes since 1.4.1 is:
>
> 2019-02-28  Even Rouault  
>
> * shpopen.c: resync with GDAL internal shapelib to avoid being
dependent
> on correctness of file size field in .shp. Fixes
>
https://lists.osgeo.org/pipermail/gdal-dev/2018-October/049218.html
>
> 2019-02-28  Even Rouault  
>
> * contrib/shpgeo.h/.c: Remove PROJ.4 dependency and functionality,
>  causing removal of SHPProject(), SHPSetProjection() and
> SHPFreeProjection()
> * contrib/shpproj.c: removed
>
> 2018-08-16  Even Rouault  
>
> * shpopen.c, dbfopen.c, shptree.c, sbnsearch.c: resyc with GDAL
> internal shapelib. Mostly to allow building those files as C++
> without warning. Also add FTDate entry in DBFFieldType
> (see https://github.com/OSGeo/gdal/pull/308). And some other
> code cleanups
>
> 2018-08-16  Even Rouault  
>
> * dbfopen.c: fix a bug where the end of file character was
> written on top of the first character of the first field name
> when deleting a field on a .dbf without records.
> Fixes https://github.com/OSGeo/gdal/issues/863
>
> 2018-06-15  Even Rouault  
>
> * safileio.c: remove duplicate test. Patch by Jaroslav Fojtik.
> Fixes http://bugzilla.maptools.org/show_bug.cgi?id=2744
>
> Even
>
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
> ___
> gdal-dev mailing list
> gdal-...@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] include a 'nearest' option in r.walk ?

[Markus Metz]

Hi Moritz,

On Wed, Apr 10, 2019 at 4:16 PM Moritz Lennert 
wrote:
>
> Hi everyone,
>
> We are looking for a way to transform the r.walk output into a nearest
> neighbor by cost map, i.e. something like the 'nearest' output of r.cost.
>
> Does anyone have an idea of how this could be done with the existing
tools ?
>
> A related question @Markus: As you know r.cost & r.walk the best, how
> difficult would it be to include a 'nearest' option also in r.walk ?

not so difficult: done in trunk r74367.

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GSoC Proposal: Python package for topology tools

[Markus Metz]

On Thu, Mar 28, 2019 at 9:02 AM Maris Nartiss  wrote:
>
> Hello Facundo,
> the easiest way would be moving functions of v.generalize into a
> library (e.g. grass_generalize) and thus make available for calling
> via ctypes.
> In the past I have had a good success manipulating GRASS vectors via
> ctypes. It takes more skill than a plain Python implementation but it
> is easier than a full blown C code and faster than pure Python one.
>
> Māris.
>
> ceturtd., 2019. g. 28. marts, plkst. 03:13 — lietotājs Facundo Ferrin
> () rakstīja:
> >
> > Hi Luca!
> >
> > Thanks for replying! In my job, there were things we had to do
programmatically. For example, to manipulate geometries that reach the
backend from a GeoJSON we use tools like these:
> >
> >
https://pcjericks.github.io/py-gdalogr-cookbook/geometry.html#create-geometry-from-wkt
> >
> > However, polygon simplification does not work very well because it does
not take topology into account. My idea was to port part of the GRASS
algorithms to be able to use them without needing the graphical interface
or command line, but only importing a library in a Python script.

In this particular case, the core of the corresponding python script would
be three lines (import, simplify, export):

-->
import grass.script as grass

grass.run_command('v.in.ogr', ...)
grass.run_command('v.generalize', ...)
grass.run_command('v.out.ogr', ...)
<--

The import step with v.in.ogr is needed because the vector to be simplified
must be a native GRASS vector with topology.

How does your proposal differ from the QGIS-GRASS interface?

Markus M
> >
> > Is it something that you have in mind to do or that might be useful to
you?
> >
> >
> > El jue., 28 de mar. de 2019 a la(s) 00:32, Luca Delucchi (
lucadel...@gmail.com) escribió:
> >>
> >> On Tue, 26 Mar 2019 at 03:11, Facundo Ferrin 
wrote:
> >> >
> >> > Hi there!
> >>
> >> Hi Facundo,
> >> >
> >> >
> >> > My name is Facundo Ferrin. I am a nuclear engineer who is taking a
master in Computer Vision in Barcelona, and finally I found my opportunity
to contribute to OSGeo by applying two things that I really like: Python
and Backend development . I do not know exactly what I should write in this
first email, so I'll start by listing the projects I'm interested in.
> >> >
> >> > I'm working in a company that is developing a platform for precision
agriculture called Auravant (https://www.auravant.com/). I work as a
backend developer and data analyst and I use daily almost every tool that
you post in the ideas: GeoServer, PostGIS, QGis. I'm also porting a tool
for polygon simplification called topoJSON (
https://github.com/fferrin/topojson).
> >> >
> >> > ---
> >> > MY MAIN IDEA is to start porting GRASS tools into a python package
that can be used in other projects (beyond the client to use by command
line). I don't know if it's something you have in mind but for offline and
automated analysis it would be very useful. I particularly had problems
when I tried to simplify geometries since the geometry of polygons was not
taken into account.
> >> > ---
> >>
> >> Your idea is not clear to me, there are already two Python library to
> >> work with GRASS. you can find some ideas in the proposal page
> >> https://trac.osgeo.org/grass/wiki/GSoC/2019 (for example
> >> Neweasy-to-useCLIandAPIforGRASSGIS) and
> >> https://trac.osgeo.org/grass/wiki/GSoC/2018 (Improve GRASS integration
> >> in QGIS 3)
> >>
> >> > Hope to hear from you soon!
> >> >
> >>
> >> --
> >> ciao
> >> Luca
> >>
> >> www.lucadelu.org
> >
> > ___
> > grass-dev mailing list
> > grass-dev@lists.osgeo.org
> > https://lists.osgeo.org/mailman/listinfo/grass-dev
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Import PostGIS vector layer into SQLite database

[Markus Metz]

On Wed, Mar 27, 2019 at 5:42 PM Luí­s Moreira de Sousa <
luis.de.so...@protonmail.ch> wrote:
>
> Hi all,
>
> I am working with vector layers (points) composed of more than 100 000
geometries. These layers are originally in a PostGis database and I import
them with v.in.ogr, using a formulation like:
>
> v.in.ogr \
> input="PG:host=127.0.0.1 dbname=gisdb user=user password=secret" \
> layer=schema.points \
> output=points \
> -o --overwrite
>
> This command is successful, however, operations with the resulting points
layer are remarkably slow. For instance, this layer must be later loaded in
R and that operation alone is taking over 3 hours.
>
> After some back-and-forth testing, I identified the problem: v.in.ogr is
not saving its output in the internal GRASS SQLite database. Knowing this,
I can create a copy of the original output, that is automatically inserted
into the SQLite database, e.g.:
>
> g.copy vector=points,points_mirror
>
> Importing the points_mirror layer to R takes 3 minutes instead of 3
hours, for instance.

Which GRASS version are you using? This should not happen with recent
versions of GRASS 7.4 and 7.6.

Markus M

>
> While mirroring vectors imported with v.in.ogr solves the issue, it is an
inelegant solution that also takes up unnecessary space. I would prefer to
instruct v.in.ogr somehow to force it to use the SQLite database. I could
not find any reference in the manual, is it possible to do?
>
> Thank you.
>
> --
> Luís Moreira de Sousa
> Email: luis.de.so...@protonmail.ch
> RingID: ring:7ca91d83f4f9dec82fec9f1144b8e5c1ef2a110c
> URL: https://ldesousa.github.io
>
> Sent with ProtonMail Secure Email.
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3790: Cleanup gettext usage for python code

[Markus Metz]

On Sat, Mar 16, 2019 at 3:32 PM Markus Neteler  wrote:
>
> Hi devs,
>
> GRASS GIS  schrieb am Di., 12. März 2019, 08:45:
>>
>> #3790: Cleanup gettext usage for python code
>> --+-
>>   Reporter:  pmav99   |  Owner:  grass-dev@…
>>   Type:  enhancement  | Status:  new
>>   Priority:  normal   |  Milestone:
>>  Component:  Python   |Version:  svn-trunk
>> Resolution:   |   Keywords:
>>CPU:  Unspecified  |   Platform:  Unspecified
>> --+-
>> Changes (by pmav99):
>>
>>  * Attachment "gettext_cleanup.diff" added.
>
>
>
> Are there any objections to apply the changes proposed by pmav99?

The patch looks good to me, it provides code simplification and the
location for the import of gettext makes sense.

Markus M
>
> Thanks,
> Markus
>
>
>> --
>> Ticket URL: 
>> GRASS GIS 
>>
>> ___
>> grass-dev mailing list
>> grass-dev@lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/grass-dev
>
> ___
> grass-dev mailing list
> grass-dev@lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] GRASS GIS + PROJ 6 + GDAL 2.5

[Markus Metz]

Dear Roger,

On Fri, Mar 8, 2019 at 10:49 AM Roger Bivand  wrote:
>
> Since rgdal::make_EPSG() is facing the same problems of listing tabulated
> EPSG fields as g.proj -l, I was very happy to see Markus' code in
> g.proj/main.c mentioned in this thread, and have used this approach in
>
https://r-forge.r-project.org/scm/viewvc.php/pkg/src/proj_info6.cpp?view=markup=rgdal
>
> However, there are plenty of messages such as: "proj_as_proj_string:
> Unsupported conversion method: Lambert Conic Conformal (West
Orientated)". I
> haven't installed GRASS trunk with PROJ6, so I can't see whether g.proj -l
> also sees the same messages. If it does, maybe we could ask on the proj
list
> how they might be captured for summary reporting. I think they are coming
> from line 5758 in src/iso19111/coordinateoperation.cpp or maybe line 906
in
> same file. Maybe PROJ now has an error handler that
>
> Another question concerns the issue of whether one needs to free objects
> created, in particular proj_crs_info and pj. Not so important for g.proj,
> which exists when done, but important for rgdal whose functions don't
exit.
>
> Anyway, very helpful to see that Markus is looking at the same issues as
we
> are!

Regarding parsing the traditional PROJ init files, I looked at pj_init.c in
PROJ 5 and earlier and attempted to improve parsing those traditional init
files in GRASS trunk r74224. I guess that R as well as GRASS is trying to
support PROJ versions from 4(.8) onwards.

Regards,

Markus M
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] r.in.gdal with option configuration

[Markus Metz]

On Tue, Mar 12, 2019 at 6:51 PM Pablo J. Zader  wrote:
>
> Hi list
>
> Currently, I need to specify in the "r.in.gdal" command, certain
configuration options provided by the gdal package. i.e .
>  --config  
> as described in the following report [1].
>
> Has this modification been implemented or is being implemented in any
version of GRASS GIS?
>
> [1] https://trac.osgeo.org/grass/ticket/3413
>
Yes, it is available as r.in.gdal gdal_config= in GRASS 7.6+
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev