Re: [GRASS-dev] [GRASS GIS] #3849: Do we need to clear the screen when entering/exiting GRASS?

2019-05-20 Thread GRASS GIS 
#3849: Do we need to clear the screen when entering/exiting GRASS?
--+-
  Reporter:  pmav99   |  Owner:  grass-dev@…
  Type:  enhancement  | Status:  new
  Priority:  normal   |  Milestone:
 Component:  Startup  |Version:  svn-trunk
Resolution:   |   Keywords:
   CPU:  Unspecified  |   Platform:  Unspecified
--+-

Comment (by pmav99):

 I closed [https://github.com/OSGeo/grass/pull/4 PR4] since it does more
 than one thing and it might need to be further discussed.
 [https://github.com/OSGeo/grass/pull/12 PR12] is a new one that only
 removes the clear screen stuff.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

2019-05-20 Thread Vaclav Petras 
On Mon, May 20, 2019 at 1:46 PM Markus Metz 
wrote:

>
> On Mon, May 20, 2019 at 7:18 PM Vaclav Petras 
> wrote:
> >
> > On Mon, May 20, 2019 at 11:57 AM Markus Metz <
> markus.metz.gisw...@gmail.com> wrote:
> >>
> >> On Mon, May 20, 2019 at 5:39 PM Vaclav Petras 
> wrote:
> >> >
> >> > Hi MarkusM and all,
> >> >
> >> > I'm trying to understand if the Segment_open() [1] function fills
> with zeros or not. I don't think it does since it is calling G_malloc
> (malloc) or Segment_format_nofill(). However, it is not completely clear to
> me what is supposed to be doing because documentation still says it calls
> Segment_format() and I don't understand context of the related commit [2,
> 3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
> [4].
> >>
> >> Segment_open() uses Segment_format_nofill() [1], if it can not use the
> all-in-memory cache. The documentation has not been updated accordingly
> (yet).
> >
> >
> > Thank you. Just be to be sure: Now, if you want to ensure that it is
> zero-filled across platform, you need to do it yourself using
> Segment_put(), right?
>
> All modules that use the segment lib use Segment_put() anyway to load data
>

I'm asking about the case when I'm writing a new raster map. Let's say with
point binning (r.in.lidar/r.in.pdal) or with a simulation I want the values
to be initialized as zeros. I can use Segment_put(), but it seems that what
Segment_format() is for. Hence the question about Segment_open_zero_fill()
(as something more costly than Segment_open() but cheaper than
Segment_open()+Segment_put()).


> > (Assuming you want to use Segment_open().)
>
> or Segment_format_nofill() + Segment_init()
>

I don't want to use Segment_init(), it is just too low level comparing to
your Segment_open() (plus it does not have the all-in-memory mode/code).


>
> > The exact role of lseek() here is still unclear to me (the hole and \0
> bytes).
> man lseek:
>lseek() allows the file offset to be set beyond the end of the file
> (but this does not change the size of the file).  If data is later  written
>at this point, subsequent reads of the data in the gap (a "hole")
> return null bytes ('\0') until data is actually written into the gap.
>
> More info: disk space is allocated and file size increases only when
> actual data are written into the gap
>

Thanks. I have seen this, but the segment library is not using the null
bytes as zeros, or does it?

> ...
>> The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().
> ...

It seems that not all modules are doing this (e.g. r.cost). How big a
problem that is? I guess the result is a truncated raster without a warning
in case of "no space left on device".

>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

2019-05-20 Thread Markus Metz 
On Mon, May 20, 2019 at 7:18 PM Vaclav Petras  wrote:
>
>
>
> On Mon, May 20, 2019 at 11:57 AM Markus Metz <
markus.metz.gisw...@gmail.com> wrote:
>>
>>
>>
>> On Mon, May 20, 2019 at 5:39 PM Vaclav Petras 
wrote:
>> >
>> > Hi MarkusM and all,
>> >
>> > I'm trying to understand if the Segment_open() [1] function fills with
zeros or not. I don't think it does since it is calling G_malloc (malloc)
or Segment_format_nofill(). However, it is not completely clear to me what
is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].
>>
>> Segment_open() uses Segment_format_nofill() [1], if it can not use the
all-in-memory cache. The documentation has not been updated accordingly
(yet).
>
>
> Thank you. Just be to be sure: Now, if you want to ensure that it is
zero-filled across platform, you need to do it yourself using
Segment_put(), right?

All modules that use the segment lib use Segment_put() anyway to load data

> (Assuming you want to use Segment_open().)

or Segment_format_nofill() + Segment_init()

> The exact role of lseek() here is still unclear to me (the hole and \0
bytes).
man lseek:
   lseek() allows the file offset to be set beyond the end of the file
(but this does not change the size of the file).  If data is later  written
   at this point, subsequent reads of the data in the gap (a "hole")
return null bytes ('\0') until data is actually written into the gap.

More info: disk space is allocated and file size increases only when actual
data are written into the gap

>
> Would Segment_open_zero_fill() make sense, i.e. do you know if
Segment_fill() faster then Segment_put()?

Segment_put() is used anyway to load the actual data, therefore the
question if a new Segment_open_zero_fill() is an alternative to
Segment_put() is invalid because you need to use Segment_put() in any case
to populate the Segment structure with meaningful data.

Markus M
>
>>
>> The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().
>
>
> Makes sense. I'll document that as well.
>
>>
>>
>> HTH,
>>
>> Markus M
>>
>> [1] https://github.com/OSGeo/grass/blob/master/lib/segment/open.c#L89
>>
>> >
>> > Markus, can you please clarify that for me? I will then update the
documentation with whatever is needed.
>> >
>> > Thanks,
>> > Vaclav
>> >
>> > [1]
https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
>> > [2]
https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
>> > [3] https://trac.osgeo.org/grass/changeset/73268
>> > [4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

2019-05-20 Thread Vaclav Petras 
On Mon, May 20, 2019 at 11:57 AM Markus Metz 
wrote:

>
>
> On Mon, May 20, 2019 at 5:39 PM Vaclav Petras 
> wrote:
> >
> > Hi MarkusM and all,
> >
> > I'm trying to understand if the Segment_open() [1] function fills with
> zeros or not. I don't think it does since it is calling G_malloc (malloc)
> or Segment_format_nofill(). However, it is not completely clear to me what
> is supposed to be doing because documentation still says it calls
> Segment_format() and I don't understand context of the related commit [2,
> 3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
> [4].
>
> Segment_open() uses Segment_format_nofill() [1], if it can not use the
> all-in-memory cache. The documentation has not been updated accordingly
> (yet).
>

Thank you. Just be to be sure: Now, if you want to ensure that it is
zero-filled across platform, you need to do it yourself using
Segment_put(), right? (Assuming you want to use Segment_open().) The exact
role of lseek() here is still unclear to me (the hole and \0 bytes).

Would Segment_open_zero_fill() make sense, i.e. do you know if
Segment_fill() faster then Segment_put()?


> The advantage of no fill, only lseek, is that it is faster, the
> disadvantage is that any "no space left on device" error will be
> encountered only later on, and you always need to check the return code of
> Segment_put().
>

Makes sense. I'll document that as well.


>
> HTH,
>
> Markus M
>
> [1] https://github.com/OSGeo/grass/blob/master/lib/segment/open.c#L89
>
> >
> > Markus, can you please clarify that for me? I will then update the
> documentation with whatever is needed.
> >
> > Thanks,
> > Vaclav
> >
> > [1]
> https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
> > [2]
> https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
> > [3] https://trac.osgeo.org/grass/changeset/73268
> > [4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
>
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] Segment library zero filling

2019-05-20 Thread Markus Metz 
On Mon, May 20, 2019 at 5:39 PM Vaclav Petras  wrote:
>
> Hi MarkusM and all,
>
> I'm trying to understand if the Segment_open() [1] function fills with
zeros or not. I don't think it does since it is calling G_malloc (malloc)
or Segment_format_nofill(). However, it is not completely clear to me what
is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].

Segment_open() uses Segment_format_nofill() [1], if it can not use the
all-in-memory cache. The documentation has not been updated accordingly
(yet). The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().

HTH,

Markus M

[1] https://github.com/OSGeo/grass/blob/master/lib/segment/open.c#L89

>
> Markus, can you please clarify that for me? I will then update the
documentation with whatever is needed.
>
> Thanks,
> Vaclav
>
> [1]
https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
> [2]
https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
> [3] https://trac.osgeo.org/grass/changeset/73268
> [4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] Segment library zero filling

2019-05-20 Thread Vaclav Petras 
Hi MarkusM and all,

I'm trying to understand if the Segment_open() [1] function fills with
zeros or not. I don't think it does since it is calling G_malloc (malloc)
or Segment_format_nofill(). However, it is not completely clear to me what
is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].

Markus, can you please clarify that for me? I will then update the
documentation with whatever is needed.

Thanks,
Vaclav

[1]
https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
[2]
https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
[3] https://trac.osgeo.org/grass/changeset/73268
[4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] [GRASS GIS] #3846: r.contour started producing broken/segmented contour lines since 7.4

2019-05-20 Thread GRASS GIS 
#3846: r.contour started producing broken/segmented contour lines since 7.4
--+---
  Reporter:  metello  |  Owner:  grass-dev@…
  Type:  defect   | Status:  new
  Priority:  normal   |  Milestone:
 Component:  Raster   |Version:  unspecified
Resolution:   |   Keywords:  r.contour ​helpwanted
   CPU:  x86-64   |   Platform:  MSWindows
--+---

Comment (by mmetz):

 Replying to [comment:3 mmetz]:
 > Replying to [comment:1 metello]:
 > > The file was too large to attach. I uploaded here:
 > > http://psr-
 shared.s3.amazonaws.com/public/admin/2C66EC59/R_CONTOUR_BUG.zip
 > >
 > > Coordinates of some points of discontinuity:
 > > (384073.870002,8827636.45371),
 > > (402242.43372,8799021.33681),
 > > (409632.732253,8796197.58953)
 >
 > I can confirm the missing segments, the previous fix to prevent contours
 from tracing back on themselves was not working. Working on it...

 r.contour has been fixed in master, relbr76, and relbr74.

-- 
Ticket URL: 
GRASS GIS 

___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

[GRASS-dev] black: Python code formatter (eg PEP8)

2019-05-20 Thread Markus Neteler 
FYI

Back - Python code formatter (eg. PEP8)
https://github.com/python/black

# example: default settings mit max. 79 Buchstaben per Zeile
$ black -l 79 ./*.py

Online playground:
https://black.now.sh/

Would that be useful for us?

cheers
Markus
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev

Re: [GRASS-dev] git: how to switch between branches?

2019-05-20 Thread Panagiotis Mavrogiorgos 
Hello Markus

I don't have real-world experience with this, but I am not sure if changing
branches and
distcleaning/recompiling is the most convenient workflow. I think I would
prefer to have
multiple local repositories with different branches checked out. Thankfully
git makes it
rather easy to have this.  E.g.:

1. You create two root directories. One for Python2 and one for Python3:

mkdir /home/user/git/grass-p{2,3}

2. Create a virtualenv inside grass-p2 and grass-p3. E.g.:

cd /home/user/git/grass-p2
virtualenv -p python2 venv

cd /home/user/git/grass-p2
python3 -m venv venv

3. Clone the remote GRASS repo and name the local repo `dev`. You will only
do this for `grass-p3`:

cd /home/user/git/grass-p3
git clone https://github.com/neteler/grass dev

5. For python 2 and for each release branch you will make a local git
clone. This way
   you won't be wasting disk space. Read more about this
   [here](
https://stackoverflow.com/questions/7187088/git-how-to-create-local-repo-with-symbolic-links-to-the-git-main-repo
).

cd /home/user/git/grass-p2
git clone ../grass-p3/dev dev
git clone ../grass-p3/dev r72
git clone ../grass-p3/dev r74
git clone ../grass-p3/dev r76

6. The dir structure should look like this:

$ tree grass*

grass-p2
├── dev
├── r72
├── r74
├── r76
└── venv
grass-p3
├── dev
└── venv

7. On each release clone you need to checkout the respective branch:

cd /home/user/git/grass-p2/r72 && git checkout releasebranch_7_2
cd /home/user/git/grass-p2/r74 && git checkout releasebranch_7_4
cd /home/user/git/grass-p2/r76 && git checkout releasebranch_7_6

7. Each directory is a full blown git repo. It is only by convention that
7.6 backports
   will happen in `r76` etc.  If you want to directly pull/push from/to
github from the release
   directories, you will probably need to setup remotes. Regardless,
setting up remotes
   etc should only be done once.

8. Obviously, when 7.8 gets released, two more directories will need to be
added (one
   for python2  and one for python 3).

The benefit of this approach is that at least for trivial backports, you
will not need
to recompile everything and, perhaps more importantly, you will not need to
recompile
"master" in order to test a fix in 7.2.

all the best,
Panos

PS1. There are other ways to achieve something like this, e.g.
[`--singlebranch`](https://stackoverflow.com/a/1911126/592289) but I don't
see much
benefit.

PS2. You can optionally use something like [direnv](
https://github.com/direnv/direnv) to
automatically activate the virtualenvs when you cd into the corresponding
directory.
This way there is no confusion WRT which python is active. I use this and
it works
marvellously.
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev