Re: [GRASS-dev] nc_spm_08_grass8?
On Fri, Sep 10, 2021 at 3:02 AM Maris Nartiss wrote:
> 2021-09-10 5:56 GMT+03:00, Vaclav Petras :
...
> = Not a big difference from existing implementation
>
Just to be clear, all these would be in addition to the proper handling of
band references and applied only when you don't want to deal with band
references directly.
...
>
> > Option 4) i.gensigset et al. labels the bands 1,2,3,... on the fly only
> in
> > the signature file and i.smap et al. uses the same numbering scheme when
> > the rasters don't have band references. Band references are handled only
> > when needed even on the module level in addition to the user level.
>
> Didn't understood how this differs from option #2.
>
The difference would be that the band references are never added to the
rasters in #4, only added to the signature file and re-created in the same
way for classification. In #2 they are actually added to rasters by
i.group. However, here is another option which I think is better than my
previous ones:
Option 5) Add a library function to read band reference which has a
fallback (band reference getter with a default). A reasonable fallback
seems to be the name of the raster map. This function can be then used on
all places in the signature/classification context instead of
Rast_read_bandref. i.gensigset et al. writes raster names as band
references to the signature file, i.smap et al. reads these names and
matches them with result from this new function unless the user didn't set
or renamed the raster map in the meantime. No additional step needed to the
v7 workflow if you don't need to use band references. The signatures are
associated with the raster map name rather than the order (as it is in v7).
Signatures from one mapset can then be used in another mapset as long as
the names of the rasters are the same (let's say each scene imported under
the same name into its own mapset for parallel processing). Something like
this function:
/** Return band reference or fallback to raster name if it is not set. */
char *Rast_read_bandref_with_fallback(const char *name, const char *mapset)
{
char *ret = Rast_read_bandref(name, mapset);
if (ret)
return ret;
else
return G_store(name);
}
___
grass-dev mailing list
grass-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming master branch to main branch
On Fri, Sep 10, 2021 at 4:36 PM Markus Metz wrote: > > ...all the previously timed-out tests for the affected PRs are passed... > As for the timed-out tests - those which are running actual test suites and time out after 6 hours - that's actually still an issue. It just usually does not happen. It is not clear to me what the cause is. You can see it stuck at different tests, usually (always?) temporal, but that's also because there are just a lot of temporal ones. I enabled the test result (artifact) download even for that case, but I didn't see anything useful there either. However, the fact that the artifact preparation step works suggests that the GitHub VM itself is not stuck and it is really something with our code or tests. ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming master branch to main branch
On Fri, Sep 10, 2021 at 8:46 PM Vaclav Petras wrote: > > On Fri, Sep 10, 2021 at 2:30 PM Markus Metz wrote: >> >> The reason for the rebase was that some tests were failing, apparently because of a combination of renaming master to main and changes to GHA, and a PR can only be squashed and merged if all tests are passed. > > Ah, that makes sense. You need to update in one way or another in this case. After `git rebase main` + `git push -f ...`, all the previously timed-out tests for the affected PRs are passed, great! Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming master branch to main branch
On Fri, Sep 10, 2021 at 2:30 PM Markus Metz wrote: > > The commit history of the PR has not been messed up, only the relevant > commits are shown. > Right, right, Git or GitHub recognizes that thoise 41 are already on main. The reason for the rebase was that some tests were failing, apparently > because of a combination of renaming master to main and changes to GHA, and > a PR can only be squashed and merged if all tests are passed. > Ah, that makes sense. You need to update in one way or another in this case. ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming master branch to main branch
On Fri, Sep 10, 2021 at 7:51 PM Vaclav Petras wrote: > > > > On Fri, Sep 10, 2021 at 1:16 PM Markus Metz wrote: >> >> ... >> $ git checkout raster_tempdir >> $ git rebase main >> >> Successfully rebased and updated refs/heads/raster_tempdir. >> >> $ git status >> On branch raster_tempdir >> Your branch and 'metzm/raster_tempdir' have diverged, >> and have 48 and 7 different commits each, respectively. >> (use "git pull" to merge the remote branch into yours) >> >> $ git push metzm raster_tempdir >> ! [rejected]raster_tempdir -> raster_tempdir (non-fast-forward) >> error: failed to push some refs to 'github.com:metzm/grass.git' >> hint: Updates were rejected because the tip of your current branch is behind >> hint: its remote counterpart. Integrate the remote changes (e.g. >> hint: 'git pull ...') before pushing again. >> hint: See the 'Note about fast-forwards' in 'git push --help' for details. > > > This is how it is supposed to behave when you do `git rebase`. Rebasing re-applies the changes you made in raster_tempdir one by one on top of your local main. This creates different commits in the sense that the commit hashes are different. This causes the "have diverged" part. 48 on the local raster_tmpdir is your 7 commits plus 41 commits which happened in the main branch since you created raster_tempdir. 7 commits on metzm/raster_tempdir is the original commits you made which have the original, different hash, so Git considers them to be different. > > Force push with `git push -f metzm raster_tempdir` is appropriate here. You will replace whatever is in the remote branch by your newly updated (rebased) local branch content. A force push helped, thanks! The commit history of the PR has not been messed up, only the relevant commits are shown. The reason for the rebase was that some tests were failing, apparently because of a combination of renaming master to main and changes to GHA, and a PR can only be squashed and merged if all tests are passed. Thanks for the fast response! Markus M ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming master branch to main branch
On Fri, Sep 10, 2021 at 1:16 PM Markus Metz wrote: > ... > $ git checkout raster_tempdir > $ git rebase main > > Successfully rebased and updated refs/heads/raster_tempdir. > > $ git status > On branch raster_tempdir > Your branch and 'metzm/raster_tempdir' have diverged, > and have 48 and 7 different commits each, respectively. > (use "git pull" to merge the remote branch into yours) > > $ git push metzm raster_tempdir > ! [rejected]raster_tempdir -> raster_tempdir (non-fast-forward) > error: failed to push some refs to 'github.com:metzm/grass.git' > hint: Updates were rejected because the tip of your current branch is > behind > hint: its remote counterpart. Integrate the remote changes (e.g. > hint: 'git pull ...') before pushing again. > hint: See the 'Note about fast-forwards' in 'git push --help' for details. > This is how it is supposed to behave when you do `git rebase`. Rebasing re-applies the changes you made in raster_tempdir one by one on top of your local main. This creates different commits in the sense that the commit hashes are different. This causes the "have diverged" part. 48 on the local raster_tmpdir is your 7 commits plus 41 commits which happened in the main branch since you created raster_tempdir. 7 commits on metzm/raster_tempdir is the original commits you made which have the original, different hash, so Git considers them to be different. Force push with `git push -f metzm raster_tempdir` is appropriate here. You will replace whatever is in the remote branch by your newly updated (rebased) local branch content. We are currently merging all PRs using the "Squash and merge" feature, this reduces the commit which will go to the main branch to only the actual changes taking care of the merge commits. So, you can use git merge or git rebase on your branches. I still prefer rebase most of the time because it is easier to review commits in the PR (since there are no merge commits). Some people also use rebase to reduce the commits on a branch to one, but I think that's usually not helpful for review. However, merge does not require the strange force pushing and it works better in case of conflicts because you are simply merging the latest state, not each commit on your branch which is the case with rebase. As far as I can tell, the master-main rename should not have any influence on this. And yes, the message from `git push` is not applicable here. ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
Re: [GRASS-dev] Renaming master branch to main branch
Probably a silly question: I'm having problems with rebasing my local feature branches for PRs, e.g. for https://github.com/OSGeo/grass/pull/1786 My local main branch is up to date with the official main branch. In my case the grass repo is the remote named origin, my fork is the remote named metzm $ git checkout raster_tempdir $ git rebase main Successfully rebased and updated refs/heads/raster_tempdir. $ git status On branch raster_tempdir Your branch and 'metzm/raster_tempdir' have diverged, and have 48 and 7 different commits each, respectively. (use "git pull" to merge the remote branch into yours) $ git push metzm raster_tempdir ! [rejected]raster_tempdir -> raster_tempdir (non-fast-forward) error: failed to push some refs to 'github.com:metzm/grass.git' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. Integrate the remote changes (e.g. hint: 'git pull ...') before pushing again. hint: See the 'Note about fast-forwards' in 'git push --help' for details. Any hints on how to proceed from here without messing up the history of my feature branch? Thanks! Markus M On Sun, Aug 22, 2021 at 9:26 PM Vaclav Petras wrote: > Dear all, > > The master branch was renamed to main. Please update your local > repositories and workflows accordingly. > > ## 1. Local repo update and branch rename > > If you are contributing code or compiling source code from Git, you need > to modify your local repository as well. Go to your fork's website on > GitHub. It will warn you about the rename in the OSGeo repo. Go to settings > and to the rename. Go to the main page of your fork again. Now it will warn > you about the rename in your repo (your fork). It will give you > instructions on renaming the local branch and making it point to the > renamed branch. (The assumption is that your fork repo is the remote named > origin.) > > Additionally, there still will be upstream/master and you can remove it > using (assuming upstream remote is the OSGeo repo. You can remove > upstream/master and update info about the default branch using: > > git fetch upstream --prune > git remote set-head upstream -a > > Alternatively, if you are not keeping your fork's master/main branch up to > date, you can follow instructions from my initial email which allow you to > skip the fork part resulting in a slightly different local setup. > > ## 2. Updating your contributing steps > > To update your local main branch (base branch) and to rebase or merge with > the latest source code, use upstream/main instead of upstream/master. > > ## 3. Updating workflows using source code from Git > > If you have a workflow which is using the source code obtained from Git , > you will need to update it if you explicitly specify the branch name. If > you do --branch, checkout, or switch, and at the same time using the master > branch, you will need to change "master" to "main". If you use only a > simple `git clone`, no changes are needed because `git clone` takes the > default branch which will continue to work. > > ## 4. Other issues > > The base branch for PRs was changed automatically. The PRs updating most > of the source code for core and addons were merged. Some CI-related changes > will still be needed for core. Additionally, I assume the workflows outside > of the source code will be updated as part of the updates to addons repo > structure and changes for the release branch for 8.0. Help in this area is > welcome. > > https://github.com/OSGeo/grass/pull/1806 > https://github.com/OSGeo/grass-addons/pull/598 > > Let me know if you face any questions. > > Best, > Vaclav > > > On Wed, Aug 18, 2021 at 10:41 PM Vaclav Petras > wrote: > >> Dear all, >> >> I will rename our default branch currently called master to main in the >> following days. We already discussed it here and there, but the rename >> should be relatively smooth. Much simpler than the addons repo >> reorganization. Users not compiling code themselves should not be affected. >> Contributors will be, but GitHub warns you about the rename when you come >> to the repo web page. Those compiling from source may need to make changes. >> >> The update of a local clone can be done with instructions similar to the >> following (I will post them again once I confirm it is the best fit): >> >> git branch -m master main >> git fetch upstream >> git branch -u upstream/main main >> git remote set-head upstream -a >> >> Additionally, to get rid of upstream/master, do: >> >> git fetch --prune >> >> You don't need to update your fork unless you want to. >> >> The changes needed in the repo are already in a PR (feel free to review): >> >> https://github.com/OSGeo/grass/pull/1806 >> >> Those compiling from source code obtained from Git will need to make >> changes when they explicitly specify the branch. A simple `git clone` takes >> the default branch which will continue to work. However, if you do >> --branch, checkout, or switch,
[GRASS-dev] GRASS releases: Let's meet to discuss/coordinate them!
Dear all, Next Tuesday, September 14th at 21:00 CEST [1] we'll (videocall) meet to discuss/coordinate upcoming GRASS GIS releases. You are all welcome to join and contribute :) The link for the meeting will be shared in this thread soon before the meeting. Looking forward to seeing you Have a great weekend! Vero [1] https://www.timeanddate.com/worldclock/meetingdetails.html?year=2021=9=14=19=0=0=485=207=48=25=197 ___ grass-dev mailing list grass-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/grass-dev
[GRASS-dev] nc_spm_08_grass8?
Had to resent the mail due to size limitations. You can take a look at the attachments here: https://karlsons.gisnet.lv/~marisn/imagery_classification_G7.png https://karlsons.gisnet.lv/~marisn/imagery_classification_G8.png 2021-09-10 5:56 GMT+03:00, Vaclav Petras : > > Why not to have both? Classify thousands of scenes while providing a simple > way of doing things? Because GRASS metadata handling implementation is inferior. See Sentinel SAFE usage in SNAP for a correct one (XML/Java discussion aside). > If all import tools assigned band references, having them ready in the > dataset for landsat could be perhaps justified. That's the answer I was > hoping for, but that's not the case. Still, the question would be what if > you can't use such a tool. Depends on the format. If imported format has band references (e.g. SAFE), they should be written in GRASS as band references too. Still this is a TODO for another day. > As for getting closer to the original behavior where 5 commands were enough > to classify, i.group could add the band references since one needs to deal > with both groups and bands anyway or the signature handling could assign > them on the fly. I'm probably missing some important details you know > about, but here are some options I can think of: To better understand my answer, take a look at two attached graphs – how imagery groups and signature files work in G7 and how they have been changed for G8. In G7 imagery group is only consistent internally. In G8 signature files from one group are consistent in any group. > Option 1) i.group has a new option bandref. It assigns band references to > the rasters. User needs to provide as many band references as inputs. Still > quite long, but i.group call is long already and it is at least technically > one step. + One long call instead of multiple calls to r.support - Raster band must be set for each group separately (think NDVI or slope map in multiple groups) - User must set the same labels in a correct order also for any other group where signature file is used = Not a big difference from existing implementation > Option 2) i.group has a new flag to auto-assign band references 1,2,3,... > (2a). Or perhaps an option taking a prefix/basename (2b). Simple to set. > Minimal change from the current workflows. Almost feels like band > references are optional. + Works well if one classifies the same group only - Signature file can not be used to classify any other group as there is no guarantee of band order in groups = This is a GRASS 7 approach with a hack of copying SIG file from one group to other group > Option 3) i.group auto-assigns automatically when band references are not > present. No dealing with band references unless one needs to. + Works well if one classifies the same group only - Signature files must be modified to contain group name - User will be able to select a signature file for classification of a different group but action will be refused if band references are missing = Confusing as one can not know in advance if signature file can or can not be used to classify a group (should we call it a pythonic way – try...except?) > Option 4) i.gensigset et al. labels the bands 1,2,3,... on the fly only in > the signature file and i.smap et al. uses the same numbering scheme when > the rasters don't have band references. Band references are handled only > when needed even on the module level in addition to the user level. Didn't understood how this differs from option #2. > With all options users still may take advantage of the flexible signature > file system if being careful about order. (Keeping the order right is > likely not an issue while scripting, so having a list of auto-assigned > numbers is just fine.) All options hide details of the actual reference > handling at least a little bit providing more space for changes later on. > The options 3 and 4 make dealing with band references completely optional. > Option 4 avoids mixing groups and band references and while option 3 hides > that part. There is no guarantee of order of bands in a REF file. If band order stays the same, you are lucky, if it isn't, you just got an incorrect result without a warning (a.k.a. flipping a coin to determine if output is correct or total bollocks as band order was messed up). Assigning band reference via scripting is not a problem if e.g. it is present in the raster name. > Other options would be modules such as r.number.bands/i.auto.bands (kind of > like options 2 and 3 but standalone) and i.band.identify (some heuristic to > determine band names from raster names perhaps taking an example or a > sensor). This is the correct way to go. I would say: 1) enhance import tools to generate band references (if possible). I assume specialized Sentinel, Landsat, Modis etc. import modules could be enhanced easily to do so. 2) provide an easy GUI tool to assign band references. Here bands from g.bands would come handy. Probably i.bands
