On 2023-02-06 16:06, Gedare Bloom wrote:
On Mon, Feb 6, 2023 at 2:26 AM Christian MAUDERER
<christian.maude...@embedded-brains.de> wrote:
On 2023-02-06 10:02, Christian MAUDERER wrote:
On 2023-02-05 11:38, Christian Mauderer wrote:
Am 04.02.23 um 01:25 schrieb Gedare Bloom:
On Fri, Feb 3, 2023 at 3:29 PM Christian Mauderer <o...@c-mauderer.de>
wrote:
Am 3. Februar 2023 22:52:48 MEZ schrieb Gedare Bloom
<ged...@rtems.org>:
On Fri, Feb 3, 2023 at 2:39 PM Christian Mauderer
<o...@c-mauderer.de> wrote:
Am 3. Februar 2023 22:12:06 MEZ schrieb Gedare Bloom
<ged...@rtems.org>:
On Fri, Feb 3, 2023 at 12:52 PM <o...@c-mauderer.de> wrote:
Hello Gedare,
Am 03.02.23 um 19:51 schrieb Gedare Bloom:
On Thu, Feb 2, 2023 at 11:24 PM Christian MAUDERER
<christian.maude...@embedded-brains.de> wrote:
Hello Karel,
On 2023-02-02 12:43, Karel Gardas wrote:
Guys,
recently I needed to work with RTEMS/NFS. As this is provided
by libbsd
I took this and following two sentences below from master branch
description provided in README I took as granted that master
does have
all the features which are currently available and provided
by the project:
"This branch must be used for libbsd development. Back ports
to the
6-freebsd-12 are allowed."
I was surprised to be proven wrong then by Fabrizio here:
https://devel.rtems.org/ticket/4723
and by later investigation which shows that 6-freebsd-12 branch
accumulated NFS work by Chris done in 2021 which is not
presented on
master. I've investigated just NFS as this was my focus here.
So if 6-freebsd-12 became development branch of some sort,
then it would
be great to have that clarified in the project README file to
prevent
users confusion? Or if the policy is still the same, then
perhaps some
branch sync is needed here?
That currently is an open issue. Basically there is a pending
patch set
that should fix that since several months. But there is a
disagreement
about some of the changes in that patch set (and about the
patches
checked in to 6-freebsd-12). Therefore, it still hasn't been
merged.
If you want to know some more about the problematic points, I
recommend
reading this (long) thread:
https://lists.rtems.org/pipermail/devel/2023-January/074164.html
The statement that development has to happen on the master
branch is
still true. The master is intended to track the FreeBSD upstream
development. Only changes on that branch are guaranteed to
live through
an upgrade to a newer base version of FreeBSD. It's very
unfortunate,
that there are some patches on the 6-freebsd-12 branch only.
On the long
term, that issue has to be resolved.
I have been investigating this problem in the background, and I
have
some findings and some questions. First, I have found that
there is a
most-common ancestor between master and 6-freebsd-12 at commit
https://git.rtems.org/rtems-libbsd/commit/?h=6-freebsd-12&id=2ce13cf6dc73855f28bc7edbbc64dc4b482a4976
This is at least promising that the discrepancy between the
branches
can be resolved.
The proposed pending patch set to "fix" the NFS issue does not
fix the
underlying problem. Instead, it introduces further divergence
between
the branches. I would instead suggest that we should resolve to
fix
the underlying problem. I can see two paths forward.
1. Abandon 6-freebsd-12 after releasing 6. This is probably not
ideal
since what I understand is some users have projects based on
6-freebsd-12 and would like an upgrade path. (I guess there is
also
the option to abandon master, which also makes little sense.)
A variant for this would be to introduce a 6-freebsd-13 that is
based on
the master branch as soon as we have one. That would allow a longer
maintenance because FreeBSD 12 reaches it's EoL December 2023.
2. Pull commits from 6-freebsd-12 into master to make sure
master is
the development head. in the future, reject patches that only go
toward release branches. This has its own problems too. It can
realistically only be done in three ways:
Please note that Sebastian mentioned that the file descriptors
broke the
NTP support (at least I think it was NTP; possible that it was
another
submodule). So picking the current version of the patches into the
master without adding fixes makes the master unusable for some
cases.
Fixing the problems after making the branches the same will be
better
for the long-term, if we can find a path to do it.
2a: Rebase master and cherry-pick commits from 6-freebsd-12 and
master
back into master. This rewrites the history of master, and
unfortunately will cause the head of 5-freebsd-12 and the tags for
rtems-5 to no longer exist on the master branch. They will
still exist
in the '5' branch. The advantage is in the end there will be a
linear
history of development on master that reflects the timeline of
actual
development that spanned both branches. Theoretically, this should
make it easier to git-bisect.
2b: Cherry-pick commits from 6-freebsd-12 to master and fix
conflicts.
This puts all the missing commits from 6-freebsd-12 on to the
current
head of master. I don't know how messy this would be. It ends up
making the history of master convoluted to understand, with
fairly old
commits from 2018 being placed on top of newer commits from 2020s.
2c: Merge 6-freebsd-12 into master and fixup conflicts in the
merge
commit. This is pretty similar to 2a but ends up with a non-linear
history and a merge commit. It may be a fairly complex merge
commit.
For all of the 2x solutions: The commits from 6-freebsd-12 can't
just be
cherry-picked. You have to re-import the NFS files from the FreeBSD
master version that is used as base for the current libbsd master.
Otherwise we mix different FreeBSD source versions. We had that
some
time back in libbsd and Sebastian needed a lot of time cleaning
that up.
Understood.
To get a sense of the difference between the two branches, I
have done
the following command:
$ git log --pretty=oneline master...6-freebsd-12 > ../log.txt
This uses the ... (three-dot) Symmetric Difference Notation. The
result of that is a 750 line file, so 750 commits are different
between the two branches. Some of those commits are actually
the same
content, but they have different parents so different hashes. In a
rebase or merge situation, those commits should end up the
same. There
may be other git-fu to find just the patches that are unique in
the
two branches.
750 commits are a bit too much. The order of patches on master and
6-freebsd-12 isn't always the same. I wrote a small python
script to
find the differences somewhere in 2021 when I needed the
differences in
a discussion with (I think) Joel. It compares based on author and
subject which gives a quite good estimate for libbsd. You can
find the
script here:
https://gist.github.com/a82e21eb250cb96c3a36f107b92dab09
That shows 138 different commits. 86 are only on 6-freebsd-12.
Complete
output is here:
https://gist.github.com/93a2b19a5bf4cd8a6263ae29aa359ac2
From these you can ignore the "Update to FreeBSD..." commits
and some
of the cleanup patches. I assume that quite some of them can be
cherry-picked with only small or no changes. For example patches
that
only add or update drivers in rtemsbsd should work without
problems. So
I think a more realistic number of problematic patches should be
30 to 50.
Oh, thanks, this is nice. It may be a reasonable place to start by
looking at what is "Only on master" and to consider back-porting to
6-freebsd-12. That may reduce the delta somewhat in a way that makes
sense.
We really should do the other direction. Not all features oft
master have to be on the stable branch. But all features of the
stable one should be on master.
I agree, but since there are many fewer changes only on master, if we
can merge those into 6-freebsd-12, then it becomes simpler to just
call 6-freebsd-12 as the new master. We can replace the current master
with this branch, and then update freebsd versions. I think this can
be a lower burden process to follow, except it requires acceptance of
the commits that are on 6-freebsd-12, which is a different matter.
You can't just make 6-freebsd-12 a new master. It tracks the FreeBSD
12 branch.
Every block of updates to a newer FreeBSD base revision was several
days of work for Sebastian. There are a few on master.
An upgrade from a branch tracking FreeBSD 12 to one tracking FreeBSD
13 is most likely more something in the several week range of work.
master tracks the FreeBSD master. As soon as it reaches the point
where FreeBSD branched off version 13, we can create a
6-freebsd-13 or 7-freebsd-13 from master (depending on which RTEMS
version we have then). All features not on master would be lost
when we do that.
The current 6-freebsd-12 on the other hand should be considered
stable. We won't upgrade it to a 7-freensd-12 or similar. It will
be a maintenance branch as soon as we reach the release. We can
backport features that have been tested on master if there are
users for that. But we don't have to. Just like we don't backport
every new feature from RTEMS 6 to 5.
I view this as equivalent to the proposals to revert commits that are
on 6-freebsd-12. Doing this means that we are discarding quite a lot
of software engineering effort that was done on 6-freebsd-12. I would
rather that we determine a way that can retain that effort. At this
point, there is probably little way to make everyone happy.
I think you misunderstood my intention. We definitively should port
the patches from 6-freebsd-12 to master so that they are on a future
libbsd branch. Like said above: I just don't think it's realistic to
base a new developed branch on the 12 one.
I see. This is good to consider.
If I remember correctly, Sebastian's branch that contains his
suggested fixes already ported Chris NFS work to master first and
then added Sebastian's patches that revert the parts where he
disagrees.
I would leave it to them to determine if the NFSv4 is fully supported.
That leaves some driver patches on the 12 branch that are most
likely a lot simpler to port to master.
My guess would be: Porting the stuff from 6-freebsd-12 to master
should be a few days maximum to a compile clean state. Upgrading
6-freebsd-12 to something that would follow FreeBSD master again
should be something in the range of weeks.
OK, then we should likely follow the path of porting from 6-freebsd-12
to master. I would leave the issue of what to do with FDs as a
separate concern that can be addressed once we solve this underlying
problem.
What would you think is the best way to handle the process though? It
would seem to still require some kind of cherry-picking + patching
FreeBSD and passing some tests? Perhaps it is best to do it that way,
on top of the current master branch, attempting to pick up patches
from the shared ancestor. ("Only in 6-freebsd-12").
I would usually just cherry-pick commits that don't import anything
from FreeBSD. All commits that do import files need a bit more work:
For these, the files from FreeBSD master have to be re-importet.
It's basically the same process that I use to back-port something from
master to 6-freebsd-12. And it's why CONTRIBUTING.md tells that you
should have one commit that only adds the unmodified imported files.
With that it's easy to re-import the files from a different version
and re-apply the patches.
Best regards
Christian
I tried to cherry-pick some of the commits yesterday (hobby time on
Sunday!). The first few patches should work quite well. There are some
minor problems that still have to be cleaned up to make it compile clean.
One of the early ones that makes problems is the path-mappings feature
from Jan Sommer because on master something changed the same code parts.
We need someone who understands both changes to port that.
About starting with Chris kernel symbol namespace tool patch
(59f652fe88) the changes start to get bigger and the patches are harder
to apply. I think the kernel symbol namespace is one of the core patches
that would be important to port, but I don't know the code surrounding
it well enough that I can do it fast.
How should we approach that work? One patch at a time? Maybe together
with pinging the original authors to ask them whether they can port
their patches to master or at least test them there?
For the VFS and NFS parts, it depends a bit on the solution of the
discussion. Sebastian already worked on a re-implementation of NFS on
master (not yet entirely finished):
https://github.com/sebhub/rtems-libbsd/commits/master-update
Of course that is based on his solution without the file descriptors and
without the central system calls. So it depends on the results of the
discussion between Sebastian and Chris whether that patch can be used to
have the NFS functionality in master.
When re-reading the text I noted that mentioning that branch here can
lead into a wrong direction. To make my intention clear: I think we
should concentrate on all unrelated patches at the moment. Parallel we
should try to push the NFS/VFS discussion and help Chris and Sebastian
to agree on one solution that is acceptable for both. Depending on that
solution, it could be simpler to just use that branch and maybe port
only small parts to have a functionally equivalent solution on both
branches with a slightly different history.
Yes, thanks. This looks like a good analysis. I would definitely
prefer to get master and 6-freebsd-12 into harmony. Then it is a
little simpler to discuss the other problems in libbsd.
Vijay had similar kind of success (and problems) as you did with
cherry-picking off of 6-freebsd-12. I think he made it a little
further. I guess one possible route forward would be to begin working
on this effort and share results, pushing to master when some
relatively stable milestones are reached.
Gedare
Hello Gedare,
I made sure to remove every patch that is not compile clean and pushed
my current working version here:
https://gitlab.com/c-mauderer/rtems-libbsd/-/tree/cm/20230206-port-patches-to-master
At the moment it's cherry-picking patches from 6-freebsd-12 to master as
long as they apply and compile. I didn't make sure yet that everything
works or is in a usefull order.
Comparison between this and 6-freebsd-12 is hiere:
https://gist.github.com/620ae08d88dca054c45100a84fdaaee8
Only 35 patches difference left if you ignore the "Update to FreeBSD"
patches that are branch specific.
A big one will be Chris' 'Implement portable kernel symbol namespace
tool' and the follow up commits that clean up the kern-symbols. That is
an important improvement but I don't know yet what is necessary to
update these patches to master.
Do you think we should create a new mailinglist thread?
Topic: 'libbsd master and 6-freebsd-12 cleanup; help wanted'
Goal: Port all patches from 6-freebsd-12 to master that are not branch
specific and not related to the currently open questions regarding the
correct aproach to NFS.
Status: Short summary of what we discussed and what we reached.
Best regards
Christian
Best regards
Christian
Note that Sebastian also mentioned, that FreeBSD changed a lot in the
NFS/VFS in the last few months. So it's possible that both solutions
need quite some changes after we reach that version of FreeBSD.
Best regards
Christian
Best regards
Christian
it will be much easier to debate the merits of proposed patches to
address optimization etc., if we have a shared basis of features and
development. The way this has been heading is going to lead us
directly into a non-viable mode of trying to support several open
development branches of libbsd. I can easily see someone wanting to
carry forward 6-freebsd-12 commits to rtems7. If we don't have a way
to do that, we will definitely end up with something like
7-freebsd-12.
We have to deal with the fact that there are legitimate developments
on 6-freebsd-12 that users may expect in future RTEMS releases. I
cannot agree with having a release of 6-freebsd-12 with rtems6 that is
followed by a release of 7-freebsd-1x that drops some features such as
support for NFSv4. It is not pragmatic. But we also don't have the
resources to be maintaining multiple branches of libbsd.
For example, I don't know of a user for the BBB framebuffer on
RTEMS 6. It's nice that it is there on master. It will become a
stable feature on the next major branch (one of 6 or 7-freebsd-13
or 14). But there is no necessity to backport it to the mostly
stable 6-freebsd-12.
I have asked Vijay to look at backporting it anyway, to reduce the
difference between the branches. I want to get us out of this backward
way of development / change management that has evolved over time in
libbsd. How best to get us to a state where new development only goes
to master is for me still the main question. Once we have this fixed,
it is my intention to thoroughly reject any development that is only
submitted to a "stable" branch in the future.
Gedare
Best regards
Christian
Best regards
Christian
In any case, doing this in a way that ensures the commits build
and
tests run is challenging due to the interactions with the
rtems.git,
toolchain, and the submodules. >
After the 6-freebsd-12 and master are made consistent, then it
becomes
possible to update freebsd and also to consider what ways can be
chosen to fix problems in 6-freebsd-12.
-Gedare
Best regards
Christian
I'm fine with either way, as a user I just need clear not
confusing
project message...
Thanks!
Karel
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
--
--------------------------------------------
embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email: christian.maude...@embedded-brains.de
phone: +49-89-18 94 741 - 18
mobile: +49-176-152 206 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen,
Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
--
--------------------------------------------
embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email: christian.maude...@embedded-brains.de
phone: +49-89-18 94 741 - 18
mobile: +49-176-152 206 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
--
--------------------------------------------
embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email: christian.maude...@embedded-brains.de
phone: +49-89-18 94 741 - 18
mobile: +49-176-152 206 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
