Re: [ptxdist] ptxdist initial impressions

[Ladislav Michl]

Hi there,

On Wed, May 24, 2023 at 12:34:21PM +0200, Alexander Dahl wrote:
> Hello Jookia,
> 
> as a long term user not affiliated with pengutronix I have some comments 
> which 
> might help.  See below.
> 
> Am Mittwoch, 17. Mai 2023, 18:11:31 CEST schrieb Jookia:
> > Hello ptxdist friends,
> > 
> > I spent some time over the past few days trying to use ptxdist and
> > documenting all the pain I hit in the process. This isn't a criticism
> > more just anecdotal data of my experience. Hopefully it's useful to
> > smooth down some rough edges.
> > 
> > I'm going to write this up in a psuedo-markdown format in sections,
> > sorry if that breaks anyone's workflow.
> > 
> > 
> > Documentation
> > -
> > 
> > First up: Documentation is quite good aside from the 'Getting a working
> > environment' section. I will be focusing on this today since I think it's
> > probably the most critical problem with ptxdist at the moment: I expect
> > that by the end of following this chapter I should have a bootable image
> > of some kind and commands for using it, such as with qemu.
> > 
> > The first issue is that I'm told to downloaded files, but I'm not linked
> > to them, quoting:
> > 
> > ptxdist-2023.05.0.tar.bz2
> > OSELAS.BSP-Pengutronix-Example.tar.bz2 (or a similar source)
> > ptxdist-2019.09.0.tar.bz2
> > OSELAS.Toolchain-2019.09.1.tar.bz2
> > 
> > The OSELAS website is linked as ptxdist.org that doesn't mention OSELAS
> > at all. There is a link to the toolchain but it's a different version.
> > I managed to find a BSP for 'Generic' but it's 9 years old.
> 
> +1
> 
> That has to be improved.  There's no easy accessible single platform example 
> BSP to get your feet wet, and DistroKit is nice for supporting some quite 
> common SBCs, but is already quite complicated.

would something like this be sufficient?

diff --git a/doc/environment.rst b/doc/environment.rst
index 18aecec4e..670f86a48 100644
--- a/doc/environment.rst
+++ b/doc/environment.rst
@@ -17,10 +17,13 @@ components which are available to the public).
 In order to build |ptxdistBSPName|, the following source archives have to be 
available
 on the development host:
 
- * ptxdist-|ptxdistVendorVersion|.tar.bz2
+ * `ptxdist-|ptxdistVendorVersion|.tar.bz2 
`_
  * |ptxdistBSPName|.tar.bz2 (or a similar source)
- * ptxdist-|oselasTCNVendorptxdistversion|.tar.bz2
- * OSELAS.Toolchain-|oselasTCNVendorVersion|.tar.bz2
+
+Additionally, those source archives are needed to build toolchain:
+
+ * `OSELAS.Toolchain-|oselasTCNVendorVersion|.tar.bz2 
`_
+ * `ptxdist-|oselasTCNVendorptxdistversion|.tar.bz2 
`_
 
 Main Parts of PTXdist
 ~

> What's also lacking: some template or documentation on how to start from 
> scratch without deriving your work from pre existing BSPs.  I tried that 
> lately and put the steps online:
> 
> https://gist.github.com/LeSpocky/31af75ab63bc6f35fd71d53f06b5a50e
> 
> I dropped that link in IRC, but I got no clear signal if that's something to 
> put into the official ptxdist documentation?

I'm hereby sending a clear signal to maintainers that I'd like to
see something like that in the documentation ;-)

> > The documentation says having a BSP is optional, but there's no
> > instructions on how to work without one. It's probably not even a good
> > idea to mention this, perhaps ptxdist should provide a skeleton project?
> > 
> > I get the distinct impression the right solution here is to use
> > DistroKit as the starter BSP. This isn't mentioned anywhere in the
> > documentation but is something I found out about on IRC.
> > 
> > I'm also not a fan of suggesting installing ptxdist. It seems strange to
> > lock yourself to a repository of packages that you will need to tweak
> > and fix. I would like to see more support for running it in tree.
> 
> Maybe this is some kind of misunderstanding.  You would never touch the 
> distributed/installed ptxdist, but "overwrite" things by placing package 
> rules 
> with the same name in your BSP.  For example if you need a version bump for 
> libfoo, you could copy /usr/local/lib/ptxdist-2100.01.0/rules/libfoo.make to 
> your BSP at rules/libfoo.make and adapt it to your needs.

or you can even put that into post/ rules in case just upgrading
package is sufficient (no configure, etc. changes)

Re: [ptxdist] ptxdist initial impressions

[Alexander Dahl]

Hello Jookia,

as a long term user not affiliated with pengutronix I have some comments which 
might help.  See below.

Am Mittwoch, 17. Mai 2023, 18:11:31 CEST schrieb Jookia:
> Hello ptxdist friends,
> 
> I spent some time over the past few days trying to use ptxdist and
> documenting all the pain I hit in the process. This isn't a criticism
> more just anecdotal data of my experience. Hopefully it's useful to
> smooth down some rough edges.
> 
> I'm going to write this up in a psuedo-markdown format in sections,
> sorry if that breaks anyone's workflow.
> 
> 
> Documentation
> -
> 
> First up: Documentation is quite good aside from the 'Getting a working
> environment' section. I will be focusing on this today since I think it's
> probably the most critical problem with ptxdist at the moment: I expect
> that by the end of following this chapter I should have a bootable image
> of some kind and commands for using it, such as with qemu.
> 
> The first issue is that I'm told to downloaded files, but I'm not linked
> to them, quoting:
> 
> ptxdist-2023.05.0.tar.bz2
> OSELAS.BSP-Pengutronix-Example.tar.bz2 (or a similar source)
> ptxdist-2019.09.0.tar.bz2
> OSELAS.Toolchain-2019.09.1.tar.bz2
> 
> The OSELAS website is linked as ptxdist.org that doesn't mention OSELAS
> at all. There is a link to the toolchain but it's a different version.
> I managed to find a BSP for 'Generic' but it's 9 years old.

+1

That has to be improved.  There's no easy accessible single platform example 
BSP to get your feet wet, and DistroKit is nice for supporting some quite 
common SBCs, but is already quite complicated.

What's also lacking: some template or documentation on how to start from 
scratch without deriving your work from pre existing BSPs.  I tried that 
lately and put the steps online:

https://gist.github.com/LeSpocky/31af75ab63bc6f35fd71d53f06b5a50e

I dropped that link in IRC, but I got no clear signal if that's something to 
put into the official ptxdist documentation?

> The documentation says having a BSP is optional, but there's no
> instructions on how to work without one. It's probably not even a good
> idea to mention this, perhaps ptxdist should provide a skeleton project?
> 
> I get the distinct impression the right solution here is to use
> DistroKit as the starter BSP. This isn't mentioned anywhere in the
> documentation but is something I found out about on IRC.
> 
> I'm also not a fan of suggesting installing ptxdist. It seems strange to
> lock yourself to a repository of packages that you will need to tweak
> and fix. I would like to see more support for running it in tree.

Maybe this is some kind of misunderstanding.  You would never touch the 
distributed/installed ptxdist, but "overwrite" things by placing package rules 
with the same name in your BSP.  For example if you need a version bump for 
libfoo, you could copy /usr/local/lib/ptxdist-2100.01.0/rules/libfoo.make to 
your BSP at rules/libfoo.make and adapt it to your needs.

> Likewise with installing a toolchain it seems a bit weird to suggest
> Debian packages and going through how to extract these, but not offer a
> tarball.
> 
> The page also talks about building a toolchain. This is kind of
> a confusing process and seems way out of scope for getting up and
> running. If things are this difficult maybe ptxdist should provide a
> container or something with batteries included.
> 
> My setup
> 
> 
> That said I managed to get a setup going like this:
> 
> ```
> git clone https://git.pengutronix.de/git/ptxdist
> git clone https://git.pengutronix.de/git/DistroKit
> wget
> https://toolchains.bootlin.com/downloads/releases/toolchains/armv7-eabihf/t
> arballs/armv7-eabihf--glibc--bleeding-edge-2022.08-1.tar.bz2 tar -xvf
> armv7-eabihf--glibc--bleeding-edge-2022.08-1.tar.bz2
> cd ptxdist
> ./autogen.sh
> ./configure
> make -j8
> cd ..
> export PATH=$PATH:$PWD/ptxdist/bin
> mkdir project
> cd project
> ln -s ../DistroKit base
> ptxdist platform base/configs/platform-v7a/platformconfig
> ptxdist toolchain ../armv7-eabihf--glibc--bleeding-edge-2022.08-1/bin
> ptxdist migrate
> ptxdist menu
> # go to platformconfig -> architecture -> toolchain
> # set toolchain to arm-buildroot-linux-gnueabihf
> # disable toolchain version checks
> # set glibc version to 2.35
> rm selected_platformconfig
> ln -s configs/platform-v7a/platformconfig selected_platformconfig

I think this is a misunderstanding from using layers.  First you select base/
configs/platform-v7a/platformconfig as your platformconfig.  Then with ptxdist 
menu -> platformconfig you make changes, but those are not put to the base 
layer, but to your top layer.  ptxdist creates configs/platform-v7a/
platformconfig and configs/platform-v7a/platformconfig.diff then.  On the next 
invocation ptxdist detects your selected_platformconfig points to the base 
layer although you have a different platformconfig in the top layer, and it 
probably reports that in some way.  You 

Re: [ptxdist] ptxdist initial impressions

[Ladislav Michl]

On Thu, May 18, 2023 at 10:23:30PM +1000, Jookia wrote:
> On Thu, May 18, 2023 at 11:28:07AM +0200, Ladislav Michl wrote:
> > Hi,
> 
> Hello,
> 
> > From all the previous lenghty explanation I didn't get what you did
> > wrong, so I cannot provide any advice. I normally just download and
> > run build inside BSP :)
> 
> It's a lot easier if you're used to the quirks so far.
> 
> > > I don't quite understand what's gained by having an installable ptxdist
> > > and a DistroKit BSP if you're just going to modify both when building
> > > your system. Maybe someone could explain this to me?
> > 
> > Well, you do not need to modify PTXdist itself, unless there is a bug
> > somewhere...
> 
> There's always bugs or out of date packages, so you have to modify ptxdist to
> fix those.

Yes and you do that by overriding rule locally (production use) or by fixing
upstream (devel), so others can benefit from those changes as well. Then you
just migrate your bsp to a newer version.

Re: [ptxdist] ptxdist initial impressions

[Jookia]

On Thu, May 18, 2023 at 11:28:07AM +0200, Ladislav Michl wrote:
> Hi,

Hello,

> From all the previous lenghty explanation I didn't get what you did
> wrong, so I cannot provide any advice. I normally just download and
> run build inside BSP :)

It's a lot easier if you're used to the quirks so far.

> > I don't quite understand what's gained by having an installable ptxdist
> > and a DistroKit BSP if you're just going to modify both when building
> > your system. Maybe someone could explain this to me?
> 
> Well, you do not need to modify PTXdist itself, unless there is a bug
> somewhere...

There's always bugs or out of date packages, so you have to modify ptxdist to
fix those.

> PTXdist already checks if it is build at each run, so actually builing
> it is not that hard. I remember we discussed it few times on the IRC,
> but there was never a conclusion. Parhaps we can discuss it now?

What does this mean?

>   ladis

Jookia. :)

Re: [ptxdist] ptxdist initial impressions

[Ladislav Michl]

Hi,

On Thu, May 18, 2023 at 02:11:31AM +1000, Jookia wrote:
> Hello ptxdist friends,
[snip]
> Compared to buildroot's 'download and run make menuconfig' ptxdist has a
> pretty hard hill to climb, especially with having to download toolchains
> and BSPs and dealing with some of the package bugs.

>From all the previous lenghty explanation I didn't get what you did
wrong, so I cannot provide any advice. I normally just download and
run build inside BSP :)

> I don't quite understand what's gained by having an installable ptxdist
> and a DistroKit BSP if you're just going to modify both when building
> your system. Maybe someone could explain this to me?

Well, you do not need to modify PTXdist itself, unless there is a bug
somewhere...

But having to install it (or any such build tool in general) is
something Rob Landley whined about two decades ago and it is one of
those rare opinions I agree with him. However I'm unable to find
his post anymore, so I cannot properly quote him.

PTXdist already checks if it is build at each run, so actually builing
it is not that hard. I remember we discussed it few times on the IRC,
but there was never a conclusion. Parhaps we can discuss it now?

ladis