Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-11 Thread Roberto E. Vargas Caballero
Hi,

After reading the opinion of the people in this thread,
I think the best option is to merge the sbase and ubase
repositories and having a mechanism in the build system
to select the set of tools to be included in the build.
The main drawback of this is that the build system will
be more complex than the one that we have now.

I am going to import the history of ubase into sbase and
I will push to a new branch. Meanwhile, I am going to
keep frozen the pending patches that we have in the hackers
mailing list until the migration is done. I would ask
later to the authors to resend them adjusted to the structure
of the project. Please, raise your concerns if you consider
that something else should be done.

More mails are expected in specific threads while we
progress with the migration.

Regards,



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread NRK
> Everything in the quoted part seems personal preference.

I've been managing my vim plugins with git submodules via vim's builtin
"packadd" directory structure (~/.config/vim/pack/plugins/start/...).
And I can tell you that trying to do anything remotely non-trivial with
them is simply not a pleasant experience.

Recent example, there's no way to simply rename a submodule (the --name
argument to `submodule add`). You need to edit `.gitmodule` file,
edit `.git/config` file along with updating the .git symlinks inside the
module and `.git/module` and probably who knows what else. At the end, I
just `rm -rf` everything and re-cloned all the modules.

Maybe in the future git submodule's UI will be better but as of now it's
a PITA to use.

- NRK



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Mattias Andrée
On Sat, 09 Mar 2024 17:28:49 +0100
Elie Le Vaillant  wrote:

> Страхиња Радић  wrote:
> > Compiling all programs into one binary is currently an option, and IMHO it 
> > should remain an option.  
> 
> I fully agree. However, the single binary situation should be improved.
> 
> > Great, combine the two versions of libutil into a single, separate
> > libutil repository  
> 
> I'm not sure whether or not this is a good idea, because it makes
> sbase and ubase dependant upon a separate repository, which needs to
> be present in the parent directory for it to build. It'd also make
> sbase development cumbersome, because we very frequently change
> libutil when we change sbase. Both are developed as one single
> project, and patches reflect this. libutil should not be isolated I
> think.
> 
> > then have a directory hierarchy like this:
> > 
> > corebox
> > ├──sbase (portable only)  \
> > ├──ubase (nonportable) depend on libutil.so and/or libutil.a
> > ├──xbase (non-POSIX)  /
> > └──libutil (option to produce .so and/or .a)  
> 
> ubase is not only nonportable, it is _linux-specific_. It is also
> non-POSIX. I think ubase should be renamed to reflect this. The
> distinction between POSIX/non-POSIX is, I think, not very useful. As

There are also multiple standards, not just POSIX. For example
tar, true, false are not POSIX (tar was removed from POSIX, and
true and false are defined only as shell built-in in POSIX), but they
are defined in LSB which a propular, but it's a Linux specific standard.
Most of POSIX but not all of POSIX is also defined by LSB.

> Mattias said, pure POSIX is quite cumbersome, and not very descriptive
> as of what you can expect from it. sh and vi are POSIX, but out-of-scope
> for sbase (from the TODO), whereas sponge is crucial for sbase (it
> allows simpler implementation of -i for sed, which _is_ POSIX, or the
> -o flag for sort (POSIX too)) and would thus be excluded from sbase
> and put into xbase.

sed -i is not POSIX.

> 
> The solution Mattias proposed (having one big repository, a portable
> subdir, a linux (and maybe others in the future, like openbsd) subdir
> and a Makefile which includes more descriptive sets than POSIX/non-POSIX
> (well, it _can_ be used, but it is not enough)) is I think the best to
> fix the problem of libutil duplication/drifting away of versions. It
> also allows a broader scope without impeding on the goals of sucklessness.
> 
> One supplementary question, more in line with the original question asked
> by Roberto E. Vargas Caballero, is: would awk and sh be out-of-scope?
> Should we rather try to implement extensions to awk, or follow the 
> specification
> as strictly possible? Should we implement POSIX sh, or some other shell, such 
> as rc?
> Or is it out-of-scope for us to implement a full-blown shell? I really am
> not sure.

I don't think there is any reason that sbase should implement
all of the standard utilities you need, I think it should only
be the small tools that you can reasonably write in one file.
Large and complicated programs like sh should be it's own project.

> 
> Regards,
> Elie Le Vaillant
> 




Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Страхиња Радић
On 24/03/09 05:28PM, Elie Le Vaillant wrote:
> Or is it out-of-scope for us to implement a full-blown shell? I really am
> not sure.

I think it would be interesting to have "official" suckless versions of all of 
the mentioned programs, which would at minimum implement their descriptions 
from POSIX while keeping suckless principles. In cases where POSIX is vague, 
there is nothing stopping the authors to implement their take on the acceptable 
behavior of those programs.



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Elie Le Vaillant
Страхиња Радић  wrote:
> Compiling all programs into one binary is currently an option, and IMHO it 
> should remain an option.

I fully agree. However, the single binary situation should be improved.

> Great, combine the two versions of libutil into a single, separate
> libutil repository

I'm not sure whether or not this is a good idea, because it makes
sbase and ubase dependant upon a separate repository, which needs to
be present in the parent directory for it to build. It'd also make
sbase development cumbersome, because we very frequently change
libutil when we change sbase. Both are developed as one single
project, and patches reflect this. libutil should not be isolated I
think.

> then have a directory hierarchy like this:
> 
> corebox
> ├──sbase (portable only)  \
> ├──ubase (nonportable) depend on libutil.so and/or libutil.a
> ├──xbase (non-POSIX)  /
> └──libutil (option to produce .so and/or .a)

ubase is not only nonportable, it is _linux-specific_. It is also
non-POSIX. I think ubase should be renamed to reflect this. The
distinction between POSIX/non-POSIX is, I think, not very useful. As
Mattias said, pure POSIX is quite cumbersome, and not very descriptive
as of what you can expect from it. sh and vi are POSIX, but out-of-scope
for sbase (from the TODO), whereas sponge is crucial for sbase (it
allows simpler implementation of -i for sed, which _is_ POSIX, or the
-o flag for sort (POSIX too)) and would thus be excluded from sbase
and put into xbase.

The solution Mattias proposed (having one big repository, a portable
subdir, a linux (and maybe others in the future, like openbsd) subdir
and a Makefile which includes more descriptive sets than POSIX/non-POSIX
(well, it _can_ be used, but it is not enough)) is I think the best to
fix the problem of libutil duplication/drifting away of versions. It
also allows a broader scope without impeding on the goals of sucklessness.

One supplementary question, more in line with the original question asked
by Roberto E. Vargas Caballero, is: would awk and sh be out-of-scope?
Should we rather try to implement extensions to awk, or follow the specification
as strictly possible? Should we implement POSIX sh, or some other shell, such 
as rc?
Or is it out-of-scope for us to implement a full-blown shell? I really am
not sure.

Regards,
Elie Le Vaillant



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Mattias Andrée
On Sat, 9 Mar 2024 14:53:07 +0100
Страхиња Радић  wrote:

> On 24/03/09 12:59AM, Mattias Andrée wrote:
> > I agree, a single repo (or alternatively making libutil it's own repo) is
> > necessary if we want one binary, and I think we do.  
> 
> Compiling all programs into one binary is currently an option, and IMHO it 
> should remain an option. In my own toy distro[1] based on Musl-LFS and using 
> sbase and ubase I compile all programs from {s,u}base separately.
> 
> The reasons why I consider that beneficial over a single executable include, 
> but are not limited to:
> 
> - Easier to maintain: if an administrator decides a utility is unnecessary or 
>   shouldn't be available, it comes down to rm-ing the file vs recompilation 
> of 
>   the entire *box.
> 
> - More robust: in case of disk corruption, all of the utilities are 
> unavailable 
>   vs only those affected.
> 
> - Fine-grained control: separate programs can be compiled using specific
>   compilation options (eg. -g -O0) vs all of the programs sharing compilation 
>   options.
> 
> etc.
> 
> 
> > Even if submodules was possible, I do not think they are a good solution.  
> 
> What makes the git submodules not possible?

I haven't actually checked, but it safe to assume that if it is not
already a problem it very well could become one (and it probably is).
If the two libutil implementations have functions with the same names
they cannot both be linked into the same binary. A "solution" would
be to use weak linkage, but then you can run into problems when the
functions that share name do not share behaviour (e.g. have different
prototype or return type).

> 
> 
> > Using submodules is unpleasant and pointless since all could is under our
> > control. I think submodules should only be used for code that you do not
> > have control over but need the source code for. Either you have separate
> > repos and have normal compile time dependencies (require that the libraries
> > are installed) or you put everything in one place, one repo.  
> 
> Everything in the quoted part seems personal preference. Git submodules offer 
> a 
> way to easily establish a hierarchy of git repositories while keeping them as 
> separate "units".

Of course these are personal preferences. There aren't any technical problems
using git submodules assuming you can put everything in the same repo.

> 
> So the libutil differs in sbase and ubase. Great, combine the two versions of 
> libutil into a single, separate libutil repository, then have a directory 
> hierarchy like this:
> 
> corebox
> ├──sbase (portable only)  \
> ├──ubase (nonportable) depend on libutil.so and/or libutil.a
> ├──xbase (non-POSIX)  /
> └──libutil (option to produce .so and/or .a)
> 
> 
> 
> [1]: https://strahinja.srht.site/galeb
> 




Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Страхиња Радић
On 24/03/09 12:59AM, Mattias Andrée wrote:
> I agree, a single repo (or alternatively making libutil it's own repo) is
> necessary if we want one binary, and I think we do.

Compiling all programs into one binary is currently an option, and IMHO it 
should remain an option. In my own toy distro[1] based on Musl-LFS and using 
sbase and ubase I compile all programs from {s,u}base separately.

The reasons why I consider that beneficial over a single executable include, 
but are not limited to:

- Easier to maintain: if an administrator decides a utility is unnecessary or 
  shouldn't be available, it comes down to rm-ing the file vs recompilation of 
  the entire *box.

- More robust: in case of disk corruption, all of the utilities are unavailable 
  vs only those affected.

- Fine-grained control: separate programs can be compiled using specific
  compilation options (eg. -g -O0) vs all of the programs sharing compilation 
  options.

etc.


> Even if submodules was possible, I do not think they are a good solution.

What makes the git submodules not possible?


> Using submodules is unpleasant and pointless since all could is under our
> control. I think submodules should only be used for code that you do not
> have control over but need the source code for. Either you have separate
> repos and have normal compile time dependencies (require that the libraries
> are installed) or you put everything in one place, one repo.

Everything in the quoted part seems personal preference. Git submodules offer a 
way to easily establish a hierarchy of git repositories while keeping them as 
separate "units".

So the libutil differs in sbase and ubase. Great, combine the two versions of 
libutil into a single, separate libutil repository, then have a directory 
hierarchy like this:

corebox
├──sbase (portable only)  \
├──ubase (nonportable) depend on libutil.so and/or libutil.a
├──xbase (non-POSIX)  /
└──libutil (option to produce .so and/or .a)



[1]: https://strahinja.srht.site/galeb



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Mattias Andrée
On Sat, 09 Mar 2024 12:10:28 +0100
Eolien55  wrote:

> Mattias Andrée  wrote:
> > I think there should be one directory called "portable" containing only 
> > tools
> > from sbase, and one directory called "linux" containing the tools from ubase
> > and maybe even symlinks to the tools in "portable". This structure would 
> > allow
> > us to add implementations for other operating systems as well. If we add
> > symlinks to the tools in "portable" to "linux", each directory could have
> > it's own makefile. But I'm not sure this is preferable over a single 
> > Makefile
> > in the root directory.  
> 
> This is a great idea! Your mail on the other branch is a great idea too.
> I think we should have platform-specific libutil for unportable functions
> in ubase's libutil (so, linux/libutil, openbsd/libutil and so on, if we
> do actually add implementations for other OSes), and a top-level libutil
> too.
> 
> This could maybe allow adding platform-agnostic implementations of some utils
> (not all because some APIs are so different that it requires full rewrites,
> but maybe some, such as clear, stty, tput, or dd maybe).
> 
> I will start hacking on it, and will post the git repository for it
> when it builds correctly.
> 
> I'm not sure how you combine two repositories into one while keeping the
> history though.

git init .

git pull git://git.suckless.org/sbase
git branch -M master sbase
mkdir sbase
mv * .gitignore sbase
git add .
git commit -m 'Move all of sbase into its own directory'

git checkout --orphan master
git rm -rf .

git pull git://git.suckless.org/ubase
git branch -M master ubase
mkdir ubase
mv * .gitignore ubase
git add .
git commit -m 'Move all of ubase into its own directory'

git checkout --orphan master
git rm -rf .
git pull --allow-unrelated-histories . sbase
git pull --allow-unrelated-histories . ubase


> 
> Regards,
> Elie Le Vaillant
> 




Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-09 Thread Eolien55
Mattias Andrée  wrote:
> I think there should be one directory called "portable" containing only tools
> from sbase, and one directory called "linux" containing the tools from ubase
> and maybe even symlinks to the tools in "portable". This structure would allow
> us to add implementations for other operating systems as well. If we add
> symlinks to the tools in "portable" to "linux", each directory could have
> it's own makefile. But I'm not sure this is preferable over a single Makefile
> in the root directory.

This is a great idea! Your mail on the other branch is a great idea too.
I think we should have platform-specific libutil for unportable functions
in ubase's libutil (so, linux/libutil, openbsd/libutil and so on, if we
do actually add implementations for other OSes), and a top-level libutil
too.

This could maybe allow adding platform-agnostic implementations of some utils
(not all because some APIs are so different that it requires full rewrites,
but maybe some, such as clear, stty, tput, or dd maybe).

I will start hacking on it, and will post the git repository for it
when it builds correctly.

I'm not sure how you combine two repositories into one while keeping the
history though.

Regards,
Elie Le Vaillant



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-08 Thread Mattias Andrée
On Fri, 08 Mar 2024 23:33:12 +0100
Eolien55  wrote:

> Страхиња Радић  wrote:
> > The problem of having separate *box executables could be solved by creating 
> > an 
> > "umbrella" *box project, perhaps having sbase, ubase and 
> > [insert_letter]base as 
> > git submodules, and deciding what to build based on the contents of 
> > config.mk.  
> 
> The problem is that sbase and ubase each include a version of libutil, whith 
> some
> functions which are the same, and some other wich serve the same function but
> vary in implementation due to version history.
> 
> git submodules are kinda meh I think for this problem, because it wouldn't 
> change
> the problem of source code duplication (and of versions drifting apart) 
> between
> the 2 projects, as libutil is part of both sbase and ubase trees (not mereley 
> the
> umbrella-box project).
> 
> I believe we should put what are currently sbase and ubsase in a single git 
> repository,
> sharing all that is portably sharable, but still separating utilities that 
> are portable
> from linux-specific ones.
> 
> I think sbase and ubase should try to provide useful, well-implemented, 
> suckless
> utilities. If we want POSIX utilities, let's add them! But I don't think we 
> should
> restrain us from doing that. sbase's sponge and cols is so useful I'm 
> constantly
> using them, even though I normally use busybox.
> 
> Regards,
> Elie Le Vaillant
> 

I agree, a single repo (or alternatively making libutil it's own repo) is
necessary if we want one binary, and I think we do.

Even if submodules was possible, I do not think they are a good solution.
Using submodules is unpleasant and pointless since all could is under our
control. I think submodules should only be used for code that you do not
have control over but need the source code for. Either you have separate
repos and have normal compile time dependencies (require that the libraries
are installed) or you put everything in one place, one repo.

I see the separation of sbase and ubase into two repositories as basically
equivalent to the a single repo with a sbase directory and a ubase directory,
except better when it comes to tagging new versions, but since there reason
to have separate releases for these, it doesn't really make a difference.
So simply putting sbase and ubase in two different directories in the same
repo, and then put a makefile there to build all of it into one binary would
be step up, of course there would be some linkage problems and making them
share libutil would be the next step up. Of course, it should be possible
to select if you want ubase included in your binary or not, is the point
of the separation in the first place.

I think there should be one directory called "portable" containing only tools
from sbase, and one directory called "linux" containing the tools from ubase
and maybe even symlinks to the tools in "portable". This structure would allow
us to add implementations for other operating systems as well. If we add
symlinks to the tools in "portable" to "linux", each directory could have
it's own makefile. But I'm not sure this is preferable over a single Makefile
in the root directory.

As mentioned in an other branch of this conversation, I think we should
have a base with only the POSIX tools but than have additional optional
tools, which could be group into overlapping categories, as you can select
what you want on your system.


Best regards,
Mattias Andrée



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-08 Thread Eolien55
Страхиња Радић  wrote:
> The problem of having separate *box executables could be solved by creating 
> an 
> "umbrella" *box project, perhaps having sbase, ubase and [insert_letter]base 
> as 
> git submodules, and deciding what to build based on the contents of config.mk.

The problem is that sbase and ubase each include a version of libutil, whith 
some
functions which are the same, and some other wich serve the same function but
vary in implementation due to version history.

git submodules are kinda meh I think for this problem, because it wouldn't 
change
the problem of source code duplication (and of versions drifting apart) between
the 2 projects, as libutil is part of both sbase and ubase trees (not mereley 
the
umbrella-box project).

I believe we should put what are currently sbase and ubsase in a single git 
repository,
sharing all that is portably sharable, but still separating utilities that are 
portable
from linux-specific ones.

I think sbase and ubase should try to provide useful, well-implemented, suckless
utilities. If we want POSIX utilities, let's add them! But I don't think we 
should
restrain us from doing that. sbase's sponge and cols is so useful I'm constantly
using them, even though I normally use busybox.

Regards,
Elie Le Vaillant



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-08 Thread Страхиња Радић
On 24/03/08 06:40AM, Roberto E. Vargas Caballero wrote:
> I would like to move the discussion here and see what alternatives
> we have and how to proceed in this case.

IMHO, if the intention behind sbase was to provide a minimal portable POSIX 
utilities implementation, anything not fitting that definition should be moved 
out of sbase.

A separate project, perhaps called [insert_letter]base, could cover the 
utilities not in POSIX which are common in Unix-like systems.

The problem of having separate *box executables could be solved by creating an 
"umbrella" *box project, perhaps having sbase, ubase and [insert_letter]base as 
git submodules, and deciding what to build based on the contents of config.mk.



Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-08 Thread Mattias Andrée
I think we should keep the implementation of each tool as minimal as
possible, but POSIX-complete, and of course common tools such as
install(1) and tar(1). However, actually using a system that is
nothing more than POSIX is very cumbersome. And I think it is a better
solution to implement non-standard tools when possible to address
usability issues, e.g. implementing sponge(1) instead of -i for sed(1).

However, if the system isn't actually intended for be used
interactively via the command line, e.g. on embedded devices
or a service running in a container, there is no for non-standard
tools such as sponge(1), and it ought to be easy to select what
you want on your system. I suggest that tools be group into a
few categories (unless they are organised into separate directories
I see no reason one tool couldn't be in multiple categories).

These categories could for example:
- "posix" for all POSIX tools,
- "lsb" for all LSB tools
- "users" for account management
- "extra" for other common tools
- "common" for "posix", "lsb", "users", and "extra"
- "interative" for tools that only make since of when
  a lot via the terminal
- "all" for every implemented tool

Maybe these should also be subdivided into "portable" and "linux".

The user could then specify the tools to include either by
setting BIN when running make(1) or by saying "yes" or "no" to
each category (of course each category would have a default
option), e.g. POSIX=yes INTERACTIVE=no.

Best regards,
Mattias Andrée


On Fri, 08 Mar 2024 11:36:27 +0100
Elie Le Vaillant  wrote:

> Hi,
> 
> I think one of the main current issues with the current
> organization of sbase's and ubase's code, is that while
> they share parts of code (some parts of libutil are shared),
> they do not actually have it in common. As a result, changes
> to shared parts of libutil in sbase are not reflected in ubase,
> and vice-versa.
> 
> Some parts of ubase's libutil are not portable, so indeed it
> makes sense that they are ubase-specific. But some, such as
> recurse, strtonum, strl*, ealloc, eprinf, and maybe others,
> serve the same exact function as in sbase, but sometimes
> vary in implementation, because they didn't receive the same
> patches.
> 
> So I wonder:
> - Is this a problem that needs fixing? (I think yes)
> - How do we fix it?
> 
> We could sync both periodically, applying whichever patch change
> both *base's libutil to both.
> 
> Another idea could be to have both in the same git repository,
> allowing libutil (and possibily more code, like libutf if we
> ever need to) to be shared between them both without syncing
> them back and forth. My idea would be something like this:
> 
> sbase/
>   portable/
> ls.c
> cols.c
> ...
>   unportable/
> ps.c
> kill.c
> ...
>   libutf
>   libutil/
> portable
> unportable
>   Makefile
> 
> This could fix the "multiple -box" problems. This would require
> rewriting some parts of the Makefile (for example, having PORTABLEBIN
> and UNPORTABLEBIN to select whether or not we want the unportable
> utilities; the mkbox script also), and could also provide a solution for
> the "moretools" repo by having it being a separate directory in this
> hypothetical repository.
> 
> Also I'm not sure whether we should keep the goal of being POSIX-compliant.
> ls doesn't columnate, we have (non-standard) cols to do this. sed doesn't
> have the -i flag, we have sponge for this. cron isn't specified by POSIX,
> only crontab is. Maybe toybox roadmap's section on POSIX is relevant:
> https://landley.net/toybox/roadmap.html
> 
> I think we should try and implement a minimal Unix-like userspace,
> and allow ourselves some freedom on what to implement. We already
> do this with sponge and cols. On ubase it is true also, with ctrlaltdel
> for example. I do not see why not do it more.
> 
> Overall I think bringing everything in the same repository, with
> what is now sbase and ubase in separate directories rather than
> separate repositories, would fix both the current situation, and
> allow for a "sextra"/"uextra" directory for supplementary tools.
> 
> Mattias Andrée already proposed this back when he proposed a patch
> for shuf(1):
> > No, we don't really need shuf(1) in sbase, but I think we
> > should have a suckless implementation available, it can be
> > a useful utility. I have a few more utilities I fund useful
> > but I haven't bothered to set up a repository yet. [...]
> > I think it might be a good idea to have sextra for portable
> > utilities and uextra for unportable utilities, if you have
> > any other suggestions I would like to hear them.   
> 
> I think this could fix the current situation, with code on
> different versions split between 2 repositories and ultimately
> 2 -box binaries, and allow a broader scope without impeding the
> goals of minimalness of sbase/ubase.
> 
> Regards,
> Elie Le Vaillant
> 




Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-08 Thread Randy Palamar
Elie Le Vaillant  wrote:
> Another idea could be to have both in the same git repository,
> [...]

This would be my idea as well. It also wouldn't be that difficult
to let people pick and choose which sets of tools to include in the
final -box via config.mk or similar. I would stick with only the
high-level sets and leave the omission of individual tools up to
the "user" (I already do this because I prefer a few tools from
OpenBSD even when using Linux).

> Also I'm not sure whether we should keep the goal of being POSIX-compliant.

I also agree with this. POSIX, like everything designed by committees,
is full of bad and/or incomplete ideas. I think that having a set
of tools that isn't GNU/spaghetti but still supports some of the
good non-POSIX options (for example sponge, xargs -P, working tar,
etc.) would be nice to have.

- Randy

-- 
https://rnpnr.xyz/
GPG Fingerprint: B8F0 CF4C B6E9 415C 1B27 A8C4 C8D2 F782 86DF 2DC5


signature.asc
Description: PGP signature


Re: [dev] [sbase] Defining scope of sbase and ubase

2024-03-08 Thread Elie Le Vaillant
Hi,

I think one of the main current issues with the current
organization of sbase's and ubase's code, is that while
they share parts of code (some parts of libutil are shared),
they do not actually have it in common. As a result, changes
to shared parts of libutil in sbase are not reflected in ubase,
and vice-versa.

Some parts of ubase's libutil are not portable, so indeed it
makes sense that they are ubase-specific. But some, such as
recurse, strtonum, strl*, ealloc, eprinf, and maybe others,
serve the same exact function as in sbase, but sometimes
vary in implementation, because they didn't receive the same
patches.

So I wonder:
- Is this a problem that needs fixing? (I think yes)
- How do we fix it?

We could sync both periodically, applying whichever patch change
both *base's libutil to both.

Another idea could be to have both in the same git repository,
allowing libutil (and possibily more code, like libutf if we
ever need to) to be shared between them both without syncing
them back and forth. My idea would be something like this:

sbase/
  portable/
ls.c
cols.c
...
  unportable/
ps.c
kill.c
...
  libutf
  libutil/
portable
unportable
  Makefile

This could fix the "multiple -box" problems. This would require
rewriting some parts of the Makefile (for example, having PORTABLEBIN
and UNPORTABLEBIN to select whether or not we want the unportable
utilities; the mkbox script also), and could also provide a solution for
the "moretools" repo by having it being a separate directory in this
hypothetical repository.

Also I'm not sure whether we should keep the goal of being POSIX-compliant.
ls doesn't columnate, we have (non-standard) cols to do this. sed doesn't
have the -i flag, we have sponge for this. cron isn't specified by POSIX,
only crontab is. Maybe toybox roadmap's section on POSIX is relevant:
https://landley.net/toybox/roadmap.html

I think we should try and implement a minimal Unix-like userspace,
and allow ourselves some freedom on what to implement. We already
do this with sponge and cols. On ubase it is true also, with ctrlaltdel
for example. I do not see why not do it more.

Overall I think bringing everything in the same repository, with
what is now sbase and ubase in separate directories rather than
separate repositories, would fix both the current situation, and
allow for a "sextra"/"uextra" directory for supplementary tools.

Mattias Andrée already proposed this back when he proposed a patch
for shuf(1):
> No, we don't really need shuf(1) in sbase, but I think we
> should have a suckless implementation available, it can be
> a useful utility. I have a few more utilities I fund useful
> but I haven't bothered to set up a repository yet. [...]
> I think it might be a good idea to have sextra for portable
> utilities and uextra for unportable utilities, if you have
> any other suggestions I would like to hear them. 

I think this could fix the current situation, with code on
different versions split between 2 repositories and ultimately
2 -box binaries, and allow a broader scope without impeding the
goals of minimalness of sbase/ubase.

Regards,
Elie Le Vaillant



[dev] [sbase] Defining scope of sbase and ubase

2024-03-07 Thread Roberto E. Vargas Caballero
Hi,

While reviewing several patches for sbase adding new tools
some discussion happened and I thought it was better to move the
discussion here.

The patches raised a concern about what should be the scope of
sbase. The idea of sbase was to provide a minimal portable POSIX
base, while having ubase for the POSIX commands that cannot be
implemented in a portable way.  Saying that, it is obvious that
there are programs in sbase that are not part of POSIX:

- md5sum
- sha256sum
- sha384sum
- sha512sum
- sponge
- sync
- tar
- install

and maybe some others. At this point is not clear to me what to do
with tools like tac or shuf. There was a small discussion about
this topic in the irc channel, and it was proposed to add a 3rd
repository to contain all these tools that are not part of POSIX
(a bit like the moretools package).  From my point of view, the
main drawback of it is that it requires a 3rd -box program (currently
we have sbase-box and ubase-box). The current situation it not good,
because the two -box are not sharing the library, and the disk space
is duplicated (main reason of -box is to minimize disk space for
restricted environments), and a 3rd -box would make the situation
even worse. But in the other hand, I don't want to add more non
POSIX tools in sbase (in fact, I personally would like to remove
the current non POSIX tools).

I would like to move the discussion here and see what alternatives
we have and how to proceed in this case.

Regards,