Re: [arch-dev-public] Moving gcc7 into [community] for CUDA

2018-05-28 Thread Allan McRae via arch-dev-public
On 29/05/18 11:00, Sven-Hendrik Haase wrote:
> Hey,
> 
> I would like to move gcc7 (based on the latest version 7 commit of gcc [0])
> into [community] because of cuda 9.2 and in return drop gcc54.
> 
> I tried to make it work with current gcc but to no avail. In earlier
> releases of cuda the incompatibilities could be patched with header hacks
> but not this time. I tested the whole setup with cuda 9.2 locally and it
> seems to work fine.
> 
> I just want to get somebody's ok on this as apparently not everyone is
> stoked about having yet another old version of gcc in there for some time.
> 
> Sven
> 
> [0]
> https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/gcc=44e0a03db1b5cabf6135f194e540d513cf959245
> .
> 

OK to add gcc7 given it will drop gcc54.

A


Re: [arch-dev-public] Automating package conflict resolution whenever possible (xorgproto/libxfont dependency conflict)

2018-02-14 Thread Allan McRae via arch-dev-public
On 14/02/18 22:28, Florian Pritz via arch-dev-public wrote:
> How about having this feature in pacman, maybe with an indicator if the
> package is still in a repository?

pacman -Qtd


Re: [arch-dev-public] Automating package conflict resolution whenever possible (xorgproto/libxfont dependency conflict)

2018-02-14 Thread Allan McRae via arch-dev-public
Just because I had to look up the details of this


- xorgproto replaces a lot of packages, including fontsproto:
 :: Replace fontsproto with extra/xorgproto? [Y/n]

- libxfont requires fontsproto, so this causes the following:
 :: libxfont: removing fontsproto breaks dependency 'fontsproto>=2.1.3'


- people with systems older than 2016-11-16, may have libxfont as
xorg-server depended on it until that time.  Now xorg-server depends on
libxfont2.

- libxfont2 does not replace libxfont, as it is a completely different API.

- libxfont was removed from the Arch repos somewhere in April or May
2017.  So nothing official depends on libxfont.


I don't think it is unreasonable to expect people to run "pacman -Qi
libxfont", see it was installed as a dependency and no package depends
on it and remove it.  There also does not seem to be a correct way of us
to handle this - joys of rolling release!


Funny thing...  people using yaourt probably removed this package as I
believe it highlights dependencies that are no longer needed after an
upgrade!

A


Re: [arch-dev-public] Package group between repositories

2018-01-04 Thread Allan McRae via arch-dev-public
On 04/01/18 22:02, Balló György via arch-dev-public wrote:
> Currently the 'xfce4-goodies' package group[1] is in split between [extra]
> and [community]. Most of its packages are in [extra], but 'ristretto' and
> 'xfce4-whiskermenu-plugin' are in [community].
> 
> My question is: is it okay, or should we avoid it by removing these two
> packages from the 'xfce4-goodies' group or moving them to [extra]?
> 
> [1] https://www.archlinux.org/groups/x86_64/xfce4-goodies/
> 

If a TU is maintaining the packages in [community], there is little
point moving them.

A


Re: [arch-dev-public] RFC: Dropping -DCMAKE_BUILD_TYPE from packages using cmake

2018-03-13 Thread Allan McRae via arch-dev-public
On 13/03/18 21:22, Jan Alexander Steffens via arch-dev-public wrote:
> For that matter, I'm all for putting an arch-configure helper into our
> autoconf package.

Don't muck around with vanilla packages.  Put it in another package
(devtools).

A


Re: [arch-dev-public] RFC: Dropping -DCMAKE_BUILD_TYPE from packages using cmake

2018-03-14 Thread Allan McRae via arch-dev-public
On 14/03/18 04:19, Bartłomiej Piotrowski via arch-dev-public wrote:
> On 2018-03-13 14:40, Allan McRae via arch-dev-public wrote:
>> On 13/03/18 21:22, Jan Alexander Steffens via arch-dev-public wrote:
>>> For that matter, I'm all for putting an arch-configure helper into our
>>> autoconf package.
>>
>> Don't muck around with vanilla packages.  Put it in another package
>> (devtools).
>>
>> A
>>
> 
> Makes no sense unless devtools become part of base-devel. It's hardly
> any meddling as it doesn't hinder possibility to use default configure
> script or meson binary.
> 

It is Arch specific, so should be in an Arch specific package.

A


Re: [arch-dev-public] .gnuog owned by root when using devtools in first place

2018-10-13 Thread Allan McRae via arch-dev-public
On 13/10/18 7:23 pm, NicoHood wrote:
> I ran extra-x86_64-build on a new system with an uninitialized gnupg
> keyring. I think it was responsible for creating a .~/gnupg directory,
> but with root privilegs.
> 
> I chowned it to my user, it is all good now. Maybe the script could be
> improved to check if the gnupg keyring was initialzed or not, to prevent
> such issues. I was confused in the first place.

If only there was a place to report bugs.  Somewhere where they could be
tracked...

A


Re: [arch-dev-public] TU application process

2018-11-06 Thread Allan McRae via arch-dev-public
On 6/11/18 7:54 pm, Christian Rebischke via arch-dev-public wrote:
> Hello everybody,
> 
> First of all, the following mail has nothing to do with the last two TU
> applications, it's a general view on the current TU application process.
> 
> I would like to propose a new process for TU applications due to several
> reasons:
> 

Read the TU bylaws.  It has specific instructions of where proposals
must be posted (hint: not here...).

A


Re: [arch-dev-public] TU application process

2018-11-06 Thread Allan McRae via arch-dev-public
On 6/11/18 8:15 pm, Bruno Pagani wrote:
> Le 06/11/2018 à 11:12, Christian Rebischke via arch-dev-public a écrit :
>> On Tue, Nov 06, 2018 at 08:09:03PM +1000, Allan McRae wrote:
>>> On 6/11/18 7:54 pm, Christian Rebischke via arch-dev-public wrote:
 Hello everybody,

 First of all, the following mail has nothing to do with the last two TU
 applications, it's a general view on the current TU application process.

 I would like to propose a new process for TU applications due to several
 reasons:

>>> Read the TU bylaws.  It has specific instructions of where proposals
>>> must be posted (hint: not here...).
>>>
>>> A
>> Hi Allan,
>>
>> This mail wasn't meant as proposal. It's just a general discussion about
>> this topic and people said in the TU IRC channel yesterday, that 
>> arch-dev-public would be the
>> right mailinglist for such discussion.
>>
>> chris
> Specifically, we are also interested in the input of devs, not just TUs.

Strange, given TUs are set-up to be an independently governed group from
developers...  But because you asked my opinion, I think a TU council is
a really, really, really bad idea.  No need to set some TUs above
others.  We have never had a formal hierarchy in the developers (apart
from our glorious leader), and are instead run by those who step up to
lead what needs done. I believe that this is what makes Arch work, and
governance would be detrimental to the distribution as a whole.

Personally, I'd get rid of all quorum for electing a TU, and make
inactive TUs be measured purely on the basis of package updating.  Most
TU application discussions are inane beyond the customary package
review.  And when someone applies and their packages are very bad, their
sponsor should be held in shame.

Finally, I don't want to hear what the minions are up to!  Get back to
your own mailing list. :)

A


Re: [arch-dev-public] Mongodb and SSPL

2019-01-16 Thread Allan McRae via arch-dev-public
On 17/1/19 12:02 am, Morten Linderud via arch-dev-public wrote:
> Yo!
> 
> As probably some of you have realized, there is a discussion regarding mongodb
> and the relicense from AGPLv3 to SSPLv1. 



> Under section "5. Conveying Modified Source Versions" the most relevant part 
> for
> us is section "a)".
> 
> a) The work must carry prominent notices stating that you modified it,
> and giving a relevant date.
> 
> Which means we need to give some heads-up that the source is changed.
> 

I looked at what changes are made:


  # Broken tls13 support, removing to fix build
  sed -i '/counts.tls13/d' src/mongo/util/net/ssl_manager_openssl.cpp

So I'd agree that is modified enough that the rest of the conditions apply.

My conclusion is, having this package in the repos would require too
much interpretation of a non-standard license to ensure compliance.
Drop the package.

Allan


Re: [arch-dev-public] Proposal: minimal base system

2019-01-21 Thread Allan McRae via arch-dev-public
On 22/1/19 8:03 am, Levente Polyak via arch-dev-public wrote:
> Everything that won’t be part of base-system needs to be added as a
> dependency to all requiring packages; alternatively don't omit any first
> level runtime dependencies at all.
> 
> This package should only depend on strictly required explicit packages
> to get a working minimal Arch Linux system.
> 
> The proposed end result is:
> - base: convenient helper group for quickly getting a working system
>   when installing, must include the new base-system package
> - base-system: package defining the minimum dependencies for a working
>   base runtime

I think the proposal is OK.  I'm not comfortable with our line about
base group packages being required given how many of them I don't have
installed.

However...  I don't like idea of the base group and base-system package
existing together.  You definition of what base-system should be is much
the same as what the base group was defined to be.  What package
justifies itself in the base group, but would not be in base-system?  It
seems we would have two very similar things where one would do.

Allan


Re: [arch-dev-public] Proposal: minimal base system

2019-01-21 Thread Allan McRae via arch-dev-public
On 22/1/19 9:41 am, Bruno Pagani wrote:
> Le 22/01/2019 à 00:23, Allan McRae via arch-dev-public a écrit :
>> On 22/1/19 8:03 am, Levente Polyak via arch-dev-public wrote:
>>> Everything that won’t be part of base-system needs to be added as a
>>> dependency to all requiring packages; alternatively don't omit any first
>>> level runtime dependencies at all.
>>>
>>> This package should only depend on strictly required explicit packages
>>> to get a working minimal Arch Linux system.
>>>
>>> The proposed end result is:
>>> - base: convenient helper group for quickly getting a working system
>>>   when installing, must include the new base-system package
>>> - base-system: package defining the minimum dependencies for a working
>>>   base runtime
>> I think the proposal is OK.  I'm not comfortable with our line about
>> base group packages being required given how many of them I don't have
>> installed.
>>
>> However...  I don't like idea of the base group and base-system package
>> existing together.  You definition of what base-system should be is much
>> the same as what the base group was defined to be.  What package
>> justifies itself in the base group, but would not be in base-system?  It
>> seems we would have two very similar things where one would do.
>>
>> Allan
> 
> In the proposal, base would really just be a convenient helper for e.g.
> beginners installing their system, so they could get all tools that are
> often used during install (e.g. cryptsetup, lvm2, various FS/network
> tools, etc.) or (POSIX) tools people coming from other distros would
> expect to be here by default (man pages, nano/vi…) but that are
> interactive ones and thus not really required for operating.
> 
> Anyone knowing their stuff could just install base-system + what they
> actually need (e.g., I would install cryptsetup and vim, and not care
> about netctl, xfsprogs or lvm2).

"Anyone knowing their stuff" is the essentially the stated Arch target
audience.

So, the definitions of the sets of packages are:

base-system - essential packages we assume everyone has installed
(previous definition of base...)

base group - base-system plus other packages some people probably
want/expect and support packages for filesystem types most people don't
actually need.

Maybe slightly facetious on that last one, but I don't see a clear need
for the base group once base-system exists.

Allan


Re: [arch-dev-public] RFC: (devtools) Changing default compression method to zstd

2019-03-24 Thread Allan McRae via arch-dev-public
Given the suggestion of using -18-, I decided to calculate how much
bigger our packages would be with the numbers given:

cuda-10.0.130-2-x86_64.pkg.tar  58.5M   104.40%
gcc-8.2.1+20181127-1-x86_64.pkg.tar 2.8M108.30%
go-2:1.12.1-1-x86_64.pkg.tar10.6M   108.70%
linux-5.0.3.arch1-1-x86_64.pkg.tar  -0.4M   99.40%
linux-headers-5.0.3.arch1-1-x86_64.pkg.tar  1.9M111.20%
tensorflow-1.13.1-2-x86_64.pkg.tar  6.2M111.20%
intellij-idea-community-edition-2:2018.3.5-18.1M102.10%

That is a decent increase.


Are the times given for decompress just a decompress, not install by
pacman?   Were they done on a SSD or pointed at /dev/null? (I assume not
a spinning disk given the times)

Allan


Re: [arch-dev-public] RFC: (devtools) Changing default compression method to zstd

2019-03-24 Thread Allan McRae via arch-dev-public
On 25/3/19 4:34 am, Robin Broda via arch-dev-public wrote:
> This change requires a new pacman release, as as of writing this, zstd 
> support is in master but hasn't landed in a release yet.

Which is a complete blocker for quite a period of time.

We need to assume every system has a copy of pacman-5.2+ before we can
switch packages to zstd.  Otherwise updates can break systems (pacman
and all dependencies will need to stay as .xz for a year at least, and
we have to hope that partial update does not break anything until a full
update is run).  Experience switching from .gz to .xz showed system
breakage was a fairly regular occurrence.

I would not do this until at least one year, maybe two after pacman-5.2
is released.

Allan


Re: [arch-dev-public] RFC: (devtools) Changing default compression method to zstd

2019-03-24 Thread Allan McRae via arch-dev-public
On 25/3/19 9:28 am, Robin Broda via arch-dev-public wrote:
> On 3/25/19 12:22 AM, Andrew Gregory wrote:
>> On 03/25/19 at 12:15am, Robin Broda via arch-dev-public wrote:
>>> On 3/24/19 11:20 PM, Evangelos Foutras via arch-dev-public wrote:
>>>> On Sun, 24 Mar 2019 at 23:45, Allan McRae via arch-dev-public
>>>>  wrote:
>>>>>
>>>>> We need to assume every system has a copy of pacman-5.2+ before we can
>>>>> switch packages to zstd.
>>>>
>>>> Why is pacman support needed here? I can already install .zstd
>>>> packages using pacman 5.1.3.
>>>>
>>>> The crucial part seems to be libarchive support, which was added in
>>>> v3.3.3 (~ September 2018).
>>>>
>>>
>>> Yes, installing zstd packages works - the pacman release is merely
>>> required for makepkg. Unless that has already landed too, which
>>> would be news to me :)
>>>
>>> Thus i don't think we need a hold-off period like this, Allan.
>>
>> We still need a hold-off period, we're just waiting to make sure
>> people have libarchive v3.3.3 instead of pacman v5.2.0.
>>
> 
> That update happened half a year ago, i'm sure that most people with an 
> installation that old will already have to fetch other packages, like the 
> keyring, separately for it to go through.

Fetching a keyring does not potentially bump sonames.

> Plus, with libarchives' release cycle, i don't think that libarchive itself 
> is gonna be rebuilt immediately after the change is implemented - providing 
> extra time to upgrade libarchive without having to download a release packed 
> as xz separately.

And if openssl gets and soname bump?

A


Re: [arch-dev-public] Python 2 modules

2019-02-17 Thread Allan McRae via arch-dev-public
On 17/2/19 10:37 am, Eli Schwartz via arch-dev-public wrote:
> On 2/16/19 4:36 PM, Christian Hesse wrote:
>> Allan McRae via arch-dev-public  on Sat,
>> 2019/02/16 11:19:
>>> Below is a list of python2 modules that are a dependency for any other
>>> package. I did not check makedepends and I did not check recursively to
>>> build this list.
>>
>> The list contains optional dependencies. How to handle these?
> 
> I see no conceptual difference between optional dependencies and hard
> dependencies. Both are equally deserving of being in the repos for the
> sake of providing dependent functionality to other packages.
> 

Yes - this was a quick pass list.  There is probably a bunch that should
not be removed and a lot more that could be removed...

A


Re: [arch-dev-public] Proposal: minimal base system

2019-02-12 Thread Allan McRae via arch-dev-public
On 13/2/19 8:17 am, Levente Polyak via arch-dev-public wrote:
> On 2/12/19 7:16 PM, Gaetan Bisson via arch-dev-public wrote:
>> [2019-02-12 16:40:08 +0100] Bruno Pagani via arch-dev-public:
>>> Just in case it wasn’t clear, my answer would have been mostly the same
>>> as Eli’s.
>>>
>>> So, Gaetan, Allan and Bartłomiej (or anyone else for that matter), do
>>> you have further comments/questions regarding this, does the existence
>>> of the base group alongside the arch/minimal-system now makes sense or
>>> would you still prefer to go without it?
>>
>> Allan and I have both stated that we don't want to introduce a new group
>> since we believe it would be highly redundant with base.
>>
>> Nothing new has been said since our last messages except Eli's post
>> which argues that the base group is largely inadequate in its current
>> state. This further supports our proposal that base should be improved
>> instead of introducing a new group.
>>
>> So I really don't see what arguments could have changed our minds...
>> It's also strange to me how you can concur with Eli's post without
>> agreeing with our conclusions.
>>
>> To go forward I suggest you propose a clear definition of the perfect
>> "minimal system" group you'd want to have, along with a proposed list of
>> packages. When consensus is reached, we adopt this list of packages for
>> base and put this definition on the wiki.
>>
>> Cheers.
>>
> 
> To make it as short as possible, the idea is not just to strip down the
> base group further but primarily to not use a group in the first place.
> It should be replaced with something that is consistent within itself
> over the whole lifetime of the system.
> Groups are the wrong tool for such a set: you explicitly install all
> those packages so they won't automatically be mark as not-required
> anymore once removed from that group, as well as new additions are not
> consistent during the lifetime of the system.

We are clear about that.  Call it a group or metapackage or whatever,
the objection is having the current base and the new "group" at the same
time.

A


[arch-dev-public] Python 2 modules

2019-02-15 Thread Allan McRae via arch-dev-public
Hi all,

Python 2 reaches End of Life on 2020-01-01.  We currently have >950
python2 modules in the repos.   A lot of these are not used by any other
package in the repositories.   I think we should work towards removing them.

Leaving only python2 modules that are really required by other software,
highlights what needs worked on to port to python3.

Note Fedora is doing a similar removal for F30:
https://fedoraproject.org/wiki/Changes/Mass_Python_2_Package_Removal

What are opinions on this?  Should I make a TODO list?


Below is a list of python2 modules that are a dependency for any other
package. I did not check makedepends and I did not check recursively to
build this list.

python2-acme
python2-antlr2
python2-anyjson
python2-anytree
python2-apache-libcloud
python2-apispec
python2-argcomplete
python2-argon2_cffi
python2-argparse
python2-args
python2-arrow
python2-aspectlib
python2-astor
python2-atspi
python2-aubio
python2-audit
python2-augeas
python2-autobahn
python2-autopep8
python2-backports.lzma
python2-basemap
python2-betamax-matchers
python2-betamax-serializers
python2-binary-memcached
python2-biopython
python2-bitvector
python2-blist
python2-blosc
python2-bluepy
python2-bottle
python2-bottleneck
python2-braintree
python2-breathe
python2-bsddb
python2-btchip
python2-btrees
python2-cached-property
python2-caja
python2-cchardet
python2-celery
python2-chai
python2-chameleon
python2-characteristic
python2-cjkwrap
python2-click-log
python2-click-threading
python2-cloudflare
python2-cmarkgfm
python2-colander
python2-colorclass
python2-configargparse
python2-construct
python2-couchdb
python2-cram
python2-crayons
python2-cryptography-vectors
python2-cson
python2-cssselect2
python2-cssutils
python2-cx_freeze
python2-d2to1
python2-daemon
python2-daemonize
python2-datrie
python2-ddt
python2-digitalocean
python2-discid
python2-distutils-extra
python2-django
python2-dnslib
python2-dockerpty
python2-docopt
python2-docs
python2-doublex-expects
python2-dpcontracts
python2-dropbox
python2-editdistance
python2-egenix-mx-base
python2-elasticsearch-curator
python2-email-validator
python2-envisage
python2-eric
python2-ethtool
python2-evdev
python2-exam
python2-exiv2
python2-eyed3
python2-factory-boy
python2-fastpbkdf2
python2-faulthandler
python2-flake8-blind-except
python2-flake8-debugger
python2-flaky
python2-flask-gravatar
python2-flask-htmlmin
python2-flask-jwt
python2-flask-mail
python2-flask-migrate
python2-flask-paranoid
python2-flask-restful
python2-flask-security
python2-flask-socketio
python2-flask-talisman
python2-flask-wtf
python2-flexmock
python2-flickrapi
python2-flup
python2-fonttools
python2-foolscap
python2-fpconst
python2-freezegun
python2-fs
python2-funcy
python2-furl
python2-fxa
python2-gasp
python2-gcp-devrel-py-tools
python2-gdal
python2-gdata
python2-genshi
python2-genty
python2-geoip
python2-gevent-websocket
python2-gflags
python2-gitpython
python2-gnupg
python2-gnupginterface
python2-gnuplot
python2-gpgme
python2-grequests
python2-gtkspellcheck
python2-gudev
python2-h2
python2-h5py
python2-h5py-openmpi
python2-hacking
python2-harparser
python2-helper
python2-hexdump
python2-hglib
python2-httpretty
python2-hunter
python2-hypothesis
python2-i3-py
python2-ibm-db-sa
python2-icalendar
python2-igraph
python2-importlib_resources
python2-inet_diag
python2-invoke
python2-iocapture
python2-ipdb
python2-irc
python2-isomd5sum
python2-iwlib
python2-jieba
python2-js2py
python2-jsbeautifier
python2-json-logger
python2-jsonrpclib-pelix
python2-kaitaistruct
python2-kajiki
python2-kaptan
python2-keybinder2
python2-keyrings-alt
python2-keyutils
python2-kitchen
python2-kivy
python2-klein
python2-langdetect
python2-language-server
python2-lark-parser
python2-levenshtein
python2-libappindicator
python2-libarchive-c
python2-libforensic1394
python2-librabbitmq
python2-libtmux
python2-linux-procfs
python2-llfuse
python2-logbook
python2-logilab-common
python2-lttngust
python2-m2r
python2-magic
python2-mamba
python2-manhole
python2-manuel
python2-marisa
python2-marshmallow
python2-memcached
python2-mimerender
python2-mockito
python2-mongoengine
python2-mongomock
python2-mpd2
python2-munkres
python2-musicbrainz2
python2-mygpoclient
python2-mysql-connector
python2-nbxmpp
python2-ndg-httpsclient
python2-neovim
python2-netcdf4
python2-netcdf4-openmpi
python2-nine
python2-nltk
python2-nose2
python2-nose-cover3
python2-nose-exclude
python2-nose-fixes
python2-nose-randomly
python2-nose-show-skipped
python2-nosexcover
python2-oauth2client
python2-objgraph
python2-olefile
python2-openapi-spec-validator
python2-openpyxl
python2-openstackclient
python2-oslo-concurrency
python2-oslo-log
python2-oslosphinx
python2-oslotest
python2-ovirt-engine-sdk
python2-owslib
python2-pacparser
python2-pam
python2-pandas-datareader
python2-pandocfilters
python2-parse
python2-parsedatetime
python2-parsel
python2-paste
python2-pastedeploy
python2-pbkdf2
python2-pdoc
python2-peewee
python2-perf
python2-periphery
python2-phonenumbers
python2-piexif

Re: [arch-dev-public] Proposal: minimal base system

2019-02-05 Thread Allan McRae via arch-dev-public
On 5/2/19 11:38 pm, Bruno Pagani wrote:
> Maybe. As I said in my answer to Bartłomiej, I don’t know if beginners
> know enough things to install what they need beyond the minimum system,
> or if they just read the wiki about doing this or that, which might
> assume they have the current base group installed.

If only the wiki wasn't a static document, and could be updated after
any change was made...

Even then I am fairly sure people wanting to set-up LVM can install the
lvm package.  But I see "competent Linux user" has been removed from the
front page, so maybe we target idiots these days.

>> If we remove the excess from base, then we are down to a very small
>> difference between that and archlinux-system.  Only e2fsprogs, man, and
>> an editor different?
>>
>> So I see the proposed archlinux-system group being essentially what base
>> should be.
> 
> That is because you see base as the minimal system. So I’ll turn this
> differently: do you have objections against having, outside of the
> minimal meta-package described in our proposal, a packages group of
> “relatively standard” tools, that is purposed at beginner wanting to
> have only one simple pacstrap command to issue in order to get started?
> 
> Or put it yet another way: outside of this base group, does our proposal
> of a minimal metapackage suits you? If so, why does it matter to you
> that there is also a base group, provided the name is not subject to
> confusion, that has this metapackage plus other tools (that e.g. people
> coming from random other distro would expect to have at hand from the
> start), knowing that you would likely have almost no interactions with
> this group? If not, then I’m even more importantly waiting for your
> comments.

I don't object to redefining the minimal set of packages we expect
installed and making it a metapackage.  Currently base is bloated, and a
metapackage makes some sense.

archlinux-system - minimal set of packages
base-devel - collection of utilities most people expect installed

So where does that leave base?

base - a smaller collection of utilities most people expect installed

I see redundancy being created.  That is what I object to.

Allan


Re: [arch-dev-public] Proposal: minimal base system

2019-02-05 Thread Allan McRae via arch-dev-public
On 5/2/19 9:06 pm, Bruno Pagani wrote:
> Le 22/01/2019 à 00:59, Allan McRae a écrit :
>> On 22/1/19 9:41 am, Bruno Pagani wrote:
>>> Le 22/01/2019 à 00:23, Allan McRae via arch-dev-public a écrit :
>>>> On 22/1/19 8:03 am, Levente Polyak via arch-dev-public wrote:
>>>>> Everything that won’t be part of base-system needs to be added as a
>>>>> dependency to all requiring packages; alternatively don't omit any first
>>>>> level runtime dependencies at all.
>>>>>
>>>>> This package should only depend on strictly required explicit packages
>>>>> to get a working minimal Arch Linux system.
>>>>>
>>>>> The proposed end result is:
>>>>> - base: convenient helper group for quickly getting a working system
>>>>>   when installing, must include the new base-system package
>>>>> - base-system: package defining the minimum dependencies for a working
>>>>>   base runtime
>>>> I think the proposal is OK.  I'm not comfortable with our line about
>>>> base group packages being required given how many of them I don't have
>>>> installed.
>>>>
>>>> However...  I don't like idea of the base group and base-system package
>>>> existing together.  You definition of what base-system should be is much
>>>> the same as what the base group was defined to be.  What package
>>>> justifies itself in the base group, but would not be in base-system?  It
>>>> seems we would have two very similar things where one would do.
>>>>
>>>> Allan
>>> In the proposal, base would really just be a convenient helper for e.g.
>>> beginners installing their system, so they could get all tools that are
>>> often used during install (e.g. cryptsetup, lvm2, various FS/network
>>> tools, etc.) or (POSIX) tools people coming from other distros would
>>> expect to be here by default (man pages, nano/vi…) but that are
>>> interactive ones and thus not really required for operating.
>>>
>>> Anyone knowing their stuff could just install base-system + what they
>>> actually need (e.g., I would install cryptsetup and vim, and not care
>>> about netctl, xfsprogs or lvm2).
>> "Anyone knowing their stuff" is the essentially the stated Arch target
>> audience.
> 
> So apparently we did not answer all concerns here. I don’t expect Arch
> users to know thing so well that they know exactly what tools are in
> which packages when they install Arch for the first time. I think we
> should not mistake Arch Power Users, people that have a level of
> knowledge above Arch Users, that are just generic Linux Power Users.
> 
>> So, the definitions of the sets of packages are:
>>
>> base-system - essential packages we assume everyone has installed
>> (previous definition of base...)
> 
> To be clearer, the new proposition would be to call this arch-system to
> avoid confusion with base. However, note that this “previous definition
> of base” is definitively not that clear: when I installed Arch, I read
> things as “base is a convenient helper to get almost every standard
> tools you could need to do your install”.
> 
>> base group - base-system plus other packages some people probably
>> want/expect and support packages for filesystem types most people don't
>> actually need.
> For me, base will be what it has ever been: a fast way to get started as
> an Arch beginner.
>> Maybe slightly facetious on that last one, but I don't see a clear need
>> for the base group once base-system exists.
> 
> Because, as an Arch dev, you definitively qualify as an Arch Power
> Users. I wouldn’t use base either for myself, but I firmly believe most
> Arch beginners would.
> 
> Does that make sense to you, or do you still think every new Arch User
> should already know exactly what is required to get started?
> 

If someone knows they want to set up logical volumes and drive
encryption, then they know enough to install lvm and cryptsetup.  Same
with jfsutils, xfsutils.   So I don't think they should be in the base
group (e.g. I would not call jfsutils a standard tool).

If we remove the excess from base, then we are down to a very small
difference between that and archlinux-system.  Only e2fsprogs, man, and
an editor different?

So I see the proposed archlinux-system group being essentially what base
should be.

A


Re: [arch-dev-public] A contrib repository

2019-03-13 Thread Allan McRae via arch-dev-public
On 3/14/19 8:46 AM, Morten Linderud via arch-dev-public wrote:
> There is a *lot* of small tools people have written over the years that 
> resides
> in bin/ directories which could be useful for more people. We also have 
> several
> such tools on soyuz, where sogrep was added to devtools this week. 
> 
> I have been thinking it could be great to have a simple `contrib` repository
> where every team member has commit access. This could work as a staging area 
> for
> tools we would like to promote to `devtools` later on.
> 
> We could maybe sort this into directories for its purpose:
> * packaging
> * security
> * devops
> * testing
> * bugwrangler
> * misc
> 
> Tools that can be added here is the `ch` scripts from Bluewind, and the pkg-*
> tools eli has created. I also have some tools to look for pkgname in archweb,
> check them out from svn and check them against a nvchecker file.
> 
> This would hopefully give us a space where we can experiment with new 
> maintainer
> tools in a collaborative manner. I'd love to hear some feedback or thoughs on
> this!
> 

I was fairly sure any user can create a git repo on our server.  Look at
"Developer Projects" on https://git.archlinux.org/ .  Or use github,
where some of these scripts are already located.

I don't see the need for another repository.

A


Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions

2019-05-24 Thread Allan McRae via arch-dev-public
On 25/5/19 10:17 am, Filipe Laíns via arch-dev-public wrote:
> Hello,
> 
> Currently there are no guidelines stating which x86 extensions (ex.
> SSE2, SEE3, SSE4, AVX, etc.) we support. This is a bit problematic
> since it lets compilers do what they want and possible generate code
> that can't run on some systems.
> 
> Even though this is an issue, it's not complete anarchy, at least yet!
> Just kidding :p. The vast majority of our native packages are compiled
> with GCC and we do default to `-mtune=generic` which is good but not
> optimal. `-mtune=generic` tells GCC to compile for a generic processor
> so it's up to GCC to decide which architecture extensions would compose
> a generic processor. I haven't been able to find any documentation on
> what x86 extensions are enabled for a "generic" processor but I was
> able to track them down to MMX, SSE (or KNI) and SSE2. Being
> undocumented they could change at any time so I don't think we should
> rely on `-mtune=generic`.
> 

I think you need to look at the difference between -march and -mtune.
We use "-march=x86-64", which defines the instruction sets that can be
used.  Adding "-mtune=generic" does not allow the inclusion of
additional instruction sets.

Look at the output of:
gcc -march=x86-64 -Q --help=target

Allan


Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions

2019-05-25 Thread Allan McRae via arch-dev-public
On 25/5/19 9:34 pm, Bruno Pagani wrote:
> Out of curiosity, what did you rebuild of [core] lead to?

I had a potentially slightly faster system for a week...  It was mainly
a test to see if I spotted some build issues of test suite failures
beyond what is seen for x86_64.  All was good.

A


Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions

2019-05-25 Thread Allan McRae via arch-dev-public
On 25/5/19 9:19 pm, Bruno Pagani via arch-dev-public wrote:
> Hi,
> 
> Le 25/05/2019 à 02:17, Filipe Laíns via arch-dev-public a écrit :
>> I would also like to explore the idea of adding an "high performance"
>> architecture which would be able to make use of SSE{,2,3,4,4.1,4.2} and
>> AVX, which seem to be the standard for newer processors (>=2013). This
>> would only be available for packages that do high performance computing
>> (ex. openblas, sdrangel, etc.). Any thoughts on this?
> 
> As said on IRC, they have been discussions before on having multiple
> targets and corresponding repos, but the starting point is that we need
> automated build before going into such a direction, and this in turn has
> several requirements. I’ve linked to you the pad where we put our ideas
> together regarding this.
> 
> In the meantime, we had the case before of whether we should package
> e.g. $pkgname-{sse4,avx} in a case where it mattered a lot, but it
> turned out the software in question (embree) is able to do runtime
> detection of available ISA. Maybe some other packages are doing this
> too, else we could discuss whether allowing such flavours as a temporary
> measure would be acceptable for selected packages.

glibc detects available instruction sets and uses the best for many
functions.

I'd be very, very, very much against providing multiple variants of a
package in our repos.  Using asp and makepkg are is a hard solution for
those who really need a few packages rebuilt.

PS - I rebuilt [core] with -march=haswell recently as a test.  Automated
building is not an issue.  Unattended package/database signing is the
major stumbling block.

Allan


Re: [arch-dev-public] Create guidelines regarding SIMD instructions/x86 extensions

2019-05-25 Thread Allan McRae via arch-dev-public
On 25/5/19 5:22 pm, Lukas Jirkovsky via arch-dev-public wrote:
> On Sat, 25 May 2019 at 04:27, Filipe Laíns via arch-dev-public
>  wrote:
>> Setting `-mtune` to generic won't add any additional instruction sets
>> by itself, but it does not prevent instruction sets from being added.
>> Looks like GCC enables MMX, SSE and SSE2 by default, it isn't related
>> at all to `-march` like I stated in the email but it still presents the
>> same issue.
> 
> As far as I know, MMX, SSE and SSE2 are mandatory part of the AMD64
> instruction set, so they are not enabled randomly just because someone
> felt like it, but because they are be present on every x86_64 cpu.
> .

Correct.  Using the command I gave in my first reply:

$ gcc -march=x86-64 -Q --help=target | grep sse
  -mfpmath= sse
  -mno-sse4 [enabled]
  -msse [enabled]
  -msse2[enabled]
  -msse2avx [disabled]
  -msse3[disabled]
  -msse4[disabled]
...

$ gcc -march=x86-64 -Q --help=target | grep mmx
  -mmmx [enabled]

-mtune just tunes instructions for a "representative" set of "current"
CPUs that run as x86-64.

Allan


Re: [arch-dev-public] Proposal for a new organisation structure

2019-06-02 Thread Allan McRae via arch-dev-public
On 3/6/19 2:39 pm, Christian Rebischke via arch-dev-public wrote:
> 3. I don't like that devs and TUs live aside each other instead of
> finally realizing themselves as community or group.
The TUs are set up as an independent organisation with their own bylaws.
 Many of those bylaws were implemented to keep independence from the
developer team.  Saying that, almost all developers have been a Trusted
User, and there is a decent overlap currently.  I mostly see this as a
responsibility divide.

> I think the
> community as itself would work much better if we would have user-access
> based package repositories and just 'maintainers', instead of this
> dev/TU split.

I agree that the distinction between [extra] and [community] is rather
limited.  I think we need a better definition of what [extra] is, and
move packages that don't fit that definition out of [extra] and into
[community] where both devs and TUs can maintain them.  Or even merge
those two repos.

>> As Gaetan already mentioned, the process is clear: somebody suggests a
>> new developer and we discuss until a consensus is reached. Feel free to
>> document that somewhere in our wiki if you think it needs to be
>> documented. If you have specific concerns with that process, feel free
>> to share them. However, I do not think anybody in the dev team ever had
>> any objections against that procedure.
>
> What interests me is why is this whole process not transparent and
> public? I mean we discuss adding new TUs publicly. Of course this
> dicussion comes with all its good and bad parts, but atleast it's
> transparent and people get a reason why they are not elected.

You are very much overthinking what goes on when deciding on a new
developer.  "Electing" a developer is very different than electing a TU.
 People given developer positions have probably been around for years
and are already doing the job before they get formally offered it.   So
it is a case of someone saying "this person should be a developer" and
the rest going "OK".  There is usually no discussion, because the
candidate is well known already, and has obviously been doing a good job
to even be nominated.  Having TU style discussions on the suitability of
a candidate makes no sense in that situation.

Allan


Re: [arch-dev-public] bringing vivaldi browser to community

2019-06-01 Thread Allan McRae via arch-dev-public
On 1/6/19 8:12 pm, Ike Devolder via arch-dev-public wrote:
> 3 years have passed since I first proposed to bring vivaldi into
> community. Now there is a clear differentiation between what vivaldi
> offers out-of-the box compared to other browsers.
> 
> Vivaldi offers a ton of customisation features out of the box, and is
> also able to just use the chrome/ium addons from the chrome webstore.
> 
> Personally I'm using vivaldi as my main browser since somewhere in 2015
> (shortly after the first beta was released) and the key features no
> other browser currently offers are:
> - webpanels
> - quick commands
> - tabtiling
> - tabstacking
> - tabbar positioning
> 
> I'll bring it in the same way as with opera, where you have the
> webbrowser + separate packge with libffmpeg.so to allow the playback of
> proprietary formats like mp4.
> 

Does the license allow us to distribute it?

And dozens of packages get added to [community] a week.  Why are you
asking here unless you think there will be an issue?  You don't seem to
explain why you need to ask in your email.

Allan


Re: [arch-dev-public] Proposal for a new organisation structure

2019-06-01 Thread Allan McRae via arch-dev-public
On 2/6/19 9:08 am, Christian Rebischke via arch-dev-public wrote:
> 1. Simplified repositories
> 
> Currently we have [core], [extra] and [community]. These three
> repositories are there, because they have grown to this structure over
> time. At the moment I don't see any real meaning for the split of [core]
> and [extra]. So one suggestion would be to either:
>   - merge [extra] and [core]

I stopped reading here.   If you don't understand the difference between
[core] and [extra], you should ask.  Particularly before proposing the
removal of [core].

Allan


Re: [arch-dev-public] bringing vivaldi browser to community

2019-06-01 Thread Allan McRae via arch-dev-public
On 2/6/19 1:53 am, Ike Devolder via arch-dev-public wrote:
> On Sat, 2019-06-01 at 21:30 +1000, Allan McRae wrote:
>> You don't seem to
>> explain why you need to ask in your email.
> 
> Because it is proprietary and I explain that now there is a valid
> reason compared to 3 years ago where there was practically no
> difference between vivaldi, chromium and opera.
> 

Does the license allow us to have it in the repos?  After a quick look,
I'd say no.

A


Re: [arch-dev-public] New ideas for notifying users about (minor) changes

2019-07-29 Thread Allan McRae via arch-dev-public
On 30/7/19 10:44 am, Christian Rebischke wrote:
> One problem I see with the News section is that it's Dev only. I
> wouldn't even know who I need to ask (and I am TU for several years
> now). I mean I could just ask around in the IRC, but it would be nice to
> have this at least documented somewhere.

I guess the assumption is changes that affect a larger proportion of
users are in packages maintained by devs. However, I'm happy opening the
news to TUs if needed.

Allan


Re: [arch-dev-public] New ideas for notifying users about (minor) changes

2019-07-29 Thread Allan McRae via arch-dev-public
On 30/7/19 8:47 am, Christian Rebischke via arch-dev-public wrote:
> Hello everybody,
> I got assigned to this issue here:
> 
> https://bugs.archlinux.org/task/62823
> 
> The users would like to have a notice in pre/post install/upgrade
> notifications via pacman .install files.
> 
> I am not a fan of spamming such news via pacman and I think the
> installation/upgrade process should be clean of such messages, but I
> don't have access to the news tool on our website as well.

I think this is a clear case of user intervention needed.   So a
post_install message (restricted to the update where intervention was
needed) is appropriate.

If the package was more widely used, we have the News section of the
website.

I don't think anything else is needed.

Allan


[arch-dev-public] Reproducible builds progress and the upcoming rebuild of [core]

2019-11-12 Thread Allan McRae via arch-dev-public
Hi all,

As you may know, we have had people busy looking at what it takes to
make our packages reproducible.

There has been a lot of progress there lately.  Our reproducible builds
team (along with the wider reproducible builds community) has been
building our packages in different environments to test how stable the
builds are [1].  The good news is that >80% of our packages could be
built twice in varying environments and give the exact same result.

However, that is only part of the picture.  Ideally, we want people to
be able to take one of our packages and rebuild it exactly.  With the
release of pacman-5.2, packages record a lot more information about
their build environment.  That means we can reconstruct a package's
build chroot, and then rebuild it.  There are two tools in the works to
do this.  One by Morton (Foxboron) [2] and one by Eli [3]. Note that
both tools need more testing to be ready for a wider release and
currently require some manual editing to run.

The good news is, we have at least 10 packages that can be precisely
reproduced using both these tools [4]!  This means you can take one of
these tools and rebuild a package from the repos, and get the exact same
package out of it.  This is an amazing effort - well done to the team!

To keep this momentum going, it would be great to rebuild every package
in [core] using makepkg from pacman-5.2+.  That way we can test which
packages are actually reproducible and work towards fixing those that
are not.  So be prepared for almost the entire repo to hit [testing]
soon, and get your sign-off shoes on!

Again, a huge congrats to our reproducible builds team.  This has been a
massive amount of work!

Allan


[1] https://tests.reproducible-builds.org/archlinux/archlinux.html
[2] https://github.com/archlinux/archlinux-repro
[3]
https://github.com/eli-schwartz/devtools/blob/reproducible/makerepropkg.in
[4] https://wiki.archlinux.org/index.php/DeveloperWiki:ReproduciblePackages


Re: [arch-dev-public] Proposal: Build a ruleset for new packages and package quality

2019-12-12 Thread Allan McRae via arch-dev-public
On 12/12/19 10:21 pm, Christian Rebischke via arch-dev-public wrote:
> 3. revive the discussion around better PKGBUILD quality (see also Eli's
> thread about  PKGBUILD quality on arch-tu:
> https://lists.archlinux.org/private/arch-tu/2019-November/83.html)

Any chance that can be posted somewhere visible to those that aren't TUs?

Allan


Re: [arch-dev-public] Using SPDX License list as identifiers

2019-10-22 Thread Allan McRae via arch-dev-public
On 22/10/19 9:59 pm, Jerome Leclanche wrote:
> It would also be a
> little more accurate; eg. the SPDX allows for distinctions such as
> "LGPL-3.0-or-later" vs. "LGPL-3.0-only".

I thought we already managed that, but it seems in rather limited use
these days looking at our -Si output.  e.g.

Licenses: LGPL-2.1
Licenses: LGPL-2.1+

A


Re: [arch-dev-public] cleanup dead Xorg packages | news draft

2019-12-19 Thread Allan McRae via arch-dev-public
On 20/12/19 8:35 am, Andreas Radke via arch-dev-public wrote:
> Packages have been rebuilt and prepared to remove obsolete libdmx and
> libxxf86dga. Xorgproto legacy support has been removed and wherever it
> was added to be a runtime dependency it is now a build time
> dependency. Some packages will need additional xorgproto makedependency
> added when missing some header now that the libraries won't pull it in
> anymore. That behavior is desired. I'm still in the process of fixing
> the move from legacy proto headers to xorgproto headers. I'm going to
> commit the changes to trunk for future builds.
> 
> There's no package replacing libdmx and libxxf86dga so manual
> intervention will be needed. So here's a small news draft:
> 
> "Xorg cleanup requires manual intervention
> 
> "In the process of Xorg cleanup [1] the update requires manual
> intervention when you hit this message:
> 
>  :: installing xorgproto (2019.2-2) breaks dependency 'inputproto' required 
> by lib32-libxi
>  :: installing xorgproto (2019.2-2) breaks dependency 'dmxproto' required by 
> libdmx
>  :: installing xorgproto (2019.2-2) breaks dependency 'xf86dgaproto'
>  required by libxxf86dga
> 
> when updating, use:
> `pacman -Rdd libdmx libxxf86dga && pacman -Syu`
> 
> to perform the upgrade. After the update it will be safe to also remove
> the "xorgproto" package.

This why does xorgproto not provides=('inputproto'  )?  Then all we
need to do is announce, update and remove.

Allan


Re: [arch-dev-public] libx11/xorgproto dependency

2019-12-21 Thread Allan McRae via arch-dev-public
On 21/12/19 7:31 pm, Evangelos Foutras via arch-dev-public wrote:
> Downstream consumers of libx11 shouldn't have to know and account for
> libx11's headers/pkg-config files referencing xorgproto. A
> libx11-devel package would depend on xorgproto. Since there's no
> separate -devel package, the dependency stays with the regular libx11
> package.

This matches my opinion on the matter.

A


Re: [arch-dev-public] Adding a "posix" metapackage

2020-01-03 Thread Allan McRae via arch-dev-public
On 4/1/20 12:47 pm, Sébastien Luttringer via arch-dev-public wrote:
> One unfortunate consequence could be to have packages rely on it to make
> dependencies shorter, and make us pull cups or cronie.

What?!  That is like saying one unfortunate consequence of pamcan hooks
is that packages can have no files and just download and compile the
source in a hook.  It is a ridiculous consideration.

A


Re: [arch-dev-public] Restricting ability to post news items

2020-01-05 Thread Allan McRae via arch-dev-public
On 6/1/20 9:12 am, Giancarlo Razzolini wrote:
> Em janeiro 5, 2020 19:25 Allan McRae via arch-dev-public escreveu:
>>
>> Read the original message and not the partial quote that you made.  I
>> explicitly said there was an exception for --overwrite type posts.
>>
>> But any restriction being made on posting due to not posting drafts to
>> the list would be complete.
>>
> 
> Hi Allan,
> 
> I think you're overreacting (again) for something that's not properly
> coded anywhere.
> And, the last time this was discussed, I have talked about cases of news
> that couldn't
> be brought on a-d-p for discussion. We have other places to discuss
> those (staff@), but
> still, until we have *clear* guidelines about this, let's not rush to
> implement anything.

My apologies.  I believed this had been covered on the mailing list and
I am told it was only on IRC, and never passed on to the TU channel,
which I will accept as an excuse despite everyone(?) involved but the
poster being on both channels...

> I have proposed a news entry regarding zstd, Robin did the statistics
> and we've discussed this
> for 2 days on #archlinux-tu. If you're looking for somebody to blame,
> look no further. Take away
> my news posting rights (you'll have to also take away my archweb admin
> rights in this case).
> 
> And let's properly codify this ok? Because it's not codified anywhere. I
> could've asked Robin
> to send an email to a-d-p, but most of the work we've done on the draft
> was on a pad. This was
> reviewed by quite a few people. I asked Robin to do it, because since
> they pushed zstd, they
> should've also be the ones doing the news entry. But I would do it
> myself this week, if they
> didn't.

Do we really need to write down everything?  Have we reached a point in
the distro where common sense has stopped?  Why would an announcement
that affects the whole distro not be run past all team members by default?

Allan


Re: [arch-dev-public] Restricting ability to post news items

2020-01-05 Thread Allan McRae via arch-dev-public
On 6/1/20 8:17 am, Morten Linderud via arch-dev-public wrote:
> On Sun, Jan 05, 2020 at 11:10:17PM +0100, Bartłomiej Piotrowski via 
> arch-dev-public wrote:
>> On 05/01/2020 23.04, Morten Linderud via arch-dev-public wrote:
>>> On Mon, Jan 06, 2020 at 07:53:21AM +1000, Allan McRae via arch-dev-public 
>>> wrote:
>>>> Following the roll out of the base metapackage, and its poorly written
>>>> news post, we agreed that all new posts should have a draft posted to
>>>> arch-dev-public.
>>>
>>> Wait, where was this agreed? I heard something about all drafts should be 
>>> headed
>>> for arch-dev, but this hasn't been announced nor discussed anywhere.
>>>
>>> Are the TUs missing from the loop here?
>>>
>>
>> If you look at the non-trivial news items, you can easily correlate them
>> with drafts posted to arch-dev-public.
> 
> You are writing about non-trivial news items, but Allan is writing explicitly
> about *all* news items. There is a disconnect here, I'm not sure what has been
> decided.

Read the original message and not the partial quote that you made.  I
explicitly said there was an exception for --overwrite type posts.

But any restriction being made on posting due to not posting drafts to
the list would be complete.

Allan


Re: [arch-dev-public] Restricting ability to post news items

2020-01-05 Thread Allan McRae via arch-dev-public
On 6/1/20 8:04 am, Morten Linderud via arch-dev-public wrote:
> On Mon, Jan 06, 2020 at 07:53:21AM +1000, Allan McRae via arch-dev-public 
> wrote:
>> Following the roll out of the base metapackage, and its poorly written
>> news post, we agreed that all new posts should have a draft posted to
>> arch-dev-public.
> 
> Wait, where was this agreed? I heard something about all drafts should be 
> headed
> for arch-dev, but this hasn't been announced nor discussed anywhere.
> 
> Are the TUs missing from the loop here?
> 

After the base metapackage pile of crap that was posted, I (and others)
reminded everyone of the process that had been held for years.

The response was complaints that this process was not followed recently,
to which we pointed at the arch-dev-public mailing list post for
multiple non-trivial news entries.  Only simple "--overwrite" type posts
have not been posted on arch-dev-public.

This is not new - it has been the standard for many, many years -
spanning from before I was around the distro.

Allan


[arch-dev-public] Restricting ability to post news items

2020-01-05 Thread Allan McRae via arch-dev-public
Hi all,

Following the roll out of the base metapackage, and its poorly written
news post, we agreed that all new posts should have a draft posted to
arch-dev-public.  There was a potential exclusion for trivial
--overwrite posts [1].

This was followed for the Xorg update.   It was not followed for the
zstd update.

Given people have shown an inability to follow simple instructions,  I
am proposing a restriction on the ability to make news posts.  This can
either be a bulk restriction now, or just removing anybody who does not
follow the rule from ever making a news post again.

Allan


[1] and why have we had so many missing library symlinks lately...
surely checkpkg detects all the missing symlinks compared to previous
packages.


Re: [arch-dev-public] Discussion - Increasing our CPU requirements

2020-03-29 Thread Allan McRae via arch-dev-public
On 29/3/20 8:52 pm, Evangelos Foutras wrote:
> If I see a SIGILL on my AMD Phenom II X6 1090T then Arch will have failed me. 
> 
> 
> I believe your proposal should only be discussed as co-existing
> optimized port(s) and even then I'm not sure it's worth the trouble.
> Performance-critical applications can and frequently are optimized for
> the running processor (I'm thinking of stuff like glibc and ffmpeg
> here).

AVX2 was a bold choice, and really a place to get a discussion started.

We are currently supporting processors from 2003.  We can be better than
that.

A


[arch-dev-public] Discussion - Increasing our CPU requirements

2020-03-29 Thread Allan McRae via arch-dev-public
Remember when Arch Linux was optimized out of the box.   We have the
blazingly fast i686 port while other distros hung out in i386 land.
Those were the days where the idea of Arch being fast started.  Now it
has degraded to stuff of legend.

Now, x86_64 is old.  We should continue to push forward and add further
optimization.

Reasonable optimizations to consider:

AVX2
FMA
SSE4.2

AVX2 is Intel Haswell and newer or AMD Ryzen and newer.  This CPUs
released 2013 to 2015.  So 5 - 7 years old.

Discuss.


Re: [arch-dev-public] Discussion - Increasing our CPU requirements

2020-03-29 Thread Allan McRae via arch-dev-public
On 29/3/20 9:27 pm, Amish wrote:
> Also if I am not wrong Arch philosophy talks only about latest software
> and no where there is mention of latest hardware being a compulsion.

It used to.  One of the original selling points was i686 optimization.
Then we got lazy, and stopped innovating.

A


Re: [arch-dev-public] Discussion - Increasing our CPU requirements

2020-03-29 Thread Allan McRae via arch-dev-public
On 30/3/20 12:39 am, Filipe Laíns wrote:
> On Sun, 2020-03-29 at 23:37 +1000, Allan McRae via arch-dev-public wrote:
>> On 29/3/20 11:17 pm, Filipe Laíns wrote:
>>> I would also like to note that rebuilding everything with forced
>>> support for AVX2 or whatever won't have much effect. Most packages do
>>> not have workloads where it would make use sense to use these CPU
>>> extensions, and as such, GCC would not use them.
>>
>> That assumes we just add AVX2.  Whereas, requiring a CPU supporting AVX2
>> would bring other optimizations that would be used.
> 
> No, it should be true for all extensions.
> 
>> As I replied earlier, AVX2 may be going too far.  But is a good starting
>> point for discussion.  If that is too far, what could we accept?
>> SSE4.2?  AVX?   Surely we can do better than pure x86_64.
> 
> No, SSE4.2 is too far. For me, the minimum should be AVX.


SSE4.2 is 2008 for Intel, 2011 for AMD.  Though I guess some processors
were released without it for some time after that.   AVX was released by
both in 2011.

So why is one too far and the other not?


>> To have a separate architecture would require automated builds, which
>> requires being able to sign packages automatically.  And we have not
>> achieved database signing in 9 years  I'm looking for a boost that
>> could be achieved now.
> 
> No, it would not. Where is this coming from? I already build split
> packages with SIMD instructions, I make the PKGBUILD build for 2
> architectures instead with a minimal patch.
> 
> If pacman is not able to handle parallel architectures, we should fix
> that. I think it's a valid use case.

No need for pacman support.  Just add higher instruction set to a new
repo and set that repo with higher priority.

But that involves developers choosing which packages to build with
higher instruction sets, which requires extra developer time.

Ideally, we would just autobuild for more optimized architectures, but
this requires auto-signing packages, which has not happened in the last
decade (but may in this one...).


Picking an instruction set that is ~10 years old and making it the
default for the distro seems a reasonable approach to me.

A


Re: [arch-dev-public] Discussion - Increasing our CPU requirements

2020-03-29 Thread Allan McRae via arch-dev-public
On 29/3/20 11:17 pm, Filipe Laíns wrote:
> I would also like to note that rebuilding everything with forced
> support for AVX2 or whatever won't have much effect. Most packages do
> not have workloads where it would make use sense to use these CPU
> extensions, and as such, GCC would not use them.

That assumes we just add AVX2.  Whereas, requiring a CPU supporting AVX2
would bring other optimizations that would be used.

As I replied earlier, AVX2 may be going too far.  But is a good starting
point for discussion.  If that is too far, what could we accept?
SSE4.2?  AVX?   Surely we can do better than pure x86_64.


To have a separate architecture would require automated builds, which
requires being able to sign packages automatically.  And we have not
achieved database signing in 9 years  I'm looking for a boost that
could be achieved now.

Allan


[arch-dev-public] Congrats to the Arch Conf Team

2020-10-11 Thread Allan McRae via arch-dev-public
A big shout out to the Arch Conf Team!

This weekend's conference was nothing short of awesome.  It pulled
together very well, particularly given the short organisation period and
it being the first one run online.

The recorded talks followed by live Q worked very well, and I enjoyed
the live commentary by the community either on IRC or twitch.

Looking forward to next year!  :)

Allan


[arch-dev-public] Reproducible builds progress report #3

2020-05-29 Thread Allan McRae via arch-dev-public
Hi all,

A quick updated on the progress of reproducible builds.

You may have noticed a couple of large rebuilds that occurred recently.
These fixed issues of non-reproducible file ordering with old versions
of makepkg. This and other hard work by the team improving our tooling
and fixing packaging issues has resulted in 96% of [core] being
reproducible, and 90% of [extra]. You can see the status of which
packages are reproducible here [1].

The remaining packages to fix in [core] are dnssec-anchors, linux,
linux-lts, nss and perl. With the possible exception of perl, these are
in the "hard" basket. There is plans on how to fix the kernel packages,
but that will require some time to sort out. We would be happy for more
people to help out so we can get [core] to 100% reproducible.

We have investigated some of the packages in [extra] that fail to
reproduce here [2]. Note that there are quite a few packages that
currently "Failed to build from source" (FTBFS) - it would be very
helpful for the reproducible builds team if their maintainers can help
fix the packages. You can also use the CI of Arch packages run by Debian
to get an overview what the issue is with these packages and see many
other packages that are currently failing to build [3].

We also need help to investigate and fix the packages that fail to
reproduce that we have not investigated as of yet. There are two easy to
use tools to attempt to reproduce a package - "makerepropkg" from
devtools and "repro" from the archlinux-repro package. Once these have
rebuilt a package, you can use the "diffoscope" tool to look at the
differences. Jump in the #archlinux-reproducible IRC channel if you want
help interpreting the output, or you could just link to a copy of it in
the wiki.

[1] https://reproducible.archlinux.org/
[2] https://wiki.archlinux.org/index.php/Reproducible_Builds/Status
[3] https://tests.reproducible-builds.org/archlinux/extra.html


Re: [arch-dev-public] Use detached package signatures by default

2020-07-08 Thread Allan McRae via arch-dev-public
On 9/7/20 1:05 pm, Anatol Pomozov wrote:
> Given this information I would like to propose to stop using embedded
> signatures and move to detached signatures by default. This will
> require pacman 6.x or as alternative backport the fix(es) to 5.x
> branch. It will help to make system updates even faster, something
> that me and many other Arch users really love.

There are several steps we need to complete:

1) backport the patch (or wait for pacman-6.0, which may be a while
yet).  I'll leave that to the distro packagers to decide!

2) adjust repo-add to optionally add signatures.

3) make a time line that all users need to have the patched/released
pacman installed - we usually require at least 6 months.

4) turn off signature inclusion in repo dbs.

Allan