Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

[ Sorry, that fell off screen ]

Roger Shimizu wrote...

> [CC Vagrant, u-boot pkg maintainer]
> 
> On Sat, Dec 10, 2016 at 11:31 PM, Christoph Biedl
>  wrote:
> > Paul Wise wrote...
> >>
> >> Is it possible to put a bootloader like u-boot in the flash partitions
> >> and have it load the Linux kernel and initrd from elsewhere?
> >
> > That how I've been running my Dockstars through all the years. As as
> > far as I know this worked with the Debian kernels as well (I use my
> > own kernels for reasons).
> 
> If I understand correctly, it means boot like:
>   u-boot shipped by original vendor => self built u-boot => kernel
> (Debian's or your own customized one)

No, it's the U-Boot provided by Jeff Doozan, in the "u-boot" partition
on mtd. Everything else runs on USB.

Christoph



signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wouter Verhelst]

On Sat, Dec 17, 2016 at 08:15:15PM +0800, Paul Wise wrote:
> On Sat, 2016-12-17 at 09:45 +0100, Wouter Verhelst wrote:
> 
> > Yes, but that still says:
> 
> Ack.
> 
> > I think a proper procedure should involve a script that:
> > 
> > - is packaged in Debian;
> 
> Ack.
> 
> > - checks whether the hardware it's running on has all the hardware
> >   requirements for the new architecture
> 
> apt show arch-test
> 
> > - is properly tested to work in (almost) all situations;
> 
> jenkins.d.n would be the place to put full-system cross-grading tests.
> piuparts would be the place to put per-package cross-grading tests.
> 
> > - is a properly supported way to move from one ABI to another.
> 
> What do you mean by "properly supported"?

Where we don't go "it might break, and if it does, you get to keep the
pieces", but instead "if it breaks, that's a bug and we will fix it, and
try to help you recover from that".

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
   people in the world who think they really understand all of its rules,
   and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Sun, Dec 18, 2016 at 11:22 PM, Roger Shimizu wrote:

> Is there any way to simplify?

Remove the obsolete armel binaries where they occur and then mark the
packages as NFU on armel:

https://wiki.debian.org/ftpmaster_Removals
https://wiki.debian.org/PackagesArchSpecific

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

On Sat, Dec 10, 2016 at 12:22 PM, Wookey  wrote:
>
> We can do poor-mans partial arch by just being fairly agressive about
> disabling armel for packages that are broken or not suitable. Not very
> clever or efficient, but it is easy to do and requires no infra or
> tooling changes at all. So long as someone is volunteering for that
> (easy but unexciting) work that could work.

If I understand correctly, you mean for those packages broke on armel
(or to be broken in the future), we can modify debian/control as following:

From:
  Architecture: any
To:
  Architecture: any [!armel]

I'm not sure whether statement above works or not, because debian
policy didn't mention this [0][1].
It only mentioned statement like "any" or "linux-any".

If we cannot use "any [!armel]", we have to list all the ARCHes
without armel, so it's horrible like:
  Architecture: alpha, amd64, arm64, armhf, hppa, i386, m68k, mips,
mips64, mips64el, mipsel, mipsn32, mipsn32el, or1k, powerpc,
powerpcspe, ppc64, ppc64el, s390, s390x, sh4, sparc, sparc64, tilegx,
x32

And we need to write a few ARCHes that may be supported later [2]:
mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el

Is there any way to simplify?

[0] 
https://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Architecture
[1] 
https://www.debian.org/doc/debian-policy/ch-customized-programs.html#s-arch-spec
[2] https://anonscm.debian.org/cgit/kernel/linux.git/tree/debian/config/defines

Cheers,
--
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Sat, 2016-12-17 at 09:45 +0100, Wouter Verhelst wrote:

> Yes, but that still says:

Ack.

> I think a proper procedure should involve a script that:
> 
> - is packaged in Debian;

Ack.

> - checks whether the hardware it's running on has all the hardware
>   requirements for the new architecture

apt show arch-test

> - is properly tested to work in (almost) all situations;

jenkins.d.n would be the place to put full-system cross-grading tests.
piuparts would be the place to put per-package cross-grading tests.

> - is a properly supported way to move from one ABI to another.

What do you mean by "properly supported"?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wouter Verhelst]

On Thu, Dec 15, 2016 at 11:19:34AM +0800, Paul Wise wrote:
> On Thu, Dec 15, 2016 at 1:40 AM, Wouter Verhelst wrote:
> 
> > One way in which the need to keep armel around would be reduced is if we
> > could somehow upgrade from armel machines to armhf ones, without
> > requiring a reinstall.
> 
> There is a script for that here:
> 
> https://wiki.debian.org/CrossGrading

Yes, but that still says:

A full backup is also strongly recommended as this procedure is still very
much work in progress. Reinstalling is still the safer option. You have
been warned! 

I think a proper procedure should involve a script that:

- is packaged in Debian;
- checks whether the hardware it's running on has all the hardware
  requirements for the new architecture (i.e., in case of armel to
  armhf, can support the ARM ABI required by armhf; or in case of i386
  to amd64, is a processor that supports the x86_64 ABI), and produces a
  proper error message in case the required support isn't there;
- is properly tested to work in (almost) all situations;
- is a properly supported way to move from one ABI to another.

We currently don't have anything remotely like the above, and I think we
should.

[1] save that, I think, at the time multiarch didn't exist yet. Yes,
this was an "interesting" experience :-)

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
   people in the world who think they really understand all of its rules,
   and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Vagrant Cascadian]

On 2016-12-16, Roger Shimizu  wrote:
> On Sat, Dec 10, 2016 at 11:31 PM, Christoph Biedl
>  wrote:
>>> Is it possible to put a bootloader like u-boot in the flash partitions
>>> and have it load the Linux kernel and initrd from elsewhere?

There's no technical reason this wouldn't be possible, just a matter of
getting patches in upstream u-boot for that platform adding support for
loading from other media (SD, USB, sata, etc.), filesystem support and
so on, and staying withint the size constraints for that platform.

You might be able to use an SPL loader as a first-stage loader early in
the boot process to load a more capable u-boot from other flash
partitions and/or other media and/or filesystems. With the space
reserved for a linux kernel on flash media, that would presumably give
quite a bit of space for a full-featured u-boot.


>> That how I've been running my Dockstars through all the years. As as
>> far as I know this worked with the Debian kernels as well (I use my
>> own kernels for reasons).
>
> If I understand correctly, it means boot like:
>   u-boot shipped by original vendor => self built u-boot => kernel
> (Debian's or your own customized one)

While technically possible, u-boot upstream discourages chainloaded
u-boot. So while you could maybe make patches to get it working for a
particular board and set of vendor and upstream u-boot versions, I'm not
sure if patches would be accepted for upstream u-boot.


live well,
  vagrant


signature.asc
Description: PGP signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

[CC Vagrant, u-boot pkg maintainer]

On Sat, Dec 10, 2016 at 11:31 PM, Christoph Biedl
 wrote:
> Paul Wise wrote...
>>
>> Is it possible to put a bootloader like u-boot in the flash partitions
>> and have it load the Linux kernel and initrd from elsewhere?
>
> That how I've been running my Dockstars through all the years. As as
> far as I know this worked with the Debian kernels as well (I use my
> own kernels for reasons).

If I understand correctly, it means boot like:
  u-boot shipped by original vendor => self built u-boot => kernel
(Debian's or your own customized one)

I'm very interesting to this idea.
Is there any manual or blog post I can take reference?

Thank you!
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Paul Wise]

On Thu, Dec 15, 2016 at 1:40 AM, Wouter Verhelst wrote:

> One way in which the need to keep armel around would be reduced is if we
> could somehow upgrade from armel machines to armhf ones, without
> requiring a reinstall.

There is a script for that here:

https://wiki.debian.org/CrossGrading

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wouter Verhelst]

On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:
[...asking for armel to be retained...]

One way in which the need to keep armel around would be reduced is if we
could somehow upgrade from armel machines to armhf ones, without
requiring a reinstall.

After all, armel has been around longer than armhf has, which means that
there may be some machines out in the wild that were installed (and
upgraded) when armel existed but armhf did not yet (or at least, was not
stable yet). Some of those machines might be armv7 machines that would
be perfectly capable of running the armhf port, except that it wasn't
around yet when they were first installed, and switching to armhf
without reinstalling isn't possible.

I once did try to do a similar migration on my Thecus (from arm to
armel, rather than armel to armhf), but that failed miserably; and since
I hadn't installed the firmware update to be able to access the console
so as to figure out what went wrong, that essentially bricked the
machine.

If there was a supported and tested way to upgrade older armel
installations on hardware that actually works with armhf, then those
machines wouldn't need to be able to run armel anymore, and part of this
problem would go away...

-- 
< ron> I mean, the main *practical* problem with C++, is there's like a dozen
   people in the world who think they really understand all of its rules,
   and pretty much all of them are just lying to themselves too.
 -- #debian-devel, OFTC, 2016-02-12

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Lennart Sorensen]

On Wed, Dec 14, 2016 at 06:40:22PM +0100, Wouter Verhelst wrote:
> On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:
> [...asking for armel to be retained...]
> 
> One way in which the need to keep armel around would be reduced is if we
> could somehow upgrade from armel machines to armhf ones, without
> requiring a reinstall.
> 
> After all, armel has been around longer than armhf has, which means that
> there may be some machines out in the wild that were installed (and
> upgraded) when armel existed but armhf did not yet (or at least, was not
> stable yet). Some of those machines might be armv7 machines that would
> be perfectly capable of running the armhf port, except that it wasn't
> around yet when they were first installed, and switching to armhf
> without reinstalling isn't possible.
> 
> I once did try to do a similar migration on my Thecus (from arm to
> armel, rather than armel to armhf), but that failed miserably; and since
> I hadn't installed the firmware update to be able to access the console
> so as to figure out what went wrong, that essentially bricked the
> machine.
> 
> If there was a supported and tested way to upgrade older armel
> installations on hardware that actually works with armhf, then those
> machines wouldn't need to be able to run armel anymore, and part of this
> problem would go away...

I actually highly doubt there are that many armv7 boxes running armel.
armhf was a nice performance improvement and worth the hassle to reinstall
if you had such a box in the first place.  I think most armel systems
are probably armv5, often the marvell chips.  Not sure if anyone is
running it on Raspberry pi (Original, not 2 or 3) systems or if those
all run Rasbian armhf instead.

-- 
Len Sorensen

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Kurt Roeckx]

On Wed, Dec 07, 2016 at 03:53:31PM +, Steve McIntyre wrote:
> AFAIK there are potentially still similar problems with ARMv5 - lack
> of architcture-defined barrier primitives for C++11 atomics to
> work. (I'd love to be corrected on this if people know better!) This
> is one of the key points here. More and more software is expecting to
> use these primitives, and a lack of them is a major problem. To
> demonstrate using gcc, you can see that lock-free atomics only started
> appearing in ARMv6 and were improved in ARMv7:
> 
> $ for arch in 4 5 6 7-a; do echo ARMv${arch}; echo | g++ -march=armv${arch} 
> -dM -E - | grep -i lock_free; done
> ARMv4
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 1
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 1
> #define __GCC_ATOMIC_INT_LOCK_FREE 1
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LONG_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 1
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 1
> ARMv5
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 1
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 1
> #define __GCC_ATOMIC_INT_LOCK_FREE 1
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LONG_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 1
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 1
> ARMv6
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 1
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 1
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 2
> #define __GCC_ATOMIC_INT_LOCK_FREE 2
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
> #define __GCC_ATOMIC_LONG_LOCK_FREE 2
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 1
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 1
> ARMv7-a
> #define __GCC_ATOMIC_CHAR_LOCK_FREE 2
> #define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
> #define __GCC_ATOMIC_BOOL_LOCK_FREE 2
> #define __GCC_ATOMIC_POINTER_LOCK_FREE 2
> #define __GCC_ATOMIC_INT_LOCK_FREE 2
> #define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
> #define __GCC_ATOMIC_LONG_LOCK_FREE 2
> #define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 2
> #define __GCC_ATOMIC_LLONG_LOCK_FREE 2
> #define __GCC_ATOMIC_SHORT_LOCK_FREE 2

What you're actually showing is that even for ARMv4 they are
sometimes lock free by using the kernel support.

> There are kernel helpers available to provide some atomic support, but
> they'll be very slow compared to real hardware support at this level.

I was under the impression that that's not the case:
https://lwn.net/Articles/314561/


Kurt

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Timo Jyrinki]

2016-12-09 0:12 GMT+02:00 Christoph Biedl :
> Same here. My Dockstars (orion5x/kirkwood) still work like a charm and
> it gives a bad feeling having to trash them some day just because
> there's no support any more.
>
> On the other hand, they face another problem I guess is typical for
> that generation, just by the age: Memory.

There are new kirkwood devices still being produced and my <2y old one
(and still on sale in some places) is 2.0GHz one with 1GB RAM.

Just pointing out that ARMv5 is less a certain generation of CPUs as
such but a selection of instruction set by a chip manufacturer. It
will of course become rarer over time.

Thanks to all who have maintained armel so far, I'll be a happy user
for the duration of stretch support.

-Timo

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Wookey]

On 2016-12-07 15:53 +, Steve McIntyre wrote:
> On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:

> >I'm ARM porter on armel/marvell (orion5x/kirkwood).
> >Stretch will be frozen and released soon, which makes me bit depressed, 
> >because it means armel will be dropped out of unstable/testing as the 
> >conclusion of Cape Town BoF.
> >
> >> Possible future options for armel:
> >> 
> >>  * Partial armel architecture?
> >> 
> >>We've talked about this in the past for various architectures, but
> >>nobody has stepped up to work on it. The vast majority of the
> >>outstanding armel use cases are going to be in headless servers so
> >>we could maybe drop the desktop tasks and dependencies and keep
> >>things like web server / mail server tasks that are much simpler.

> >>Downside: There's no clear plan for how to create/maintain a
> >>partial architecture, let alone how to release one. Potentially
> >>huge amount of work needed.

We can do poor-mans partial arch by just being fairly agressive about
disabling armel for packages that are broken or not suitable. Not very
clever or efficient, but it is easy to do and requires no infra or
tooling changes at all. So long as someone is volunteering for that
(easy but unexciting) work that could work.

Obviously something fancier and more centralised/general would be
'better' but it requires a different skill-set and realistically will
probably take a long time.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Ben Hutchings]

On Fri, 2016-12-09 at 22:14 +0900, Roger Shimizu wrote:
> On Fri, 09 Dec 2016 00:53:17 +
> > Ben Hutchings  wrote:
> 
> > Also, dedicated tiny flash partitions for the kernel and initrd.  I
> > wouldn't be surprised to be find that by the time we want to release
> > buster we can't build a useful kernel that fits into the 2 MB partition
> > that most of these devices seem to have.
> 
> Actually those limitations are only for a few old orion5x models.
> For Linkstation kirkwood series I maintain for, kernel can be 2.5MB
> and initrd can be 8MB.

OK, that's much less of a problem.  Please add this information to the
comment on size limitations (debian/config/armel/defines).

> I'm not sure about the exact limitation size because the vendor doesn't
> disclose this specification clearly.

I think we should assume that whatever fits in the flash partition will
work, unless we discover another limitation in the boot loader.

Ben.

> But size listed above should be much easier to achieve for kernel 
> maintainers.
> 
> > As it is, stretch will be supported until 2020, maybe 2022 on armel. 
> > Is it really worthwhile to add another 2 years to that?
> 
> If the effort to maintain the armel is limited under certain level,
> and the those hardware are still in good condition, IMHO the longer
> the better.
> 
> Cheers,
-- 
Ben Hutchings
Logic doesn't apply to the real world. - Marvin Minsky



signature.asc
Description: This is a digitally signed message part

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

On Fri, 09 Dec 2016 00:53:17 +
Ben Hutchings  wrote:

> Also, dedicated tiny flash partitions for the kernel and initrd.  I
> wouldn't be surprised to be find that by the time we want to release
> buster we can't build a useful kernel that fits into the 2 MB partition
> that most of these devices seem to have.

Actually those limitations are only for a few old orion5x models.
For Linkstation kirkwood series I maintain for, kernel can be 2.5MB
and initrd can be 8MB.

I'm not sure about the exact limitation size because the vendor doesn't
disclose this specification clearly.
But size listed above should be much easier to achieve for kernel 
maintainers.

> As it is, stretch will be supported until 2020, maybe 2022 on armel. 
> Is it really worthwhile to add another 2 years to that?

If the effort to maintain the armel is limited under certain level,
and the those hardware are still in good condition, IMHO the longer
the better.

Cheers,
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1


pgpK8PTxHYXhU.pgp
Description: PGP signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Christoph Biedl]

Roger Shimizu wrote...

> I'm ARM porter on armel/marvell (orion5x/kirkwood).
> Stretch will be frozen and released soon, which makes me bit depressed, 
> because it means armel will be dropped out of unstable/testing as the 
> conclusion of Cape Town BoF.

Same here. My Dockstars (orion5x/kirkwood) still work like a charm and
it gives a bad feeling having to trash them some day just because
there's no support any more.

On the other hand, they face another problem I guess is typical for
that generation, just by the age: Memory. So for quite some time I
wanted to start a thread here but there are too many other things I
have to take care of. Now however that the discussion has started ...
I would have written something like the following:


As far as I know, there is no such thing as "minimal hardware
requirement" for Debian besides some lines in the installer pages. For
RAM, the minimum value is 128 megabytes, and I think it's about time
to raise that. Yes, you can run Debian on that but it's not fun:

Locale generation needs a lot of RAM. You can work around it by
installing locales-all which however takes long time to install on
slow flash drives. Or disable locales entirely. Err.

Side-effects of over-eagerly used xz compression. This is a relict of
time when xz was pretty new and maintainers added the -9 compression
option, probably completely unaware this makes no sense for small
packages, unlike gzip or bzip2. Also, even unpacking unconditionally
allocates some 65 Mbyte memory then, easiliy driving a 128 Mbyte-box
to the limits. One of the worst is the traceroute package that
actually carries some 110 kbyte; however I was unable to install it
due to the above problem.

The amount of data apt deals with reflects more or less the size of
the Packages indexes. As they get bigger each release, this will
become a problem. Already now apt gets OOM-killed every now and then.


About xz, I considered doing a MBF (severity normal the most) asking
to end this, it's about 45 packages.

About apt I have an idea to provide "reduced" Packages indexes. They
would be smaller so the memory usage is no longer a problem, also apt
should become somewhat faster. But since it's virtually impossible to
create a complete subset, I'd declare incompleteness a feature: If
somebody manages to hit the limits by requesting installation of
packages that are referenced only - tough luck, use the full index,
please. But somebody would have to promote and maintain that feature.


Another way to deal with this however was to raise the minimum memory
requirements. To 256 MB, or perhaps even 512 MB. It feels wrong since
it works around a problem instead of solving it. But since all
components become more greedy over time, this will have to be done
sooner or later anyway. And will render armel de-facto obsolete.

Still I wouldn't mind armel in buster, perhaps restricted to armv5.
But I understand the odds aren't quite in favour.

Christoph


signature.asc
Description: Digital signature

Re: armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Steve McIntyre]

On Wed, Dec 07, 2016 at 08:50:40PM +0900, Roger Shimizu wrote:
>[ intentionally keep d-d CCed ]
>
>On Fri, 22 Jul 2016 02:36:05 +0100
>Steve McIntyre  wrote:
>
>> [ Please note the cross-post and Reply-To ]
>> 
>> Hi folks,
>> 
>> As promised, here's a quick summary of what was discussed at the ARM
>> ports BoF session in Cape Town.
>
>Thanks for the summary!
>
>I'm ARM porter on armel/marvell (orion5x/kirkwood).
>Stretch will be frozen and released soon, which makes me bit depressed, 
>because it means armel will be dropped out of unstable/testing as the 
>conclusion of Cape Town BoF.
>
>So I'm writing to ask if there's any chance ...
>
>> We spoke about the past/present/future state of ARM ports in Debian.
>> 
>> armel
>> =
>> 
>> armel is the longest-running of our current ARM ports in Debian,
>> starting in 2007. It's starting to become difficult to support. Debian
>> is the only common distro still supporting ARMv4t. While there is
>> still good upstream support in most major packages (e.g. Linux and
>> gcc), we're seeing a growing number of issues. Some packages
>> (e.g. NodeJS) don't support pre-ARMv7 at all any more. Another problem
>> is the lack of direct support for C++11 atomics - it's possible to use
>> a kernel helper to cover for this lack, but performance will be
>> terrible.
>> 
>> Possible future options for armel:
>> 
>>  * Partial armel architecture?
>> 
>>We've talked about this in the past for various architectures, but
>>nobody has stepped up to work on it. The vast majority of the
>>outstanding armel use cases are going to be in headless servers so
>>we could maybe drop the desktop tasks and dependencies and keep
>>things like web server / mail server tasks that are much simpler.
>> 
>>Downside: There's no clear plan for how to create/maintain a
>>partial architecture, let alone how to release one. Potentially
>>huge amount of work needed.
>> 
>>  * Update to ARMv5t (and maybe go headless)
>> 
>>We might be able to extend the life of armel by upping the minimum
>>spec, and would be able to continue supporting Kirkwood and Orion
>>based machines (e.g. NAS boxes) for a while longer. This would kill
>>support for v4t devices like Openmoko, but there are precious few
>>of these older devices still around.
>
>Dropping v4t devices seems won't cause big problem, as you said
>there's few devices around currently. So I personally prefer this
>option.

AFAIK there are potentially still similar problems with ARMv5 - lack
of architcture-defined barrier primitives for C++11 atomics to
work. (I'd love to be corrected on this if people know better!) This
is one of the key points here. More and more software is expecting to
use these primitives, and a lack of them is a major problem. To
demonstrate using gcc, you can see that lock-free atomics only started
appearing in ARMv6 and were improved in ARMv7:

$ for arch in 4 5 6 7-a; do echo ARMv${arch}; echo | g++ -march=armv${arch} -dM 
-E - | grep -i lock_free; done
ARMv4
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __GCC_ATOMIC_POINTER_LOCK_FREE 1
#define __GCC_ATOMIC_INT_LOCK_FREE 1
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
#define __GCC_ATOMIC_LONG_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
ARMv5
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 1
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __GCC_ATOMIC_POINTER_LOCK_FREE 1
#define __GCC_ATOMIC_INT_LOCK_FREE 1
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 1
#define __GCC_ATOMIC_LONG_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
ARMv6
#define __GCC_ATOMIC_CHAR_LOCK_FREE 1
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
#define __GCC_ATOMIC_BOOL_LOCK_FREE 1
#define __GCC_ATOMIC_POINTER_LOCK_FREE 2
#define __GCC_ATOMIC_INT_LOCK_FREE 2
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
#define __GCC_ATOMIC_LONG_LOCK_FREE 2
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 1
#define __GCC_ATOMIC_LLONG_LOCK_FREE 1
#define __GCC_ATOMIC_SHORT_LOCK_FREE 1
ARMv7-a
#define __GCC_ATOMIC_CHAR_LOCK_FREE 2
#define __GCC_ATOMIC_CHAR32_T_LOCK_FREE 2
#define __GCC_ATOMIC_BOOL_LOCK_FREE 2
#define __GCC_ATOMIC_POINTER_LOCK_FREE 2
#define __GCC_ATOMIC_INT_LOCK_FREE 2
#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
#define __GCC_ATOMIC_LONG_LOCK_FREE 2
#define __GCC_ATOMIC_CHAR16_T_LOCK_FREE 2
#define __GCC_ATOMIC_LLONG_LOCK_FREE 2
#define __GCC_ATOMIC_SHORT_LOCK_FREE 2

There are kernel helpers available to provide some atomic support, but
they'll be very slow compared to real hardware support at this level.

>>Downside: as above if we try to do the partial architecture route;
>>if we don't we'll still have to support a full range of software on
>>older hardware.
>
>I don't see any detailed 

armel after Stretch (was: Summary of the ARM ports BoF at DC16)

[Roger Shimizu]

[ intentionally keep d-d CCed ]

On Fri, 22 Jul 2016 02:36:05 +0100
Steve McIntyre  wrote:

> [ Please note the cross-post and Reply-To ]
> 
> Hi folks,
> 
> As promised, here's a quick summary of what was discussed at the ARM
> ports BoF session in Cape Town.

Thanks for the summary!

I'm ARM porter on armel/marvell (orion5x/kirkwood).
Stretch will be frozen and released soon, which makes me bit depressed, 
because it means armel will be dropped out of unstable/testing as the 
conclusion of Cape Town BoF.

So I'm writing to ask if there's any chance ...

> We spoke about the past/present/future state of ARM ports in Debian.
> 
> armel
> =
> 
> armel is the longest-running of our current ARM ports in Debian,
> starting in 2007. It's starting to become difficult to support. Debian
> is the only common distro still supporting ARMv4t. While there is
> still good upstream support in most major packages (e.g. Linux and
> gcc), we're seeing a growing number of issues. Some packages
> (e.g. NodeJS) don't support pre-ARMv7 at all any more. Another problem
> is the lack of direct support for C++11 atomics - it's possible to use
> a kernel helper to cover for this lack, but performance will be
> terrible.
> 
> Possible future options for armel:
> 
>  * Partial armel architecture?
> 
>We've talked about this in the past for various architectures, but
>nobody has stepped up to work on it. The vast majority of the
>outstanding armel use cases are going to be in headless servers so
>we could maybe drop the desktop tasks and dependencies and keep
>things like web server / mail server tasks that are much simpler.
> 
>Downside: There's no clear plan for how to create/maintain a
>partial architecture, let alone how to release one. Potentially
>huge amount of work needed.
> 
>  * Update to ARMv5t (and maybe go headless)
> 
>We might be able to extend the life of armel by upping the minimum
>spec, and would be able to continue supporting Kirkwood and Orion
>based machines (e.g. NAS boxes) for a while longer. This would kill
>support for v4t devices like Openmoko, but there are precious few
>of these older devices still around.

Dropping v4t devices seems won't cause big problem, as you said there's few 
devices around currently.
So I personally prefer this option.

>Downside: as above if we try to do the partial architecture route;
>if we don't we'll still have to support a full range of software on
>older hardware.

I don't see any detailed downside reason here.
I think armel dropping v4t is just like i386 dropping 586-class CPU [0].
If we can dropping 586-class CPU support for i386, by changing a few 
configure files in gcc/dpkg/kernel packages, why we cannot do the same for 
armel?

[0] https://lists.debian.org/debian-devel-announce/2016/05/msg1.html

I'm a "high-level" ARM porter, merely knowing the basic device-tree stuff 
to support new device.
So I'm lack of knowledge in lower chipset level and maybe missed something 
here.
Could you kindly help to list the detailed work other than listed above?

>Will need volunteers to make this work in either form, and while
>some people are prepared to *help* (e.g. bdale and zumbi), nobody
>stepped up in the BoF to lead such an effort. It needs both skill
>and commitment of time.

If specific working items are clearly listed, I think there would be someone 
stepping out, since the armel user/devel base is still big.

Thanks and look forward to your feeback!

Cheers!
-- 
Roger Shimizu, GMT +9 Tokyo
PGP/GPG: 4096R/6C6ACD6417B3ACB1


pgp9gpmmdzHtl.pgp
Description: PGP signature

Re: Summary of the ARM ports BoF at DC16

[Steve McIntyre]

On Thu, Aug 11, 2016 at 11:47:12AM +, Riku Voipio wrote:
>On Fri, Jul 22, 2016 at 02:36:05AM +0100, Steve McIntyre wrote:
>> armhf
>> =
>> Current port, first released with Wheezy. Due to cross-distro effort,
>> this setup (ARMv7 EABI using VFPv3D-16) is the default supported
>> 32-bit ARM architecture in all distros now. We've got a couple of
>> kernel variants that will support just about any new devices shipping,
>> given updated drivers and device tree support.
>
>One issue with armhf is NEON support. Currently we support NEON as long
>upstreams provide a runtime mechanism to use it. There are however a
>couple of issues here
>
>- Some upstreams (chromium) do neon compile-time or require neon to
>  work at all (rustc)
>- The amount of non-NEON armv7 hw is very minimal (dove, tragra2, ?), so
>  majority of armhf users have less performance than they could
>- None of the armhf buildd's have NEON support, so no NEON code can be
>  run build-time (testuites etc)
>
>Once the next armhf buildd upgrade comes around (preferrably in form of
>armhf vm's running on arm64 servers), I think we should discuss make NEON a
>requirement for armhf.

I disagree strongly, for multiple reasons:

 * there are a non-negligible number of non-NEON ARMv7 machines in the
   wild
 * we spent a very long time agreeing across the distros on the spec
   for armhf, and I don't want to lose the cross-distro binary
   compatibility that we fought for
 * I don't think that moving the ABI is a valid thing to do for the
   sake of some upstreams doing the wrong/lazy thing.

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
Is there anybody out there?

Re: ARM64 (was: Summary of the ARM ports BoF at DC16)

[Wookey]

On 2016-07-27 00:53 -0400, Jeffrey Walton wrote:
> >> The Mustang board is a nice test platform because its an early ARMv8
> >> board. While its ARMv8, it lacks CRC and Crypto extensions.
> >
> > That's interesting.  Having almost given up on any AMD ARM
> > hardware ever appearing, I've been considering getting a
> > Gigabyte MP30-AR0 (which I thought for a long time was
> > vapourware,
> 
> Yeah, I purchased a SoftIron Overdrive 1000 recently. I also purchased
> the 3-day shipping for an additional $50 USD. I'm into my second week
> and the machine still has not arrived. The support@ contact, called
> out on the website, does not respond to emails. Other email addresses,
> like sales@, simply bounces.
> 
> I'm beginning to think SoftIron is a scam. I'm guessing I'm going to
> have to call VISA and to straighten things out.

SoftIron definitely do exist, and definitely have made hardware (I've
seen some). Not sure why they seem to be incommunicado/failing to send
you stuff.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: ARM64 (was: Summary of the ARM ports BoF at DC16)

[Jeffrey Walton]

>> The Mustang board is a nice test platform because its an early ARMv8
>> board. While its ARMv8, it lacks CRC and Crypto extensions.
>
> That's interesting.  Having almost given up on any AMD ARM
> hardware ever appearing, I've been considering getting a
> Gigabyte MP30-AR0 (which I thought for a long time was
> vapourware,

Yeah, I purchased a SoftIron Overdrive 1000 recently. I also purchased
the 3-day shipping for an additional $50 USD. I'm into my second week
and the machine still has not arrived. The support@ contact, called
out on the website, does not respond to emails. Other email addresses,
like sales@, simply bounces.

I'm beginning to think SoftIron is a scam. I'm guessing I'm going to
have to call VISA and to straighten things out.

Jeff

Re: Summary of the ARM ports BoF at DC16

[Sune Vuorela]

On 2016-07-22, Steve McIntyre  wrote:
> is the lack of direct support for C++11 atomics - it's possible to use
> a kernel helper to cover for this lack, but performance will be
> terrible.

This is going to be a real problem for Qt in the very near future. A
requirement for qt 5.7 is a c++11 compliant compiler. Or well. The
whatever subset that msvc2012 can handle.
And one of them is the atomics.

I do think that in 2016, it is a fair requirement to be able to use
languages standardized in 2011.

/Sune

Re: ARM64 (was: Summary of the ARM ports BoF at DC16)

[Wookey]

On 2016-07-23 18:46 +, Phil Endecott wrote:
> > > Affordable, usable machines are available now, e.g. the Cello
> 
> Is the Cello actually available?  96boards is still saying "pre-order".

I have seen one, but that was '1st batch'. I don't know current
status, but yes I guess it's still under 'Real Soon Now'.

> And the ODROID-C2.  (Can someone add that to the list at 
> https://wiki.debian.org/Arm64Port#Hardware.2C_emulators_and_models ?)

Do you think the 'wiki' part of that URL gives a clue, Phil?
Still, I've looked up the home page for that device and done it for you :-)
 
> > The Mustang board is a nice test platform because its an early ARMv8
> > board. While its ARMv8, it lacks CRC and Crypto extensions.
> 
> That's interesting.  Having almost given up on any AMD ARM 
> hardware ever appearing, I've been considering getting a 
> Gigabyte MP30-AR0 (which I thought for a long time was 
> vapourware, but it seems does really exist to buy); it's based 
> on the same x-gene as the Mustang.  But this suble instruction 
> set difference is worrying; will I discover in a few years that 
> the baseline requirements for Debian arm64, or something 
> else, exclude this device?

The baseline won't change from v8 to v8.1, but upstream bits of
software might be written to not work without those instructions, and
Debian may not have the resources to do much about this. Everyone
_should_ be supporting v8, but they may not do a good job of this and
if there isn't much hardware about which checks the
non-CRC-in-hardware & non-crypto-in-hardware code path then people may
not notice that they've broken it.

However currently Debian is building on APM hardware and it provides
(part of) the Linaro developer cloud so for the time being this is not
going to be a problem.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: ARM64 (was: Summary of the ARM ports BoF at DC16)

[Phil Endecott]

> > Affordable, usable machines are available now, e.g. the Cello

Is the Cello actually available?  96boards is still saying "pre-order".

> There are three other ARM64 gadgets worth mentioning... 

And the ODROID-C2.  (Can someone add that to the list at 
https://wiki.debian.org/Arm64Port#Hardware.2C_emulators_and_models ?)

> The Mustang board is a nice test platform because its an early ARMv8
> board. While its ARMv8, it lacks CRC and Crypto extensions.

That's interesting.  Having almost given up on any AMD ARM 
hardware ever appearing, I've been considering getting a 
Gigabyte MP30-AR0 (which I thought for a long time was 
vapourware, but it seems does really exist to buy); it's based 
on the same x-gene as the Mustang.  But this suble instruction 
set difference is worrying; will I discover in a few years that 
the baseline requirements for Debian arm64, or something 
else, exclude this device?


Cheers, Phil.

Re: Summary of the ARM ports BoF at DC16

[Paul Wise]

On Fri, Jul 22, 2016 at 9:36 AM, Steve McIntyre wrote:

> armel
> =
>
> armel is the longest-running of our current ARM ports in Debian,
> starting in 2007. It's starting to become difficult to support. Debian
> is the only common distro still supporting ARMv4t.

I note armel is still more popular than armhf but the gap is slowly
being closed and looks like the situation will be reversed by two
years time.

http://popcon.debian.org/
http://popcon.debian.org/stat/submission-1year.png

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

Re: Summary of the ARM ports BoF at DC16

[peter green]

On 22/07/16 02:36, Steve McIntyre wrote:

Affordable, usable machines are available now, e.g. the Cello
Does anyone know what is going on with this. Have boards actually 
started shipping?

Re: ARM64 (was: Summary of the ARM ports BoF at DC16)

[Marcin Juszkiewicz]

W dniu 22.07.2016 o 04:02, Jeffrey Walton pisze:


There are three other ARM64 gadgets worth mentioning...



* Raspberry Pi 3 Model B (http://www.amazon.com/dp/B01CD5VC92), $38 USD


This device has armv8 core but no support for running Linux in 64-bit mode.


The Mustang board is a nice test platform because its an early ARMv8
board. While its ARMv8, it lacks CRC and Crypto extensions.


CRC and Crypto were optional features at that time. They are mandatory 
for ARMv8.1 (or .2) designs.

ARM64 (was: Summary of the ARM ports BoF at DC16)

[Jeffrey Walton]

> arm64
> =
>
> Most recent ARM port. All looking good now - we've been mostly able to
> move on from Juno development platforms to real server hardware
> now. We're using some APM Mustang machines and an AMD Seattle box
> hosted by ARM and Linaro at the moment, and even real arm64 server
> machines are finally coming available. Looking to move more of our ARM
> buildds over onto these arm64 servers in the future, as this will
> improve reliability and manageability.
>
> We're using a single kernel for arm64, using DTB or ACPI for
> configuration. Works well.
>
> Affordable, usable machines are available now, e.g. the Cello or
> SoftIron 1000 for ~$600. Linaro's 96boards machines are not using a
> standard form factor which is sub-optimal. There's a Marvell arm64
> board due soon (H2 2016) in ATX. The SoftIron 1000 doesn't have PCI,
> but should be good for development/buildd otherwise. Another cheap
> option is the Pine64. It's stuck on a vendor kernel for now, but
> that's being steadily worked on. Cheap: quad A53, 2GB RAM, $29.

Hats off for the ARM64 support. The best I can tell, the support is
complete - ARM64 performs just like its i386 and x86_64 older
brothers.

There are three other ARM64 gadgets worth mentioning... We purchased
them after our package maintainer, László Böszörményi, filed some bug
reports against us. We did not like playing catch-up in the QEMU
Chroot's; we wanted to get ahead of things.

* LeMaker HiKey (http://www.amazon.com/dp/B019O3QTSA), $169 USD
* SnapDragon 410c (http://www.amazon.com/dp/B01600X7IU), $75 USD
* Raspberry Pi 3 Model B (http://www.amazon.com/dp/B01CD5VC92), $38 USD

The Mustang board is a nice test platform because its an early ARMv8
board. While its ARMv8, it lacks CRC and Crypto extensions. Robert
Nelson lends us time on his Mustang, and it uncovered two bugs in our
CPU feature detection logic.

Jeff

Summary of the ARM ports BoF at DC16

[Steve McIntyre]

[ Please note the cross-post and Reply-To ]

Hi folks,

As promised, here's a quick summary of what was discussed at the ARM
ports BoF session in Cape Town.

This year, unfortunately the session did not have video so I can't
link to it. I've taken a copy of the Gobby notes from the session,
alongside my small set of slides for the session (that also didn't get
shown!). [1]

We spoke about the past/present/future state of ARM ports in Debian.

armel
=

armel is the longest-running of our current ARM ports in Debian,
starting in 2007. It's starting to become difficult to support. Debian
is the only common distro still supporting ARMv4t. While there is
still good upstream support in most major packages (e.g. Linux and
gcc), we're seeing a growing number of issues. Some packages
(e.g. NodeJS) don't support pre-ARMv7 at all any more. Another problem
is the lack of direct support for C++11 atomics - it's possible to use
a kernel helper to cover for this lack, but performance will be
terrible.

Possible future options for armel:

 * Partial armel architecture?

   We've talked about this in the past for various architectures, but
   nobody has stepped up to work on it. The vast majority of the
   outstanding armel use cases are going to be in headless servers so
   we could maybe drop the desktop tasks and dependencies and keep
   things like web server / mail server tasks that are much simpler.

   Downside: There's no clear plan for how to create/maintain a
   partial architecture, let alone how to release one. Potentially
   huge amount of work needed.

 * Update to ARMv5t (and maybe go headless)

   We might be able to extend the life of armel by upping the minimum
   spec, and would be able to continue supporting Kirkwood and Orion
   based machines (e.g. NAS boxes) for a while longer. This would kill
   support for v4t devices like Openmoko, but there are precious few
   of these older devices still around.

   Downside: as above if we try to do the partial architecture route;
   if we don't we'll still have to support a full range of software on
   older hardware.

   Will need volunteers to make this work in either form, and while
   some people are prepared to *help* (e.g. bdale and zumbi), nobody
   stepped up in the BoF to lead such an effort. It needs both skill
   and commitment of time.

 * Release armel with stretch, then drop it

   **This is the favoured option.** That would give EOL for armel in
   2020, *potentially* later if there's an LTS release for stretch and
   armel is included (as it is in wheezy). That would depend on
   external labour or funding.

   It's not easy (or popular) to remove an architecture from Debian,
   but we're giving 4 years' notice of end of support.

armhf
=

Much as in the Heidelberg BoF in terms of support and future.

Current port, first released with Wheezy. Due to cross-distro effort,
this setup (ARMv7 EABI using VFPv3D-16) is the default supported
32-bit ARM architecture in all distros now. We've got a couple of
kernel variants that will support just about any new devices shipping,
given updated drivers and device tree support.

Despite the ubiquity of v7 hardware that works, some people are still
making CPUs that *don't* work, e.g. new CPUs this year without any
hardware FPU. Not much we can or will do to support such broken
hardware.

Ignoring known-broken hardware stuff, almost any currently-shipping
ARMv7 device is likely to work with armhf. Yay!

There's been a recent effort to add an EFI shim layer into modern
U-Boot, which may allow for easier consistent boot support but there's
still issues to think about in terms of EFI variables etc.

arm64
=

Most recent ARM port. All looking good now - we've been mostly able to
move on from Juno development platforms to real server hardware
now. We're using some APM Mustang machines and an AMD Seattle box
hosted by ARM and Linaro at the moment, and even real arm64 server
machines are finally coming available. Looking to move more of our ARM
buildds over onto these arm64 servers in the future, as this will
improve reliability and manageability.

We're using a single kernel for arm64, using DTB or ACPI for
configuration. Works well.

Affordable, usable machines are available now, e.g. the Cello or
SoftIron 1000 for ~$600. Linaro's 96boards machines are not using a
standard form factor which is sub-optimal. There's a Marvell arm64
board due soon (H2 2016) in ATX. The SoftIron 1000 doesn't have PCI,
but should be good for development/buildd otherwise. Another cheap
option is the Pine64. It's stuck on a vendor kernel for now, but
that's being steadily worked on. Cheap: quad A53, 2GB RAM, $29.

Other issues:

 * ci.debian.net could do with more arm64 hardware, we'll see what we
   can do to help
 * UEFI Secure Boot for arm64 has an issue - there's nobody providing
   a root CA. HP are doing their own for now; not sure what other
   vendors can/will do. It's complicated (TM).

arm64 boards -