Re: Usage of real m68k hardware

[Ingo Jürgensmann]

Am 17.04.2018 um 19:15 schrieb Thorsten Glaser :

>> Yes, of course. But Andreas hit a nerve with this on me. This project
>> has cost me lots of blood, tears and sweat and if someone is asking
>> for it to be completely thrown out out of nothing, I'm getting a bit
>> stressed out.
> I completely agree here. While I’m no longer involved with the
> m68k port specifically, after having spent THREE YEARS of blood,
> sweat and pain to resurrect it, there are several reasons:

I’m still very thankful for your efforts! Really!

> • I have come to actually like that, having been a die-hard 8088
>  user in my childhood, and found the people and community very
>  interesting
>  ‣ there are fun projects like a PCI bridge, which allows using
>a PCI Radeon graphics card with LCDs at 1900x12something
>resolution, currently with GEM/AES only, not yet in Linux

Actually there are some nice developments like 
http://www.apollo-accelerators.com/ to increase the speed of m68k for quite a 
few bugs. 

> • it sends a signal, and the wrong signal in my eyes, that
>  everything not-mainstream is not worth to be supported
>  ‣ specialisation is for downstreams, Debian should stay universal
>  ‣ read up on monoculture in agriculture and why everyone, by now,
>thinks it’s a bad idea
>⇒ hint: Meltdown/Spectre…

Yes, I think this is the main problem since m68k has been kicked out as a 
release arch. This whole second class architecture is a mistake, IMHO. Another 
approach would have been better, like focussing on being release-ready only for 
base and other essential packages, but not the whole archive. 
This effectively killed m68k in the long run. Other archs followed then. 

> • I found Debian ports very useful to gain deep insight on
>  how Debian and all of its components work, and can recommend
>  porting a new or resurrecting an old architecture to everyone
>  wishing to peek below the surface

That’s maybe the only positive thing that evolved in the aftermaths of kicking 
out m68k: a parallel infrastructure was developped that could act without all 
those complicated formalisms of official buildds (at least in the early days). 
But I think this could have been achieved without kicking archs out of Debian. 

I think especially m68k did a great job in teaching many DDs how to deal with 
autobuilders and such. Buildd & co were built, because of m68k and Debian. The 
very first buildd was running on kullervo. 

> On the more technical side, while Adrian’s buildds are qemu,
> I’ve continued running an ARAnyM (also emulation, but different
> and thanks to Doko even FPU-complete) buildd for as long as the
> system it was hosted on allowed me to do so. (That GPLhost domU
> is currently unusable because of spontaneous reboots and other
> problems. I might look into running one on some other system;
> I have a couple of VMs on my workplace desktop but can’t use
> those as they are bridged into the company LAN.)

I’m still not a big fan of emulated buildds. ;-) 
But I have to admit that they are way faster than the old, real hardware.

> We also have a number of Amiga and Atari and I believe at least
> one or two Macintosh systems which, at one point or the other,
> are or were in use as buildds and/or porterboxen.

Well, the last info from buildd.net database: 

buildd=# select name, model, cpu, ram from status where arch='m68k';
name|   model   |  cpu   | ram
+---++-
 washi  | Atari Falcon CT60 | 68060/66   | 256
 prometheus | Aranym/distcc | 733MHz PowerPC | 256
 minthe | Aranym| 8*Xeon 2G  | 768
 phoebe | Aranym| 8*Xeon 2G  | 768
 hobbes | Atari Falcon CT60 | 68060/95   | 512
 merlin | Amiga 1200| 68030/56   |  64
 elgar  | Amiga 4000| 68060/50   | 128
 kullervo   | Amiga 3000UX  | 68060/50   | 128
 crest  | Amiga 4000| 68060/50   | 128
 pacman | ARAnyM| VM040/240  | 512
 vivaldi| Amiga 4000T   | 68060/50   | 384
 theia  | Aranym| Dual 1.8 GHz   | 750
 wario  | ARAnyM| VM040/180  | 768
 zlin3  | Aranym| i386   |  64
 spice  | Amiga 3000| 68040/40   | 320
 aahz   | Amiga 2000| 68060/50   | 128
 akire  | Amiga 2000| 68060/50   | 128
 ara5   | ARAnyM| VM040/170  | 782
 arrakis| Amiga 3000| 68060/50   | 384
 kirby  | ARAnyM| VM040/214  | 512
 pikachu| ARAnyM| VM040/200  | 768
(21 rows)

At least crest, akire and elgar might be still online, maybe kullervo as well, 
but Christian can comment on this, while spice, arrakis & vivaldi are currently 
offline as in powered off or has a NIC that is currently not supported by Linux 
(spice).

I’m not totally opposed in powering on one or two machines again, but 

Re: Usage of real m68k hardware

[Thorsten Glaser]

Adrian wrote:

>Yes, of course. But Andreas hit a nerve with this on me. This project
>has cost me lots of blood, tears and sweat and if someone is asking
>for it to be completely thrown out out of nothing, I'm getting a bit
>stressed out.

I completely agree here. While I’m no longer involved with the
m68k port specifically, after having spent THREE YEARS of blood,
sweat and pain to resurrect it, there are several reasons:

• odd architectures *do* help in finding odd bugs, often before
  they hit other architectures where they’re hidden by, for
  example, compiler optimisations (more aggressive inlining)
  or arch-specific asm code, until these hiding things no longer
  appear

  ‣ granted, m68k has this specific “2-byte alignment” thing,
but then, anything that actually relies on the precise amount
of struct padding is relying on IB/UB in the first place and,
with that, buggy

• I have come to actually like that, having been a die-hard 8088
  user in my childhood, and found the people and community very
  interesting
  ‣ there are fun projects like a PCI bridge, which allows using
a PCI Radeon graphics card with LCDs at 1900x12something
resolution, currently with GEM/AES only, not yet in Linux

• it sends a signal, and the wrong signal in my eyes, that
  everything not-mainstream is not worth to be supported

  ‣ specialisation is for downstreams, Debian should stay universal

  ‣ read up on monoculture in agriculture and why everyone, by now,
thinks it’s a bad idea
⇒ hint: Meltdown/Spectre…

• I’m running (and helping to work on) x32… another port

• I found Debian ports very useful to gain deep insight on
  how Debian and all of its components work, and can recommend
  porting a new or resurrecting an old architecture to everyone
  wishing to peek below the surface

• I’ve heard someone’s working on making dak dports-capable,
  solving the current worst problem of the fact we use mini-dak
  in that NBS packages are removed from the archive even if
  they’re still Depended upon, which made me really excited
  about dports work


On the more technical side, while Adrian’s buildds are qemu,
I’ve continued running an ARAnyM (also emulation, but different
and thanks to Doko even FPU-complete) buildd for as long as the
system it was hosted on allowed me to do so. (That GPLhost domU
is currently unusable because of spontaneous reboots and other
problems. I might look into running one on some other system;
I have a couple of VMs on my workplace desktop but can’t use
those as they are bridged into the company LAN.)

We also have a number of Amiga and Atari and I believe at least
one or two Macintosh systems which, at one point or the other,
are or were in use as buildds and/or porterboxen.

I’ve also made a point in making ready-to-use ARAnyM images
in the Debian wiki which any maintainer could use to run a
box locally, due to the lack of currently-accessible porterboxen.
This has eroded a bit since I moved away from m68k.

I don’t know how the actual hardware can be helped to become
more usable. I also don’t know if the standard Debian porterbox
setup can be used on/for them. DSA normally does these things;
in dports we want to make things as closely to the main Debian
as possible, but as long as dports are officially unsupported,
it’s hard. (Also, you’d have to talk to Ingo, perhaps Adrian
and ragnar76 about the actual hardware.)

As for the “qemu bug” issue, using an ARAnyM, Amiga, Atari
or Macintosh machine to retry the build (since they all are
slower, although my previous desktop could emulate a 200 MHz
m68k with ARAnyM) before complaining would certainly help.
But this is also not easy, and only a few problems are caused
by qemu issues (I’m actually surprised, I’d have not thought
a qemu-based buildd a viable solution, and I recall Adrian
and me fighting a bit over it initially), so I don’t understand
An3as’ violent reaction.

Contrasted to that, x32 hardware is actually easy: use an amd64
system with “syscall.x32=y” in GRUB_CMDLINE_LINUX then just use
a foreign-arch chroot with pbuilder/cowbuilder or schroot/sbuild
like you would with i386. (I’ve had two cases in which an FTBFS
was actually a hiccup of the buildd, or a difference in host CPU,
and which built just fine on my system, again, in a clean
environment, so I just did a porter upload.) These are dead useful
to reproduce issues, by the way.


It might also be useful to create one or two buildds with
large hard discs (and possibly RAM) since some of the recent
packages (gcc-*-cross-* most prominently) make Adrian’s
systems explode… especially as his virtual buildds share
(limited) space.


Adrian is currently the single most-involved person driving
debian-ports forwards, on a *lot* of architectures, (not saying
there are no other porters) so I can understand his frustration.

I might even look if I can help any further. Unfortunately, as
I said above, I have no easy solution for running a buildd or
porterbox (company 

Re: Usage of real m68k hardware

[Michael Gilbert]

On Wed, Mar 28, 2018 at 2:24 PM, Adrian Bunk wrote:
> The core problem is that in the control file the permitted syntax for
> the Architecture: filed is much more restricticted than the permitted
> syntax for (build) dependencies.

This is dpkg bug #807264.

Best wishes,
Mike

Re: Usage of real m68k hardware

[Andreas Tille]

On Thu, Mar 29, 2018 at 02:07:41PM +0200, Wouter Verhelst wrote:
> > I conclude that the Debian project is running no real m68k hardware any
> > more (please correct me if I'm wrong) and we are possibly doing a
> > service for some users who potentially also run qemu (wild guess of
> > mine).  I'm wondering when it might be time to fully drop a hardware
> > port instead of draining developer time for ethernity.
> 
> I understand your desire to not be bothered with bugs that are not
> yours. That makes sense.
> 
> However, it does *not* make sense to then tell others that their work is
> not welcome anymore. If people want to spend time doing what you think
> is useless busywork, then, as long as they don't bother you with things
> that aren't your fault, why not let them?

ACK.  I confirm that my wording sucks and will be more carefull next
time.  Sorry again

 Andreas.

-- 
http://fam-tille.de

Re: Usage of real m68k hardware

[Wouter Verhelst]

On Wed, Mar 28, 2018 at 08:38:10AM +0200, Andreas Tille wrote:
> Hi,
> 
> recently some R packages received bugs that seem to stem from a problem
> with the build setup (specifically, a qemu bug).  When I asked back in
> one of the bugs[1] whether there are real m68k users I've got the answer
> 
>   ... there are still some users with actual hardware, but the
>   autobuilders use qemu for better performance and/or reliability
> 
> I conclude that the Debian project is running no real m68k hardware any
> more (please correct me if I'm wrong) and we are possibly doing a
> service for some users who potentially also run qemu (wild guess of
> mine).  I'm wondering when it might be time to fully drop a hardware
> port instead of draining developer time for ethernity.

I understand your desire to not be bothered with bugs that are not
yours. That makes sense.

However, it does *not* make sense to then tell others that their work is
not welcome anymore. If people want to spend time doing what you think
is useless busywork, then, as long as they don't bother you with things
that aren't your fault, why not let them?

Under "don't bother you with things", I mean to say that:
- Bugs should not be filed unless it can be reproduced on actual
  hardware.
- Bugs should not be filed at RC severity unless the bug is reproducible
  on a release architecture, or can be proven to be an actual bug in the
  package

As for the former: Adrian, I recently passed a bunch of m68k machines to
you. Would it make sense to set (some of) those up as secondary buildd
hosts? That is, try to build everything on emulated machines first
(because those are much faster). If something FTBFS, try to rebuild it
on hardware. If it FTBFS there, bugs can be filed; if not, upload the
package and track down the emulator bug...?

(if you're already working on that, then ignore what I said :-)

-- 
Could you people please use IRC like normal people?!?

  -- Amaya Rodrigo Sastre, trying to quiet down the buzz in the DebConf 2008
 Hacklab

Re: Usage of real m68k hardware

[Russ Allbery]

Philipp Kern  writes:
> On 03/28/2018 07:26 PM, Russ Allbery wrote:

>> Back when I was maintaining OpenAFS, I frequently wanted some way as a
>> maintainer to easily tag a package as "this will never for the
>> forseeable future be supported on this architecture" and move on.  We
>> don't have a great mechanism for doing this right now -- there's a
>> thing on the buildds that's pretty opaque and that I don't know how to
>> set as a maintainer, and one can list a bunch of specific architectures
>> on the package but that's really awkward and interacts poorly with
>> arch: all packages.

> The recommended way today is to annotate within the package. It does not
> actually interact poorly with arch:all packages. When dpkg builds the
> source package and there's no arch:any package it will list all
> architectures explicitly in the .dsc and if there's an arch:all package
> all will be added in addition.

Ah, good to know.  However, this still means that you have to explode
"any" into a list of every architecture Debian supports except one or two,
and then every time someone wants to port Debian to a new architecture,
they have to ask you to change your package (my prior experience is that
the package would work on new architectures more often than not, since
they were often variations of existing supported architectures, like
64-bit versions of things that already worked with 32 bits).

So unless I'm missing something this still seems less than ideal

-- 
Russ Allbery (r...@debian.org)   

Re: Enhanced syntax for Architecture field (Was: Usage of real m68k hardware)

[Adam Borowski]

On Wed, Mar 28, 2018 at 08:28:23PM +0200, Andreas Tille wrote:
> On Wed, Mar 28, 2018 at 09:24:12PM +0300, Adrian Bunk wrote:
> > 
> > Dependency fields have negative syntax like !m68k, for the Architecture: 
> > field this is only possible with a complete list of all architectures
> > except the one you want to exclude.
> 
> I agree this could really simplify things and I would have used this
> syntax in several cases.

Note that pretty much every place we have a list of architectures expects a
different syntax.  This causes confusion, and, as you say, is painful due to
a lack of such functionality.

I don't remember the details, but IIRC dependencies also have a limit of not
allowing both positive and negative specifiers.

Thus, if we'd change this, what about changing the rules to:
an architecture is allowed iff:
it matches at least one positive specifier -or- there's none,
and it doesn't match any negative specifier.
(Ie, negatives have priority, no positives assumes "any".)

The other change I'd make would be adding extra wildcards:
* {big,little}-endian
* {32,64,128¹}-bit
* "fast" (and/or it's near-complement "slow")

Thus, Andreas would mark his package, avoiding wasting time of either his or
small-arch porters.


Meow!

[1]. riscv128 already exists, although not in dpkg's arch table.  And at
least Google has enough storage in its servers to require more than 64-bits
to address if you use it as virtual memory.  And, recent extension to la57
was just 12 years after la48, which gives an estimate for when 64 bits won't
be enough for anyone even for a single machine.
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.

Re: Usage of real m68k hardware

[Philipp Kern]

On 03/28/2018 07:26 PM, Russ Allbery wrote:
> Back when I was maintaining OpenAFS, I frequently wanted some way as a
> maintainer to easily tag a package as "this will never for the forseeable
> future be supported on this architecture" and move on.  We don't have a
> great mechanism for doing this right now -- there's a thing on the buildds
> that's pretty opaque and that I don't know how to set as a maintainer, and
> one can list a bunch of specific architectures on the package but that's
> really awkward and interacts poorly with arch: all packages.

The recommended way today is to annotate within the package. It does not
actually interact poorly with arch:all packages. When dpkg builds the
source package and there's no arch:any package it will list all
architectures explicitly in the .dsc and if there's an arch:all package
all will be added in addition.

Kind regards
Philipp Kern

Enhanced syntax for Architecture field (Was: Usage of real m68k hardware)

[Andreas Tille]

Hi Adrian,

On Wed, Mar 28, 2018 at 09:24:12PM +0300, Adrian Bunk wrote:
> 
> Dependency fields have negative syntax like !m68k, for the Architecture: 
> field this is only possible with a complete list of all architectures
> except the one you want to exclude.

I agree this could really simplify things and I would have used this
syntax in several cases.

Kind regards

 Andreas.

-- 
http://fam-tille.de

Re: Usage of real m68k hardware

[Adrian Bunk]

On Wed, Mar 28, 2018 at 10:26:28AM -0700, Russ Allbery wrote:
>...
> The chances of anyone really wanting to run some of this scientific
> software on m68k seem remote, so it feels like it would be an overall
> reduction of friction if the maintainer could just say "I don't support
> this arch" and the porters could stop looking at those packages (unless
> for some reason they disagree with the maintainer and think users of that
> arch are really interested).

The core problem is that in the control file the permitted syntax for 
the Architecture: filed is much more restricticted than the permitted 
syntax for (build) dependencies.

Dependency fields have negative syntax like !m68k, for the Architecture: 
field this is only possible with a complete list of all architectures
except the one you want to exclude.

cu
Adrian

-- 

   "Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
   "Only a promise," Lao Er said.
   Pearl S. Buck - Dragon Seed

Re: Usage of real m68k hardware

[Russ Allbery]

Holger Levsen  writes:

> I'd suggest you let go and stop caring about m68k. m68k has been dropped
> from Debian many releases ago, thus IMO bugs affecting only m68k are
> probably at most normal severity, though minor or wishlist IMO make
> equally sense. Or just closing them as out of scope. Or you could tag
> them "help" and move on. The important thing is to let go and move on.
>  
> And please, m68k folks, dont get me wrong, I'm fascinated by your work
> and applaud your efforts. I just dont think they should get in the way
> of working on Debian (on our supported hardware plattforms).

Back when I was maintaining OpenAFS, I frequently wanted some way as a
maintainer to easily tag a package as "this will never for the forseeable
future be supported on this architecture" and move on.  We don't have a
great mechanism for doing this right now -- there's a thing on the buildds
that's pretty opaque and that I don't know how to set as a maintainer, and
one can list a bunch of specific architectures on the package but that's
really awkward and interacts poorly with arch: all packages.

The chances of anyone really wanting to run some of this scientific
software on m68k seem remote, so it feels like it would be an overall
reduction of friction if the maintainer could just say "I don't support
this arch" and the porters could stop looking at those packages (unless
for some reason they disagree with the maintainer and think users of that
arch are really interested).

-- 
Russ Allbery (r...@debian.org)   

Re: Usage of real m68k hardware

[Sean Whitton]

Hello,

There is clearly some dissatisfaction between the science team and the
m68k porters.  Several behaviours have been identified in various mails
to this thread that are causing friction for the other team's work.

ISTM that these issues could well arise between any group of porters and
any team maintaining a large number of packages.  We should be able to
develop BTS usage strategies that reduce this friction.  We've done this
before (e.g. how we do NMUs) and can surely come up with something
sensible.

What do we have right now?  AFAICT our current documentation is just
sections 5.10.1 and 5.10.2 of the Developers' Reference.  These sections
are quite short.  I would like to suggest that we expand those sections
with further best practices.  Perhaps Andreas and Adrian could propose
some patches to devref and get members of the other team to review.

I don't have any concrete suggestions myself because my packages are
almost all arch:all so I have barely had to deal with porting issues.
It seems clear, though, that our best practices are a bit lacking in
this area.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Re: Usage of real m68k hardware

[Holger Levsen]

Hi Andreas,

I'd suggest you let go and stop caring about m68k. m68k has been dropped
from Debian many releases ago, thus IMO bugs affecting only m68k are
probably at most normal severity, though minor or wishlist IMO make
equally sense. Or just closing them as out of scope. Or you could tag
them "help" and move on. The important thing is to let go and move on.
 
And please, m68k folks, dont get me wrong, I'm fascinated by your work and
applaud your efforts. I just dont think they should get in the way of
working on Debian (on our supported hardware plattforms). 


-- 
cheers,
Holger


signature.asc
Description: PGP signature

Re: Usage of real m68k hardware

[Ian Jackson]

Andreas Tille writes ("Re: Usage of real m68k hardware"):
> Just to not make that mistake again:  Is it the
> 
>   "I'm wondering when it might be time to fully drop a hardware
>port instead of draining developer time for ethernity"
> 
> in my mail you stumbled upon.  May be its the language barrier but I do
> not think that if a random developer is wondering about something this
> is equivalent to "asking for the ultima ratio".  I was honestly thinking
> about whether we are keeping alive an architecture by everybody using
> emulators.  I try to be more careful in my wording in the future when
> asking questions about ports (and hope I will manage).

I read your the part you quote above as a rhetorical question[1], and
I also found it very aggressive.

I don't think that there is a way to "ask the question" about whether
someone's work should be destroyed which would not be perceived as
very hostile and aggressive.

I would recommend completely avoiding questions like "should we drop
port X" unless you actually mean that you think the port should be
dropped.  Since that is the way everyone will read it.

Thanks,
Ian.
(who has fond memories of m68k from 1993 and earlier)

[1] https://en.wikipedia.org/wiki/Rhetorical_question

Re: Usage of real m68k hardware

[Geert Uytterhoeven]

Hi Andreas,

On Wed, Mar 28, 2018 at 12:17 PM, Andreas Tille  wrote:
> On Wed, Mar 28, 2018 at 08:11:04PM +1100, Finn Thain wrote:
>> Or is there some shortcoming of the BTS that is driving this?
>
> I think "wontfix" is exactly the feature of the BTS that was invented to
> solve the problem I described.  The bug is not closed and remains listed
> - so everybody is free to ignore that tag and close the bug.  I'm very
> keen in applying patches for bugs very quickly and several contributors
> know that I frequently upload a package in the next 24h after receiving
> a patch.

Doesn't "wontfix" mean that you acknowledge the bug, but decided not to
fix it?

If this is indeed a QEMU problem, IMHO you should not acknowledge the bug.

Gr{oetje,eeting}s,

Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

Re: Usage of real m68k hardware

[Emilio Pozuelo Monfort]

On 28/03/18 12:00, Andreas Tille wrote:
> Hi John,
> 
> On Wed, Mar 28, 2018 at 06:36:16PM +0900, John Paul Adrian Glaubitz wrote:
>> Yes, of course. But Andreas hit a nerve with this on me.
> 
> Sorry, this was not intended.
> 
>>> In my experience, most arguments (not "mere" disagreements) have stemmed
>>> from regrettable miscommunication but all of them have ever helped by an
>>> argumentative or combative character, especially ones underlined with
>>> threats.
>>
>> Well, if Andreas wouldn't have asked for the ultima ratio right from
>> the beginning, there would probably have been a more constructive
>> discussion.
> 
> Just to not make that mistake again:  Is it the
> 
>   "I'm wondering when it might be time to fully drop a hardware
>port instead of draining developer time for ethernity"
> 
> in my mail you stumbled upon.  May be its the language barrier but I do
> not think that if a random developer is wondering about something this
> is equivalent to "asking for the ultima ratio".  I was honestly thinking
> about whether we are keeping alive an architecture by everybody using
> emulators.  I try to be more careful in my wording in the future when
> asking questions about ports (and hope I will manage).

Note that m68k is not a release architecture, so those bugs do not affect the
release status of your packages. Thus, if you don't have the time to work on
such issues, you should feel free to note that and tag the bug 'help' or
similar, and wait for someone to provide a sensible patch.

OTOH, I'm not sure if those FTBFS for non-release architectures, without a patch
or a clue, coming from a non-porter, are that useful...

Cheers,
Emilio

Re: Usage of real m68k hardware

[Andreas Tille]

Hi Finn,

On Wed, Mar 28, 2018 at 08:11:04PM +1100, Finn Thain wrote:
> Is there actual value in increasing the closed bug count with "wontfix"?

For me there is some value.  Last year I've closed more than 300 bugs in
the Debian Med team and in my Debian work more than 1300[1] just in this
team.  I try to work on any bug severity (including wishlist).  When
seeking for bug targets to squash it helps to have means to find out
quickly which ones are the most relevant bugs for users.  Tagging a bug
with wontfix will move that bug to a separate category.  There is no
need to look into a bug report repeatedly if it is in this category and
thus this tag saves my time.
 
> Or is there some shortcoming of the BTS that is driving this?

I think "wontfix" is exactly the feature of the BTS that was invented to
solve the problem I described.  The bug is not closed and remains listed
- so everybody is free to ignore that tag and close the bug.  I'm very
keen in applying patches for bugs very quickly and several contributors
know that I frequently upload a package in the next 24h after receiving
a patch.

Kind regards

   Andreas.

[1] http://blends.debian.net/liststats/bugs_debian-med.png

-- 
http://fam-tille.de

Re: Usage of real m68k hardware

[Andreas Tille]

Hi John,

On Wed, Mar 28, 2018 at 06:36:16PM +0900, John Paul Adrian Glaubitz wrote:
> Yes, of course. But Andreas hit a nerve with this on me.

Sorry, this was not intended.

> > In my experience, most arguments (not "mere" disagreements) have stemmed
> > from regrettable miscommunication but all of them have ever helped by an
> > argumentative or combative character, especially ones underlined with
> > threats.
> 
> Well, if Andreas wouldn't have asked for the ultima ratio right from
> the beginning, there would probably have been a more constructive
> discussion.

Just to not make that mistake again:  Is it the

  "I'm wondering when it might be time to fully drop a hardware
   port instead of draining developer time for ethernity"

in my mail you stumbled upon.  May be its the language barrier but I do
not think that if a random developer is wondering about something this
is equivalent to "asking for the ultima ratio".  I was honestly thinking
about whether we are keeping alive an architecture by everybody using
emulators.  I try to be more careful in my wording in the future when
asking questions about ports (and hope I will manage).

Kind regards and sorry again if I tipped on somebodies shoes

  Andreas.

-- 
http://fam-tille.de

Re: Usage of real m68k hardware

[Ole Streicher]

John Paul Adrian Glaubitz  writes:
> I have worked as a physicist myself for a long time and also did numerical
> physics and have dealt with a lot of software written by scientists. The
> quality of scientific software is usually of exceptional low quality
> because 90% of those scientists are neither programmers nor do they
> care of adhering to any common quality standards when developing
> software.

Being a physcist myself: Honestly, you are distributing FUD here. Sure,
there software like this. But this is rarely the software we have in
Debian, for a number of reasons (speaking for astrophysics, which is my
topic):

 * There is a specialization that people concentrate on doing software,
   still being scientists,

 * software engineers are specifically hired to develop or support
   science software,

 * Communication and critics within the community (wrt. software)
   strongly increased, partly due to Github, which is a great platform
   here. This is accompanied with a general move towards the Bazaar
   development model.

As the highlight example, you may take Astropy, https://www.astropy.org
This is developed by more than hundred people (github says: "241
contributors"), mostly volunteers, backed by (very) few professionals. A
great community, excellent communication into all directions, and a very
high code quality.

Astropy comes with an "affiliated package" concept, which invites people
to contribute their own software, with a review process that keeps the
software quality high there as well.

Ofcourse, there is still low-quality software. But apart from very
specific code, the major problem here is *old* (*OLD*!) software that
was written by software engineers. Software written in the 80s or 90s,
with very questionable hacks to save 10 bytes, only partially
documented, often unclear copyright and licensing, bad upstream
communication etc. But still needed, either since they still are the
only solution, or as a reference. IRAF is the archetype for all this in
my field.

So, it is a wrong prejudice that we have low quality software because of
unexperienced and ignorant scientists.

> Furthermore, from my own experience, most people compile scientific
> software from source anyway due to performance reasons, especially
> when it comes to using them for large calculations.

This is not true, and it was not true in the past:

Since there are now many professional and optimized libraries for a lot
of standard tasks (FFT, algebra etc.), and thanks to the raise of
Python, many scientists (astrophysicists) do not require self-compiled
software anymore. Since we move away from the NIH syndrom, dependencies
play a greater role than before, and are solved by packaging frameworks
(PIP, {Astro}Conda as distribution agnostic examples).

And in the Good Old Times, the installation from source was so
complicated (IRAF!) that people just took the binary tarballs.

The field (in astrophysics) where your assumption is correct is
cosmological simulations ("gadget" as an example here). Unpackaged
exactly because of this.

> So I don't see a point for having to package all of that in Debian.

Some statistics: in astrophysics, we have a "Source Code Library"
https://ascl.net which tries to collect all the source code people wrote
for their (refereed) science. About 6 percent (93 out of ~1600) of them
are currently packaged for Debian.

That is not exactly "all of that", but it is what people need to do
astrophysics (well, almost. White areas still exist).

Best regards

Ole

Re: Usage of real m68k hardware

[John Paul Adrian Glaubitz]

Hi Chris!

On 03/28/2018 06:01 PM, Chris Lamb wrote:
> May I gently and cordially ask for a toning down of the rhethoric
> in this thread? :)

Yes, of course. But Andreas hit a nerve with this on me. This project
has cost me lots of blood, tears and sweat and if someone is asking
for it to be completely thrown out out of nothing, I'm getting a bit
stressed out.

> Whilst everyone would agree that the m68k port has its problems and is
> certainly capable of imposing undue drain on developer time, I'm sure
> most would also understand and perhaps even relate to Andreas'
> frustrations.

Well, we as porters and buildd maintainers have also had lots of frustration
with packages from Debian Science. There were many issues even on release
architectures which arose due to the poor quality of the scientific packages.
Some people from the Debian Science team have previously abused the buildds
as test machines instead of using the various porter boxes the project offers
for such purposes so that I eventually had to send out a mail asking them
to stop doing that.

On the other hand, we often send patches not just for ports architectures
but also release architectures to help fix bugs in packages from the
Debian Science team. So it's not that we are not willing to help and
make their life easier.

> However, that's no real excuse for such an opening salvo to a public
> list and, whilst I do not condone them either, one should not be overly
> surprised to receive defensive and generally unproductive replies to
> such a posting.

Indeed. The point is: Everyone sees a different purpose and usecase in
Debian. And while for Andreas Debian is the basis for doing scientific
work, for me Debian is the basis for hacking on projects like QEMU,
various compilers and the kernel. And many people in- and outside Debian
Ports enjoy doing that as well. It's a valid usecase in my opinion. And,
yes, as crazy as this all sounds, people actually use Debian on m68k,
those machines are particularly popular in Germany, for example.

Debian calls itself the "Universal Operating System" after all and
therefore no single user or developer should get to tell others what
to do with Debian or not.

> In my experience, most arguments (not "mere" disagreements) have stemmed
> from regrettable miscommunication but all of them have ever helped by an
> argumentative or combative character, especially ones underlined with
> threats.

Well, if Andreas wouldn't have asked for the ultima ratio right from
the beginning, there would probably have been a more constructive
discussion. I am always open to discussion and would like to help
the project and other members whereever I can.

In this particular case the solution is actually rather simple: My
suggestion would be that anyone who is about to write bug reports
against issues on the ports architectures just joins #debian-ports
and quickly asks for feedback before filing a bug report.

In most cases, people will get a helpful answer as we have many
competent and motivated folks on the channel who are always open
to help.

So, I'd like to invite people to please just ask on #debian-ports
in the future before filing bug reports which could cause such
misunderstandings.

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Usage of real m68k hardware

[Finn Thain]

On Wed, 28 Mar 2018, Andreas Tille wrote:

> I'm personally drawng the decision that I will tag any bug that concerns 
> scientific software (which is my main focus) and that bug is on m68k 
> only as wontfix and will set severity to minor.  I'm sorry but my focus 
> is on real use cases and stumbling upon bugs in BTS which are not 
> helping any real user is just draining time.
> 

Computer science was productive before HPC came along, so I don't really 
understand why "scientific software" should need special treatment.

However, I agree that there are packages that probably don't get used on 
m68k. (And I imagine that the same can be said for other architectures.)

But isn't it better to let the porters figure out which packages are 
relevant or not?

Is there actual value in increasing the closed bug count with "wontfix"?

Or is there some shortcoming of the BTS that is driving this?

With regard to numerical software, computational science etc.: both small 
and large systems depend upon efficient code, so maintainers need not be 
in conflict AFAICS.

I've written more about my own personal views on the m68k Linux port here 
if you are interested,
http://mac.linux-m68k.org/docs/obsolete.php

-- 

> Kind regards
> 
>   Andreas.
> 
> [1] https://bugs.debian.org/887682
> 
> 

Re: Usage of real m68k hardware

[Chris Lamb]

Dear -devel,

[…]

May I gently and cordially ask for a toning down of the rhethoric
in this thread? :)

Whilst everyone would agree that the m68k port has its problems and is
certainly capable of imposing undue drain on developer time, I'm sure
most would also understand and perhaps even relate to Andreas'
frustrations.

However, that's no real excuse for such an opening salvo to a public
list and, whilst I do not condone them either, one should not be overly
surprised to receive defensive and generally unproductive replies to
such a posting.

In my experience, most arguments (not "mere" disagreements) have stemmed
from regrettable miscommunication but all of them have ever helped by an
argumentative or combative character, especially ones underlined with
threats.

In that light, are we missing anything here? Apart from a chance to
demonstrate how excellent we can be to each other, of course…


Best wishes,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

Re: Usage of real m68k hardware

[Carsten Strotmann]

(excluded debian-...@lists.debian.org, as I've posted this text already
there in mistake. Re-Posted here for completeness)

Hello Andreas,

On 28.03.18 08:38, Andreas Tille wrote:
> Hi,
> 
> recently some R packages received bugs that seem to stem from a problem
> with the build setup (specifically, a qemu bug).  When I asked back in
> one of the bugs[1] whether there are real m68k users I've got the answer
> 
>   ... there are still some users with actual hardware, but the
>   autobuilders use qemu for better performance and/or reliability
> 
> I conclude that the Debian project is running no real m68k hardware any
> more (please correct me if I'm wrong) and we are possibly doing a
> service for some users who potentially also run qemu (wild guess of
> mine).  I'm wondering when it might be time to fully drop a hardware
> port instead of draining developer time for ethernity.
> 
> I'm personally drawng the decision that I will tag any bug that concerns
> scientific software (which is my main focus) and that bug is on m68k
> only as wontfix and will set severity to minor.  I'm sorry but my focus
> is on real use cases and stumbling upon bugs in BTS which are not
> helping any real user is just draining time.
> 

I use m68k Debian on real m68k hardware (but not R or other scientific
software). I know at least four other people that also run/use Debian
m68k on real hardware.

My focus use of m68k is tools and development software (interpreters,
compiler, assembler).

Supporting diverse hardware platforms can expose bugs that are hidden on
mainstream platforms (such as x86 and ARM) and increase the overall code
quality. But I agree that the m68k community is quite small.

Best regards

Carsten Strotmann

Re: Usage of real m68k hardware

[Carsten Strotmann]

Hi,

Andreas Tille  writes:


> Getting fake bugs of severity important due to the fact that no real
> hardware is used since it is to weak is not really convincing for
> maintainers to spent time on it.

the new 68080 CPU core might be powerful enough to build on real
hardware again:



If needbe, I'm willing to invest in and operate build hardware.

But still, hardware that is too weak to build packages does not mean
that the hardware is to weak to *run* the resulting software. A couple
of MIPS based platforms are also too weak to build packages on, but
Debian *runs* there just fine. Some build toolchains are quite bloated
these days.

Greetings

Carsten

Re: Usage of real m68k hardware

[Carsten Strotmann]

(excluded debian-...@lists.debian.org, as I've posted this text already
there in mistake. Re-Posted here for completeness)

Hello Andreas,

On 28.03.18 08:38, Andreas Tille wrote:
> Hi,
> 
> recently some R packages received bugs that seem to stem from a problem
> with the build setup (specifically, a qemu bug).  When I asked back in
> one of the bugs[1] whether there are real m68k users I've got the answer
> 
>   ... there are still some users with actual hardware, but the
>   autobuilders use qemu for better performance and/or reliability
> 
> I conclude that the Debian project is running no real m68k hardware any
> more (please correct me if I'm wrong) and we are possibly doing a
> service for some users who potentially also run qemu (wild guess of
> mine).  I'm wondering when it might be time to fully drop a hardware
> port instead of draining developer time for ethernity.
> 
> I'm personally drawng the decision that I will tag any bug that concerns
> scientific software (which is my main focus) and that bug is on m68k
> only as wontfix and will set severity to minor.  I'm sorry but my focus
> is on real use cases and stumbling upon bugs in BTS which are not
> helping any real user is just draining time.
> 

I use m68k Debian on real m68k hardware (but not R or other scientific
software). I know at least four other people that also run/use Debian
m68k on real hardware.

My focus use of m68k is tools and development software (interpreters,
compiler, assembler).

Supporting diverse hardware platforms can expose bugs that are hidden on
mainstream platforms (such as x86 and ARM) and increase the overall code
quality. But I agree that the m68k community is quite small.

Best regards

Carsten Strotmann







signature.asc
Description: OpenPGP digital signature

Re: Usage of real m68k hardware

[Laurent Vivier]

Le 28/03/2018 à 10:10, John Paul Adrian Glaubitz a écrit :
> On 03/28/2018 04:59 PM, Andreas Tille wrote:
>> I see no point in your repeated "to be honest" and blame others about
>> low quality.  If in doubt read these three bug logs:
>>
>>#882555, #887680, #887682
>>
>> All say
>>
>>this failure turned out to stem from a problem with the build
>>setup (specifically, a qemu bug); sorry for the noise.
> 
> I did not report any of those issues as I am aware of some QEMU bugs
> that can occur from time to time. QEMU bugs still deserve to be fixed.
> 

And QEMU bugs are generally fixed...

And I want to add I have real m68k hardware running (a Quadra 800)

Thanks,
LAurent

Re: Usage of real m68k hardware

[John Paul Adrian Glaubitz]

On 03/28/2018 04:59 PM, Andreas Tille wrote:
> I see no point in your repeated "to be honest" and blame others about
> low quality.  If in doubt read these three bug logs:
> 
>#882555, #887680, #887682
> 
> All say
> 
>this failure turned out to stem from a problem with the build
>setup (specifically, a qemu bug); sorry for the noise.

I did not report any of those issues as I am aware of some QEMU bugs
that can occur from time to time. QEMU bugs still deserve to be fixed.

> Getting fake bugs of severity important due to the fact that no real
> hardware is used since it is to weak is not really convincing for
> maintainers to spent time on it.

How is it a fake bug if the bug was actually in QEMU? Also, again, I
did not report any of those bugs and I wouldn't have reported them
as I usually start looking at QEMU myself first.

> Besides the fact that you went totally off topic with blaming scientific
> software about its quality:  Yes, there is some share of low quality
> software in science as in every other field.  We are working hard to get
> it fixed.  If there are real bugs in the code these will occure not only
> on m68k but also on other less used architectures and we try to sort
> this out with upstream.

I have worked as a physicist myself for a long time and also did numerical
physics and have dealt with a lot of software written by scientists. The
quality of scientific software is usually of exceptional low quality
because 90% of those scientists are neither programmers nor do they
care of adhering to any common quality standards when developing
software.

Furthermore, from my own experience, most people compile scientific
software from source anyway due to performance reasons, especially
when it comes to using them for large calculations. So I don't see
a point for having to package all of that in Debian.

> Your mail was quite convincing to me to simply do what I said (severity
> minor + wontfix) since receiving agressive responses is one more reason
> for me not to spent my time on this.

Honestly, what kind of response did you expect when your first suggestion
to address these issues is to completely destroy the work of others? Do
you actually think this is acceptable? You weren't asking for a compromise
or a constructive discussion, you right away wanted to use the most
radical solution and now you tell me that I am aggressive. Great.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: Usage of real m68k hardware

[Andreas Tille]

On Wed, Mar 28, 2018 at 03:50:52PM +0900, John Paul Adrian Glaubitz wrote:
> 
> To be honest, lots of that scientific code has questionable quality
> and I have seen lots of packages from the Debian Science team with
> hard-coded compiler options and other non-sense. So, to be honest,
> I could send the same statement into your direction: Those science
> packages should be kicked out due to their low quality.

I see no point in your repeated "to be honest" and blame others about
low quality.  If in doubt read these three bug logs:

   #882555, #887680, #887682

All say

   this failure turned out to stem from a problem with the build
   setup (specifically, a qemu bug); sorry for the noise.

Getting fake bugs of severity important due to the fact that no real
hardware is used since it is to weak is not really convincing for
maintainers to spent time on it.

Besides the fact that you went totally off topic with blaming scientific
software about its quality:  Yes, there is some share of low quality
software in science as in every other field.  We are working hard to get
it fixed.  If there are real bugs in the code these will occure not only
on m68k but also on other less used architectures and we try to sort
this out with upstream.

Your mail was quite convincing to me to simply do what I said (severity
minor + wontfix) since receiving agressive responses is one more reason
for me not to spent my time on this.

Kind regards

Andreas.

-- 
http://fam-tille.de

Re: Usage of real m68k hardware

[Ingo Jürgensmann]

On 28.03.2018 08:38, Andreas Tille wrote:


I conclude that the Debian project is running no real m68k hardware any
more (please correct me if I'm wrong) and we are possibly doing a
service for some users who potentially also run qemu (wild guess of
mine).  I'm wondering when it might be time to fully drop a hardware
port instead of draining developer time for ethernity.


Well, officially Debian has no real m68k hardware anymore, because the 
project decided to official drop m68k as a release arch and DSA no 
longer admins kullervo. Correct me if I'm wrong.


Alas, there is still m68k hardware around. Kullervo should be under 
control of Christian Steigies and then there should be some more m68k 
machines under control of Adrian.


My 3 Amigas are currently turned off or currently lack the support of my 
new X-Surf 100 NIC in the netinstall image provided by Adrian.


In September there is a m68k meeting planned at Linux Hotel in Essen. 
So, the m68k port is still alive and kicking! :-)


--
Ciao...  //http://blog.windfluechter.net
  Ingo \X/ XMPP: i...@jabber.windfluechter.net


gpg pubkey:  http://www.juergensmann.de/ij_public_key.asc

Re: Usage of real m68k hardware

[John Paul Adrian Glaubitz]

On 03/28/2018 03:38 PM, Andreas Tille wrote:
> I conclude that the Debian project is running no real m68k hardware any
> more (please correct me if I'm wrong) and we are possibly doing a
> service for some users who potentially also run qemu (wild guess of
> mine).  I'm wondering when it might be time to fully drop a hardware
> port instead of draining developer time for ethernity.

If that happens, I leave the Debian project for good. The answer is no!

> I'm personally drawng the decision that I will tag any bug that concerns
> scientific software (which is my main focus) and that bug is on m68k
> only as wontfix and will set severity to minor.  I'm sorry but my focus
> is on real use cases and stumbling upon bugs in BTS which are not
> helping any real user is just draining time.

To be honest, lots of that scientific code has questionable quality
and I have seen lots of packages from the Debian Science team with
hard-coded compiler options and other non-sense. So, to be honest,
I could send the same statement into your direction: Those science
packages should be kicked out due to their low quality.

As for the m68k port: There is a large user base around the Amiga
and Atari groups and it would therefore be not acceptable to remove
something that people use.

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Usage of real m68k hardware

[Andreas Tille]

Hi,

recently some R packages received bugs that seem to stem from a problem
with the build setup (specifically, a qemu bug).  When I asked back in
one of the bugs[1] whether there are real m68k users I've got the answer

  ... there are still some users with actual hardware, but the
  autobuilders use qemu for better performance and/or reliability

I conclude that the Debian project is running no real m68k hardware any
more (please correct me if I'm wrong) and we are possibly doing a
service for some users who potentially also run qemu (wild guess of
mine).  I'm wondering when it might be time to fully drop a hardware
port instead of draining developer time for ethernity.

I'm personally drawng the decision that I will tag any bug that concerns
scientific software (which is my main focus) and that bug is on m68k
only as wontfix and will set severity to minor.  I'm sorry but my focus
is on real use cases and stumbling upon bugs in BTS which are not
helping any real user is just draining time.

Kind regards

  Andreas.

[1] https://bugs.debian.org/887682

-- 
http://fam-tille.de