Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Cyril Brulebois]

Colin Watson  (2016-11-26):
> Yep, as Thomas said, this changed back to the original state in
> 2.02~beta3.  I think that in fact allows us to close #741656 if you also
> revert your previous workaround in d-i, but please confirm.  Compare:
> 
>   
> https://anonscm.debian.org/cgit/pkg-grub/grub.git/commit/?id=496cca85259c77b5fc8ad39a9064c83c3a39b7a6
> 
> (Sorry, I forgot that d-i had worked around this.)

(No worries.)

Yeah, a revert was what I had in mind, but I thought I'd double check
anyway. Just pushed it, triggered a build on falla and fischer, and
builds seem happier now. Thanks!

I think this bug report should be closed now?


KiBi.


signature.asc
Description: Digital signature

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Colin Watson]

On Sat, Nov 26, 2016 at 06:17:28AM +0100, Cyril Brulebois wrote:
> kfreebsd-* builds are now failing to build with:
> | # Create the ISO with Joliet extensions, needed for win32-loader.ini
> | # NOTE: "-- -J" is a workaround for #741656 in grub-common
> | grub-mkrescue --output=./tmp/netboot-10/mini.iso ./tmp/netboot-10/cd_tree 
> -- -J
> | xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
> | 
> | xorriso : FAILURE : Not a known command:  '-J'
> | 
> | xorriso : aborting : -abort_on 'FAILURE' encountered 'FAILURE'
> | install -m 644 -D ./tmp/netboot-10/mini.iso dest/netboot-10/mini.iso
> | install: cannot stat './tmp/netboot-10/mini.iso': No such file or directory
> | Makefile:798: recipe for target 'dest/netboot-10/mini.iso' failed
> 
> I think that started with grub2 (2.02~beta3-1) reaching unstable:
>   https://tracker.debian.org/news/805858
> 
> as upload timing would match the start of the FTBFS according to graphs
> on the summary page:
>   https://d-i.debian.org/daily-images/daily-build-overview.html
> 
> Also, util/grub-mkrescue.c's argument handling got changed between grub2
> 2.02~beta2-36 and 2.02~beta3-1.

Yep, as Thomas said, this changed back to the original state in
2.02~beta3.  I think that in fact allows us to close #741656 if you also
revert your previous workaround in d-i, but please confirm.  Compare:

  
https://anonscm.debian.org/cgit/pkg-grub/grub.git/commit/?id=496cca85259c77b5fc8ad39a9064c83c3a39b7a6

(Sorry, I forgot that d-i had worked around this.)

-- 
Colin Watson   [cjwat...@debian.org]

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Thomas Schmitt]

Hi,

the decision to stay with the old shell script's command interface
was made 2 years ago:

  http://lists.gnu.org/archive/html/grub-devel/2014-11/msg00076.html
  Vladimir 'φ-coder/phcoder' Serbinenko wrote:
  "I think that old parser is better. The only reason the change happened
   is that it's a bug that sneaked in during migration to C. It should be
   fixed."

I.e. the correct way to add -J is without preceeding "--".

Afaik, this decision was implemented in march 2015:

  
http://git.savannah.gnu.org/cgit/grub.git/commit/?id=cf47a2fba5852014bc59959c5e357e8313933414
  "grub-mkrescue: pass all unrecognized options unchanged to xorriso."


Have a nice day :)

Thomas

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Cyril Brulebois]

Cyril Brulebois  (2014-03-16):
> Control: severity -1 important
> 
> Colin Watson  (2014-03-15):
> > On Sat, Mar 15, 2014 at 12:37:17PM +0100, Cyril Brulebois wrote:
> > > I'm tempted to commit the '--' addition in debian-installer for now
> > > anyway, including a comment pointing here, and to lower the severity
> > > to important (since other callers might be affected as well).
> > > 
> > > Does that look OK to you?
> > 
> > It's not ideal, but I think it's reasonable for now, yes.
> 
> Great, thanks for the confirmation, and also for the initial suggestion;
> pushed on the d-i side as:
>   
> http://anonscm.debian.org/gitweb/?p=d-i/debian-installer.git;a=commitdiff;h=d85d105

kfreebsd-* builds are now failing to build with:
| # Create the ISO with Joliet extensions, needed for win32-loader.ini
| # NOTE: "-- -J" is a workaround for #741656 in grub-common
| grub-mkrescue --output=./tmp/netboot-10/mini.iso ./tmp/netboot-10/cd_tree -- 
-J
| xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
| 
| xorriso : FAILURE : Not a known command:  '-J'
| 
| xorriso : aborting : -abort_on 'FAILURE' encountered 'FAILURE'
| install -m 644 -D ./tmp/netboot-10/mini.iso dest/netboot-10/mini.iso
| install: cannot stat './tmp/netboot-10/mini.iso': No such file or directory
| Makefile:798: recipe for target 'dest/netboot-10/mini.iso' failed

I think that started with grub2 (2.02~beta3-1) reaching unstable:
  https://tracker.debian.org/news/805858

as upload timing would match the start of the FTBFS according to graphs
on the summary page:
  https://d-i.debian.org/daily-images/daily-build-overview.html

Also, util/grub-mkrescue.c's argument handling got changed between grub2
2.02~beta2-36 and 2.02~beta3-1.


KiBi.


signature.asc
Description: Digital signature

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Thomas Schmitt]

Hi,

Steven Chamberlain:
 I'm unfamiliar with xorriso but the man page mentions -joliet on

The problem is that argument -- marks the end of the parameter
list of xorriso command -as mkisofs.
Both versions of grub-mkrescue start xorriso with that command and
use mkisofs-ish options to define the job.

The old (bash) one does not interpret -- by itself, but appends
it to the xorriso arguments. So xorriso leaves mkisofs emulation
and does not understand mkisofs option -J any more.

The new (C language) grub-mkrescue takes -- as reason to append
all following arguments to the xorriso argument list. It consumes
this first --. So xorriso stays in mkisofs emulation and
understands -J.


Cyril Brulebois:
 | grub-mkrescue: invalid option -- 'J'

This is the new grub-mkresue. It complains because there is no --
which indicates the start of options to be forwarded to xorriso.

 | xorriso : FAILURE : Not a known command:  '-J'

This is when the old one fowarded both -- and -J.


Colin Watson:
 (We've talked about changing the
 behaviour back upstream, but the thread petered out a bit and I don't
 believe it's done yet.)

We seem to wait for Vladimir Serbinenko to decide whether he wants
to unify the CLIs of some tools and thus really break the CLI of
grub-mkrescue.

If the decision is made in favor of CLI change, then the new
grub-mkrescue should get means to tell its boss scripts that the
new behavior is active.
Again, it would be up to Vladimir to officially define that indicator.


Currently i see no fully satisfying way to consistently use both
grub-mkrescue versions by the same shell command line.

At most one could ensure that xorriso interprets all its user
provided arguments as native commands. This would be achieved by
prepending -- -- to the user arguments:

  grub-mkrescue ... -- -- -joliet on ...

Old grub-mkrescue would execute:

  xorriso -as mkisofs ...args.from.mkrescue... -- -- -joliet on ...

The orphan -- will be silently ignored by xorriso.

New grub-mkrescue would execute:

  xorriso -as mkisofs ...args.from.mkrescue... -- -joliet on ...

xorriso commands are documented in man xorriso. If you want to
stay with man xorrisofs, you may run a second -as mkisofs
command to enable interpretation of mkisofs-ish options:

  grub-mkrescue ... -- -- -as mkisofs -J ...

Nevertheless, the end of the first -as mkisofs triggers some
processing of its mkisofs options. This can create interesting
stumblestones.
Enabling Joliet production is not such a stumblestone, though.
Adding a few files should be ok too. (In native mode, use -add.)


Have a nice day :)

Thomas


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Colin Watson]

On Sat, Mar 15, 2014 at 02:22:05AM +0100, Cyril Brulebois wrote:
 Cyril Brulebois k...@debian.org (2014-03-15):
  so there are some grub-* packages used in debian-installer's build-deps,
  and grub-common is among them, since grub-mkrescue is used on kfreebsd-*.
  It apparently lost its -J flag, making debian-installer FTBFS on those
  architectures. Excerpt from a daily build log after 2014-03-10:
  | # Create the ISO with Joliet extensions, needed for win32-loader.ini
  | grub-mkrescue --output=./tmp/netboot-9/mini.iso ./tmp/netboot-9/cd_tree -J
  | grub-mkrescue: invalid option -- 'J'
  | Try 'grub-mkrescue --help' or 'grub-mkrescue --usage' for more 
  information.
  | make[2]: *** [arch_miniiso] Error 64
 
 That this option gets rejected isn't entirely surprising given we
 previously had a pass-through for unknown options:

Right.  Can you try -- -J instead?  (We've talked about changing the
behaviour back upstream, but the thread petered out a bit and I don't
believe it's done yet.)

 [BTW the tag for -7 isn't to be found in the git repository right now.]

Sure is, it's just called debian/2.02.beta2-7 because git.

-- 
Colin Watson   [cjwat...@debian.org]


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Cyril Brulebois]

Colin Watson cjwat...@debian.org (2014-03-15):
 Right.  Can you try -- -J instead?  (We've talked about changing the
 behaviour back upstream, but the thread petered out a bit and I don't
 believe it's done yet.)

That seems to do the trick (and I've checked through a tiny wrapper that
the -J flag is indeed passed to xorriso). I haven't tried to run any
resulting ISO though. But at least dpkg-buildpackage in d-i manages to
build all images in my kfreebsd-amd64 VM.

After having downgraded the installed grub-* packages, I've checked that
using this syntax wouldn't work with past versions:
| grub-mkrescue --output=./tmp/netboot-9/mini.iso ./tmp/netboot-9/cd_tree -- -J
| Enabling BIOS support ...
| xorriso 1.3.2 : RockRidge filesystem manipulator, libburnia project.
| 
| xorriso : FAILURE : Not a known command:  '-J'

I'm tempted to commit the '--' addition in debian-installer for now
anyway, including a comment pointing here, and to lower the severity to
important (since other callers might be affected as well).

Does that look OK to you?


  [BTW the tag for -7 isn't to be found in the git repository right now.]
 
 Sure is, it's just called debian/2.02.beta2-7 because git.

Got distracted by sleepiness, -b experimental, and git describe
returning debian/2.02.beta2-6-32-gbd08b3c; sorry about that.

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Colin Watson]

On Sat, Mar 15, 2014 at 12:37:17PM +0100, Cyril Brulebois wrote:
 After having downgraded the installed grub-* packages, I've checked that
 using this syntax wouldn't work with past versions:
 | grub-mkrescue --output=./tmp/netboot-9/mini.iso ./tmp/netboot-9/cd_tree -- 
 -J
 | Enabling BIOS support ...
 | xorriso 1.3.2 : RockRidge filesystem manipulator, libburnia project.
 | 
 | xorriso : FAILURE : Not a known command:  '-J'
 
 I'm tempted to commit the '--' addition in debian-installer for now
 anyway, including a comment pointing here, and to lower the severity to
 important (since other callers might be affected as well).
 
 Does that look OK to you?

It's not ideal, but I think it's reasonable for now, yes.

-- 
Colin Watson   [cjwat...@debian.org]


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Cyril Brulebois]

Control: severity -1 important

Colin Watson cjwat...@debian.org (2014-03-15):
 On Sat, Mar 15, 2014 at 12:37:17PM +0100, Cyril Brulebois wrote:
  I'm tempted to commit the '--' addition in debian-installer for now
  anyway, including a comment pointing here, and to lower the severity
  to important (since other callers might be affected as well).
  
  Does that look OK to you?
 
 It's not ideal, but I think it's reasonable for now, yes.

Great, thanks for the confirmation, and also for the initial suggestion;
pushed on the d-i side as:
  
http://anonscm.debian.org/gitweb/?p=d-i/debian-installer.git;a=commitdiff;h=d85d105

Mraw,
KiBi.


signature.asc
Description: Digital signature

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Steven Chamberlain]

On 15/03/14 11:37, Cyril Brulebois wrote:
 That seems to do the trick (and I've checked through a tiny wrapper that
 the -J flag is indeed passed to xorriso).

It seems to be passed through, but xorriso doesn't recognise it.  Is
this unexpected?

 grub-mkrescue --output=./tmp/netboot-9/mini.iso ./tmp/netboot-9/cd_tree -- -J
 Enabling BIOS support ...
 xorriso 1.2.2 : RockRidge filesystem manipulator, libburnia project.
 
 xorriso : FAILURE : Not a known option:  '-J'
 
 make[7]: *** [arch_miniiso] Error 5

I'm unfamiliar with xorriso but the man page mentions -joliet on

Regards,
-- 
Steven Chamberlain
ste...@pyro.eu.org



signature.asc
Description: OpenPGP digital signature

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Cyril Brulebois]

Package: grub-common
Version: 2.02~beta2-7
Severity: serious
File: /usr/bin/grub-mkrescue
Tags: d-i
Justification: triggers FTBFS

Hi,

so there are some grub-* packages used in debian-installer's build-deps,
and grub-common is among them, since grub-mkrescue is used on kfreebsd-*.
It apparently lost its -J flag, making debian-installer FTBFS on those
architectures. Excerpt from a daily build log after 2014-03-10:
| # Create the ISO with Joliet extensions, needed for win32-loader.ini
| grub-mkrescue --output=./tmp/netboot-9/mini.iso ./tmp/netboot-9/cd_tree -J
| grub-mkrescue: invalid option -- 'J'
| Try 'grub-mkrescue --help' or 'grub-mkrescue --usage' for more information.
| make[2]: *** [arch_miniiso] Error 64

Not too nice when I'm about to upload debian-installer for jessie alpha 1. :(

Mraw,
KiBi.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#741656: grub-common: grub-mkrescue lost its -J flag, d-i now FTBFS on kfreebsd-*

[Cyril Brulebois]

[ cc-ing debian-boot@ for information. ]

Cyril Brulebois k...@debian.org (2014-03-15):
 Package: grub-common
 Version: 2.02~beta2-7
 Severity: serious
 File: /usr/bin/grub-mkrescue
 Tags: d-i
 Justification: triggers FTBFS
 
 Hi,
 
 so there are some grub-* packages used in debian-installer's build-deps,
 and grub-common is among them, since grub-mkrescue is used on kfreebsd-*.
 It apparently lost its -J flag, making debian-installer FTBFS on those
 architectures. Excerpt from a daily build log after 2014-03-10:
 | # Create the ISO with Joliet extensions, needed for win32-loader.ini
 | grub-mkrescue --output=./tmp/netboot-9/mini.iso ./tmp/netboot-9/cd_tree -J
 | grub-mkrescue: invalid option -- 'J'
 | Try 'grub-mkrescue --help' or 'grub-mkrescue --usage' for more information.
 | make[2]: *** [arch_miniiso] Error 64

That this option gets rejected isn't entirely surprising given we
previously had a pass-through for unknown options:

grub2-2.00/util/grub-mkrescue.in (shell script):
| # Check the arguments.
| while test $# -gt 0
| do
| option=$1
| shift
| 
| case $option in
…
| *)
| source=${source} ${option} $@; break ;;
| esac
| done

while lastest upload has a version rewritten in C and rejecting unknown
options instead of passing them through:

grub2-2.02~beta2/util/grub-mkrescue.c:
| static struct argp_option options[] = {
…
|   {0, 0, 0, 0, 0, 0}
| };
| 
| argp_parser (int key, char *arg, struct argp_state *state)
| {
|   if (grub_install_parse (key, arg))
| return 0;
|   switch (key)
| {
…
| default:
|   return ARGP_ERR_UNKNOWN;
| }
| }

[BTW the tag for -7 isn't to be found in the git repository right now.]

Mraw,
KiBi.


signature.asc
Description: Digital signature