Re: pkg chroot issues?

[Julian Elischer]

On 30/05/2016 12:33 AM, Tim Kientzle wrote:

On May 29, 2016, at 8:55 AM, Julian Elischer  wrote:

I was thinking that in order to do this properly the chrooted child should do 
all it's fetch requests etc via the non-chrooted parent, but that would have 
probably been a bit too complicated. (including add requests)

How complex would it be to perform all of the fetch requests before chroot?
I don't know but you'd have to read the DB in the chroot before 
knowing what dependencies you need to fix.




Tim





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: pkg chroot issues?

[Tim Kientzle]

> On May 29, 2016, at 8:55 AM, Julian Elischer  wrote:
> 
> I was thinking that in order to do this properly the chrooted child should do 
> all it's fetch requests etc via the non-chrooted parent, but that would have 
> probably been a bit too complicated. (including add requests)

How complex would it be to perform all of the fetch requests before chroot?

Tim

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: pkg chroot issues?

[Julian Elischer]

On 23/05/2016 4:32 AM, b...@freebsd.org wrote:

On Sun, May 22, 2016 at 10:31:08PM +0200, b...@freebsd.org wrote:

On Sun, May 22, 2016 at 01:24:12PM -0700, Tim Kientzle wrote:

Crochet has some experimental hooks to install packages onto the system being 
built, but this seems to be hitting problems due to limitations in 'pkg -c'.  
In particular, it seems that pkg performs the chroot before it does any network 
lookups.  This is a problem if the chroot is not a complete system environment 
(which it cannot be when you're building an image for another system).

There's some further discussion on github:

   https://github.com/freebsd/crochet/issues/141

Any suggestions?


I'll reply directly to github thanks for pointing me to the ticket

Best regards,
Bapt

As people might only follow this thread and not the ticket here is what I
answered:

pkg supports an option for that which is pkg -o NAMESERVER= to avoid having to
copy resolv.conf it was broken in 1.8 but I fixed it in pkg 1.8.0 which has been
released today.

the problem with pkg -c is that it calls chroot very early. To avoid that
problem we have added pkg -r which does not perform any chroot at all therefore
having not network issue, but the ports tree are is not yet entirely aware of it
and some scripts (preinstall/postinstall) might cause some issues.
at least creating users from the script is safe in that regard. for most simple
case that should work.


As Bapt knows (I've been harassing him) pkg needs an "install all this 
over there" mode.
The answer is "-r" but it does suffer from the fact that it requires 
postinstall scripts to cooperate
by knowing to look for the appropriate environment value (I forget its 
name).


I've used three different methods to get around this..
1/ use -c and prepopulate it with a copy of /rescue, and copying all 
the pkg files I need in first into /pkg, which I delete afterwards.
I'd like clarification in the Man page about setting a good $PATH in 
the chrooted part of pkg currently I just copy /rescue into /bin but 
pkg seems to sometimes use full paths for things so that meant also 
/usr/bin and usr/sbin. (or maybe that was an install script.).
(in my application I could just link all 4 locations to be the same 
place).


2/ use   PKG_DBDIR=$(TARGET_DIR)/var/db/pkg pkg add --relocate 
$(TARGET_DIR) $(PKGFILE)

Almost the same as -r from my perspective.

3/ because I need to work as non-root, and pkg doesn't have a -u and 
-g option like chroot does, there are times when I need to do:


sudo chroot -u $ME -g $MYGROUP sh -c "INSTALL_AS_USER pkg add -f -M 
$(PKGFILE)"


instead of using pkg -c.

I worked up a patch for -u and -g but lost it.. it was only about 20 
lines including the usage() change.  I'm sure bapt could redo it much 
better.

using -u automatically enabled install_as_user.

I was thinking that in order to do this properly the chrooted child 
should do all it's fetch requests etc via the non-chrooted parent, but 
that would have probably been a bit too complicated. (including add 
requests)







Best regards,
Bapt



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: pkg chroot issues?

[Baptiste Daroussin]

On Sun, May 22, 2016 at 02:18:07PM -0700, Matthew Macy wrote:
> 
> 
> 
>   On Sun, 22 May 2016 13:43:13 -0700 Tim Kientzle  
> wrote  
>  >  
>  > > On May 22, 2016, at 1:28 PM, K. Macy  wrote: 
>  > >  
>  > >  
>  > >  
>  > > On Sunday, May 22, 2016, Tim Kientzle  wrote: 
>  > > Crochet has some experimental hooks to install packages onto the system 
> being built, but this seems to be hitting problems due to limitations in 'pkg 
> -c'.  In particular, it seems that pkg performs the chroot before it does any 
> network lookups.  This is a problem if the chroot is not a complete system 
> environment (which it cannot be when you're building an image for another 
> system). 
>  > >  
>  > > There's some further discussion on github: 
>  > >  
>  > >   https://github.com/freebsd/crochet/issues/141 
>  > >  
>  > > Any suggestions? 
>  > >  
>  > > Cheers, 
>  > >  
>  > > Tim 
>  > >  
>  > >  
>  > > Just like you need to mount devfs you should have a resolv.conf in your 
> chroot first. Just copy it over before running pkg. This works for me in my 
> image creation script. 
>  >  
>  > Sometimes the image does already have a resolv.conf, but if it does, it's 
> for the target environment (where the image will ultimately be running) and 
> may not be appropriate for the environment where the image is being built. 
>  
> Setting NAMESERVER to "10.0.1.1" crashed pkg for me. Maybe it's been fixed in 
> the meantime. If a resolv.conf already exists I would just rename it before 
> and then rename it back after the call to pkg -c.
> 
Setting nameserver will inject this name server directly via the libc resolv api
so it won't touch the resolv.conf at all no need to back it up

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: pkg chroot issues?

[Matthew Macy]

  On Sun, 22 May 2016 13:43:13 -0700 Tim Kientzle  wrote 
 
 >  
 > > On May 22, 2016, at 1:28 PM, K. Macy  wrote: 
 > >  
 > >  
 > >  
 > > On Sunday, May 22, 2016, Tim Kientzle  wrote: 
 > > Crochet has some experimental hooks to install packages onto the system 
 > > being built, but this seems to be hitting problems due to limitations in 
 > > 'pkg -c'.  In particular, it seems that pkg performs the chroot before it 
 > > does any network lookups.  This is a problem if the chroot is not a 
 > > complete system environment (which it cannot be when you're building an 
 > > image for another system). 
 > >  
 > > There's some further discussion on github: 
 > >  
 > >   https://github.com/freebsd/crochet/issues/141 
 > >  
 > > Any suggestions? 
 > >  
 > > Cheers, 
 > >  
 > > Tim 
 > >  
 > >  
 > > Just like you need to mount devfs you should have a resolv.conf in your 
 > > chroot first. Just copy it over before running pkg. This works for me in 
 > > my image creation script. 
 >  
 > Sometimes the image does already have a resolv.conf, but if it does, it's 
 > for the target environment (where the image will ultimately be running) and 
 > may not be appropriate for the environment where the image is being built. 
 
Setting NAMESERVER to "10.0.1.1" crashed pkg for me. Maybe it's been fixed in 
the meantime. If a resolv.conf already exists I would just rename it before and 
then rename it back after the call to pkg -c.

Cheers.

-M

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: pkg chroot issues?

[Tim Kientzle]

> On May 22, 2016, at 1:28 PM, K. Macy  wrote:
> 
> 
> 
> On Sunday, May 22, 2016, Tim Kientzle  wrote:
> Crochet has some experimental hooks to install packages onto the system being 
> built, but this seems to be hitting problems due to limitations in 'pkg -c'.  
> In particular, it seems that pkg performs the chroot before it does any 
> network lookups.  This is a problem if the chroot is not a complete system 
> environment (which it cannot be when you're building an image for another 
> system).
> 
> There's some further discussion on github:
> 
>   https://github.com/freebsd/crochet/issues/141
> 
> Any suggestions?
> 
> Cheers,
> 
> Tim
> 
> 
> Just like you need to mount devfs you should have a resolv.conf in your 
> chroot first. Just copy it over before running pkg. This works for me in my 
> image creation script.

Sometimes the image does already have a resolv.conf, but if it does, it's for 
the target environment (where the image will ultimately be running) and may not 
be appropriate for the environment where the image is being built.

Tim


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: pkg chroot issues?

[b...@freebsd.org]

On Sun, May 22, 2016 at 10:31:08PM +0200, b...@freebsd.org wrote:
> On Sun, May 22, 2016 at 01:24:12PM -0700, Tim Kientzle wrote:
> > Crochet has some experimental hooks to install packages onto the system 
> > being built, but this seems to be hitting problems due to limitations in 
> > 'pkg -c'.  In particular, it seems that pkg performs the chroot before it 
> > does any network lookups.  This is a problem if the chroot is not a 
> > complete system environment (which it cannot be when you're building an 
> > image for another system).
> > 
> > There's some further discussion on github:
> > 
> >   https://github.com/freebsd/crochet/issues/141
> > 
> > Any suggestions?
> > 
> I'll reply directly to github thanks for pointing me to the ticket
> 
> Best regards,
> Bapt

As people might only follow this thread and not the ticket here is what I
answered:

pkg supports an option for that which is pkg -o NAMESERVER= to avoid having to
copy resolv.conf it was broken in 1.8 but I fixed it in pkg 1.8.0 which has been
released today.

the problem with pkg -c is that it calls chroot very early. To avoid that
problem we have added pkg -r which does not perform any chroot at all therefore
having not network issue, but the ports tree are is not yet entirely aware of it
and some scripts (preinstall/postinstall) might cause some issues.
at least creating users from the script is safe in that regard. for most simple
case that should work.


Best regards,
Bapt


signature.asc
Description: PGP signature

Re: pkg chroot issues?

[b...@freebsd.org]

On Sun, May 22, 2016 at 01:24:12PM -0700, Tim Kientzle wrote:
> Crochet has some experimental hooks to install packages onto the system being 
> built, but this seems to be hitting problems due to limitations in 'pkg -c'.  
> In particular, it seems that pkg performs the chroot before it does any 
> network lookups.  This is a problem if the chroot is not a complete system 
> environment (which it cannot be when you're building an image for another 
> system).
> 
> There's some further discussion on github:
> 
>   https://github.com/freebsd/crochet/issues/141
> 
> Any suggestions?
> 
I'll reply directly to github thanks for pointing me to the ticket

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: pkg chroot issues?

[K. Macy]

On Sunday, May 22, 2016, Tim Kientzle  wrote:

> Crochet has some experimental hooks to install packages onto the system
> being built, but this seems to be hitting problems due to limitations in
> 'pkg -c'.  In particular, it seems that pkg performs the chroot before it
> does any network lookups.  This is a problem if the chroot is not a
> complete system environment (which it cannot be when you're building an
> image for another system).
>
> There's some further discussion on github:
>
>   https://github.com/freebsd/crochet/issues/141
>
> Any suggestions?
>
> Cheers,
>
> Tim
>
>
Just like you need to mount devfs you should have a resolv.conf in your
chroot first. Just copy it over before running pkg. This works for me in my
image creation script.


-M


> ___
> freebsd-current@freebsd.org  mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org
> "
>
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

pkg chroot issues?

[Tim Kientzle]

Crochet has some experimental hooks to install packages onto the system being 
built, but this seems to be hitting problems due to limitations in 'pkg -c'.  
In particular, it seems that pkg performs the chroot before it does any network 
lookups.  This is a problem if the chroot is not a complete system environment 
(which it cannot be when you're building an image for another system).

There's some further discussion on github:

  https://github.com/freebsd/crochet/issues/141

Any suggestions?

Cheers,

Tim

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"