Re: CFT: FreeBSD Package Base

2019-05-02 Thread Zane C. B-H.

On 2019-04-30 17:03, Miroslav Lachman wrote:

David Chisnall wrote on 2019/04/30 10:22:

On 29/04/2019 21:12, Joe Maloney wrote:
With CFT version you chose to build, and package individual 
components such as sendmail with a port option.  That does entirely 
solve the problem of being able to reinstall sendmail after the fact 
without a rebuild of the userland (base) port but perhaps base 
flavors could solve that problem assuming flavors could extend beyond 
python.


This sounds very much like local optimisation. It's now easy to create 
a custom base image.  Great.  But how do I express dependencies in 
ports on a specific base configuration? This is easy if I depend on a 
specific base package, but how does this work in your model?  For 
example, if I have a package that depends on a library that is an 
optional part of the base system, how do I express that pkg needs to 
either refuse to install it, or install a userland pkg that includes 
that library in place of my existing version as part of the install 
process?


More importantly for the container use case, if I want to take a 
completely empty jail and do pkg ins nginx (for example), what does 
the maintainer of the nginx port need to do to express the minimum set 
of the base system that needs to be installed to allow nginx to work?


One of the goals for the pkg base concept was to allow this kind of 
use case, easily creating a minimal environment required to run a 
single service. With a monolithic base package set, you're going to 
need some mechanism other than packages to express the specific base 
subset package that you need and I think that you need to justify why 
this mechanism is better than using small individual packages.


Will it not be maintainer's nightmare to take care of all the
dependencies on the base packages for each port we have in the ports
tree?


Speaking as a ports maintainer, it will be very annoying. Splitting it 
into a handful of large ass packages, same as you are presented with 
during install, would be best.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-05-01 Thread Miroslav Lachman

Cy Schubert wrote on 2019/05/01 05:56:

In message <292eadc6-3662-ec43-1175-53fc25248...@quip.cz>, Miroslav
Lachman wri
tes:

David Chisnall wrote on 2019/04/30 10:22:

On 29/04/2019 21:12, Joe Maloney wrote:

With CFT version you chose to build, and package individual components
such as sendmail with a port option.  That does entirely solve the
problem of being able to reinstall sendmail after the fact without a
rebuild of the userland (base) port but perhaps base flavors could
solve that problem assuming flavors could extend beyond python.


This sounds very much like local optimisation. It's now easy to create a
custom base image.  Great.  But how do I express dependencies in ports
on a specific base configuration? This is easy if I depend on a specific
base package, but how does this work in your model?  For example, if I
have a package that depends on a library that is an optional part of the
base system, how do I express that pkg needs to either refuse to install
it, or install a userland pkg that includes that library in place of my
existing version as part of the install process?

More importantly for the container use case, if I want to take a
completely empty jail and do pkg ins nginx (for example), what does the
maintainer of the nginx port need to do to express the minimum set of
the base system that needs to be installed to allow nginx to work?

One of the goals for the pkg base concept was to allow this kind of use
case, easily creating a minimal environment required to run a single
service. With a monolithic base package set, you're going to need some
mechanism other than packages to express the specific base subset
package that you need and I think that you need to justify why this
mechanism is better than using small individual packages.


Will it not be maintainer's nightmare to take care of all the
dependencies on the base packages for each port we have in the ports tree?


No more than it is today. Remember, people have been doing this sort of
thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and
IBM can do it, I'm sure we can too. The dependency lists will be
longer. We may require dependency lists that allow the choice of one of
many prereqs or coreqs.


They are experts and they are paid for their work. I am not. I am 
maintaining a few packages and the reality is I don't know what they 
need in base. Till these days I don't care about this kind of 
dependency. I am not system developer or programmer and I think there 
are more than just me who see this as a kind of problem.

So in this case, pkg base gives me nothing but more work on those packages.

Miroslav Lachman
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-30 Thread Cy Schubert
In message <292eadc6-3662-ec43-1175-53fc25248...@quip.cz>, Miroslav 
Lachman wri
tes:
> David Chisnall wrote on 2019/04/30 10:22:
> > On 29/04/2019 21:12, Joe Maloney wrote:
> >> With CFT version you chose to build, and package individual components 
> >> such as sendmail with a port option.  That does entirely solve the 
> >> problem of being able to reinstall sendmail after the fact without a 
> >> rebuild of the userland (base) port but perhaps base flavors could 
> >> solve that problem assuming flavors could extend beyond python.
> > 
> > This sounds very much like local optimisation. It's now easy to create a 
> > custom base image.  Great.  But how do I express dependencies in ports 
> > on a specific base configuration? This is easy if I depend on a specific 
> > base package, but how does this work in your model?  For example, if I 
> > have a package that depends on a library that is an optional part of the 
> > base system, how do I express that pkg needs to either refuse to install 
> > it, or install a userland pkg that includes that library in place of my 
> > existing version as part of the install process?
> > 
> > More importantly for the container use case, if I want to take a 
> > completely empty jail and do pkg ins nginx (for example), what does the 
> > maintainer of the nginx port need to do to express the minimum set of 
> > the base system that needs to be installed to allow nginx to work?
> > 
> > One of the goals for the pkg base concept was to allow this kind of use 
> > case, easily creating a minimal environment required to run a single 
> > service. With a monolithic base package set, you're going to need some 
> > mechanism other than packages to express the specific base subset 
> > package that you need and I think that you need to justify why this 
> > mechanism is better than using small individual packages.
>
> Will it not be maintainer's nightmare to take care of all the 
> dependencies on the base packages for each port we have in the ports tree?

No more than it is today. Remember, people have been doing this sort of 
thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and 
IBM can do it, I'm sure we can too. The dependency lists will be 
longer. We may require dependency lists that allow the choice of one of 
many prereqs or coreqs.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  http://www.FreeBSD.org

The need of the many outweighs the greed of the few.


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-30 Thread Miroslav Lachman

David Chisnall wrote on 2019/04/30 10:22:

On 29/04/2019 21:12, Joe Maloney wrote:
With CFT version you chose to build, and package individual components 
such as sendmail with a port option.  That does entirely solve the 
problem of being able to reinstall sendmail after the fact without a 
rebuild of the userland (base) port but perhaps base flavors could 
solve that problem assuming flavors could extend beyond python.


This sounds very much like local optimisation. It's now easy to create a 
custom base image.  Great.  But how do I express dependencies in ports 
on a specific base configuration? This is easy if I depend on a specific 
base package, but how does this work in your model?  For example, if I 
have a package that depends on a library that is an optional part of the 
base system, how do I express that pkg needs to either refuse to install 
it, or install a userland pkg that includes that library in place of my 
existing version as part of the install process?


More importantly for the container use case, if I want to take a 
completely empty jail and do pkg ins nginx (for example), what does the 
maintainer of the nginx port need to do to express the minimum set of 
the base system that needs to be installed to allow nginx to work?


One of the goals for the pkg base concept was to allow this kind of use 
case, easily creating a minimal environment required to run a single 
service. With a monolithic base package set, you're going to need some 
mechanism other than packages to express the specific base subset 
package that you need and I think that you need to justify why this 
mechanism is better than using small individual packages.


Will it not be maintainer's nightmare to take care of all the 
dependencies on the base packages for each port we have in the ports tree?


Miroslav Lachman
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-29 Thread Cy Schubert
In message <201904291441.x3tefmid072...@gndrsh.dnsmgr.net>, "Rodney W. 
Grimes"
writes:
> > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes <
> > freebsd-...@gndrsh.dnsmgr.net> wrote:
> > 
> > > >
> > > > Correct, this is ZFS only. And it's something we're using specific to
> > > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of
> > > our CFT.
> > >
> > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only",
> > > calling this FreeBSD pkg base when it is not was wrong,
> > > and miss leading.
> > >
> > 
> > Sorry, I disagree.
> Which is fine.
>
> > This pkg base is independent of the ZFS tool we're using
> > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT.
> > These base packages work the same as existing in-tree pkg base on UFS, no
> > difference. If anything are probably safer due to being able to update all
> > of userland in single extract operation, so you don't have out of order
> > extraction of libc or some such.
>
> You missed the major string change and focused on the edge,
> No comment on calling iXsystems :stuff: FreeBSD instead of FreeNAS/TrueOS?
>
> That was the major point of my statement, your miss leading the user
> community, you yourself said this would never be imported into FreeBSD
> base, so I see no reason that it should be called "FreeBSD package Base",
> as it is not, that is a different project.

Taking the last comment on this thread to ask a question and maybe 
refocus a little.

The discussion about granularity begs the question, why pkgbase in the 
first place? My impression was that it allowed people to select which 
components they wanted to either create a lean installation or mix and 
match base packages and ports (possibly with flavours to install in 
/usr rather than $LOCALBASE) such that maybe person A wanted a stock 
install while person B wanted to replace, picking a random example, BSD 
tar with GNU tar. Isn't that the real advantage of pkgbase?

If OTOH it's binary updates V 2.0, what's the point? I'm a little 
rhetorical here but you get my point. If I want ipfw instead pf or 
ipfilter instead of the others I should have the freedom. Similarly if 
I want vim instead of vi I should have the choice to install vim as 
/usr/bin/vi. Otherwise all the effort to replace binary updates makes 
no sense.


-- 
Cheers,
Cy Schubert 
FreeBSD UNIX: Web:  http://www.FreeBSD.org

The need of the many outweighs the greed of the few.


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


RE: CFT: FreeBSD Package Base

2019-04-29 Thread kris

> -Original Message-
> From: Matthias Apitz 
> Sent: Monday, April 29, 2019 10:50 AM
> To: Emmanuel Vadot 
> Cc: Kris Moore ; FreeBSD Stable  sta...@freebsd.org>; freebsd-...@freebsd.org; freebsd-
> hack...@freebsd.org; FreeBSD Current ;
> freebsd-pkgb...@freebsd.org; freebsd-ports@freebsd.org
> Subject: Re: CFT: FreeBSD Package Base
> 
> 
> Why this thread has to go to all these lists? I receive any mail 5 times!
> 
>   Matthias

Fair point. I'll restrict my replies to the -pkgbase list from here on out, 
suggest others do the same. Sorry about the noise 😊

> --
> Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-
> 38902045 Public GnuPG key: http://www.unixarea.de/key.pub N € I N zur EU!
> "Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber.
> Für ein soziales und friedliches Europa der Völker." DKP

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


RE: CFT: FreeBSD Package Base

2019-04-29 Thread kris


> -Original Message-
> From: Rodney W. Grimes 
> Sent: Monday, April 29, 2019 10:41 AM
> To: Kris Moore 
> Cc: Rodney W. Grimes ; Goran Mekić
> ; Emmanuel Vadot ; FreeBSD
> Stable ; FreeBSD Current  curr...@freebsd.org>; freebsd-pkgb...@freebsd.org; freebsd-
> p...@freebsd.org; freebsd-hack...@freebsd.org; freebsd-ports@freebsd.org
> Subject: Re: CFT: FreeBSD Package Base
> 
> > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes <
> > freebsd-...@gndrsh.dnsmgr.net> wrote:
> >
> > > >
> > > > Correct, this is ZFS only. And it's something we're using specific
> > > > to
> > > FreeNAS / TrueOS, which is why I didn't originally mention it as
> > > apart of our CFT.
> > >
> > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", calling
> > > this FreeBSD pkg base when it is not was wrong, and miss leading.
> > >
> >
> > Sorry, I disagree.
> Which is fine.
> 
> > This pkg base is independent of the ZFS tool we're using
> > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT.
> > These base packages work the same as existing in-tree pkg base on UFS,
no
> > difference. If anything are probably safer due to being able to update
all
> > of userland in single extract operation, so you don't have out of order
> > extraction of libc or some such.
> 
> You missed the major string change and focused on the edge,
> No comment on calling iXsystems :stuff: FreeBSD instead of
> FreeNAS/TrueOS?
> 
> That was the major point of my statement, your miss leading the user
> community, you yourself said this would never be imported into FreeBSD
> base, so I see no reason that it should be called "FreeBSD package Base",
> as it is not, that is a different project.
> 
> --
> Rod Grimes
rgri...@freebsd.org

I think somehow you've missed the entire point here. This is being brought
forth as a FreeBSD CFT in the hopes of upstream adoption. No misleading here
whatsoever. The only thing that I wouldn't expect to be imported into base
was this external tool we use on FreeNAS/TrueOS to handle our specific
use-case of ZFS only. Total strawman here.

Seriously, suggest you bother looking at it and reading further to get the
full context. If anything this is far less invasive since it doesn't require
lots of hacking on base, and can even be used to package old versions of
FreeBSD if desired. The only thing I changed to make these images was a
patch to bsdinstall to replace dist-file extraction with 'pkg install
userland kernel pkg ...'.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-29 Thread Matthias Apitz

Why this thread has to go to all these lists? I receive any mail 5
times!

matthias
-- 
Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045
Public GnuPG key: http://www.unixarea.de/key.pub
N € I N zur EU!
"Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber.
Für ein soziales und friedliches Europa der Völker." DKP


signature.asc
Description: PGP signature


RE: CFT: FreeBSD Package Base

2019-04-29 Thread kris


> > >
> > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on
> > 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are
> > the /usr/lib/debug bits.
> 
>  I only see kernel-20190420203550_1.txz and kernel-debug-
> 20190420203550.txz in https://pkg.trueos.org/pkg/freebsd-
> pkgbase/FreeBSD%3A13%3Aamd64/latest/All/
> and kernel-debug only contain the debug files.
>  If I'm not looking in the right directory please correct me.
> 
> 
> --
> Emmanuel Vadot  


Ahh, you are correct. I checked and those packages haven't pushed to the
mirrors yet, Jenkins is still chewing on a build of them here. I was using
the 12-stable packages yesterday which has these changes. They should be
synced up to the mirrors in the next 24-48 hours. Sorry about the confusion.


-- 
Kris Moore
Vice President of Engineering
iXsystems, Inc
Ph: (408) 943-4100
Ph: (408) 943-4101
The Groundbreaking TrueNAS M-Series -
Enterprise Storage & Servers Driven By Open Source

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-29 Thread Kris Moore
On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes <
freebsd-...@gndrsh.dnsmgr.net> wrote:

> >
> > Correct, this is ZFS only. And it's something we're using specific to
> FreeNAS / TrueOS, which is why I didn't originally mention it as apart of
> our CFT.
>
> Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only",
> calling this FreeBSD pkg base when it is not was wrong,
> and miss leading.
>

Sorry, I disagree. This pkg base is independent of the ZFS tool we're using
to wrangle boot-environments. Hence why it wasn't mentioned in the CFT.
These base packages work the same as existing in-tree pkg base on UFS, no
difference. If anything are probably safer due to being able to update all
of userland in single extract operation, so you don't have out of order
extraction of libc or some such.


>
> > For UFS, there will need to be additional care taken when doing updates.
> >
> > --
> > Kris Moore
> > Vice President of Engineering
> > iXsystems, Inc
> > Ph: (408) 943-4100
> > Ph: (408) 943-4101
> > The Groundbreaking TrueNAS M-Series -
> > Enterprise Storage & Servers Driven By Open Source
> >
> > -Original Message-
> > From: Goran Meki? 
> > Sent: Monday, April 29, 2019 9:43 AM
> > To: Kris Moore 
> > Cc: Emmanuel Vadot ; FreeBSD Stable <
> freebsd-sta...@freebsd.org>; FreeBSD Current ;
> freebsd-pkgb...@freebsd.org; freebsd-...@freebsd.org;
> freebsd-hack...@freebsd.org; freebsd-ports@freebsd.org
> > Subject: Re: CFT: FreeBSD Package Base
> >
> > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote:
> > > We've written our own tool "sysutils/sysup" in GO which handles this.
> > > It performs updates using Boot-Environments to ensure that
> > > kernel/world are updated at same time.
> >
> > If I'm right, UFS doesn't support boot environments, so how would it
> work for UFS based installs?
> >
> > I personally feel GO is a bit ackward choice of language for something
> that practically should be part of base. At least I would expect OS
> update/upgrade not to require any external package.
> >
> > Regards,
> > meka
> >
> > ___
> > freebsd-curr...@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "
> freebsd-current-unsubscr...@freebsd.org"
> >
> >
>
> --
> Rod Grimes
> rgri...@freebsd.org
>
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-29 Thread Kris Moore
On Mon, Apr 29, 2019 at 9:55 AM Emmanuel Vadot 
wrote:

> On Mon, 29 Apr 2019 09:25:05 -0400
> Kris Moore  wrote:
>
> > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot 
> > wrote:
> >
> > >
> > >  Hi Kris,
> > >
> > > On Sun, 28 Apr 2019 15:52:21 -0400
> > >  wrote:
> > >
> > > > FreeBSD Community,
> > > >
> > > >
> > > >
> > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and
> > > 13-current
> > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images
> > > which
> > > > will allow users to perform all updating via the 'pkg' command
> directly.
> > > > Rather than trying to answer all questions in this announcement,
> we've
> > > > created a FAQ page with more details. Please refer to this page, and
> let
> > > us
> > > > know if you have additional questions that we can include on that
> page
> > > going
> > > > forward.
> > > >
> > >
> > >  While I appreciate the effort I have some doubt about your
> > > "re-implementation" of pkgbase. I don't see any improvement compared to
> > > what is in base currently, I even see downside of your implementation.
> > >
> > >  - How do you plan with the need of updating kernel first, reboot and
> > > updating the rest of the userland after ? (Needed for major and minor
> > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and
> > > -HEAD branch). This is still a problem with the base pkgbase.
> > >
> >
> > We've written our own tool "sysutils/sysup" in GO which handles this. It
> > performs updates using Boot-Environments to ensure that kernel/world are
> > updated at same time.
> >
>
>  Which could never be imported into FreeBSD.
>

Not suggesting it should be. Just information on how we solved that problem
in our own appliance / platforms. For FreeBSD it would need some tooling
still to handle this style of updating, regardless of which pkg base is
used.

And for what it's worth, FreeBSD is all the poorer for not being able to
bring modern language based tools into the base. Personally I'm hoping the
shift to base-packages makes this a moot point since the idea of 'what is
base' can be diluted to just a manifest of what gets installed out of box.
Just my 2C on the matter though :)


>
> >
> >
> > >  - This is even worse because you are using the same repository for
> > > base and pkg so if a user pkg update and both kernel and pkg(8) needs
> > > to be updated and pkg use a new syscall or capsicum thing it will be
> > > updated first and couldn't proceed with the rest of the update (this is
> > > a supposition, I haven't personally tested).
> > >
> >
> > See above.
>

You can selectively update os/kernel and reboot before doing rest.


> >
> >
> > >  - It seems that multiple kernels isn't supported in your
> > > implementation, this is already supported in pkgbase but still need
> > > some love. This is an important point as it will allow user to choose
> > > easily the kernel that they want to use and will also allow us
> > > developper to push kernels with new features to help testing.
> > >
> >
> > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll
> get
> > the Witness-enabled kernel installed alongside non-debugging one.
>
>  Mhm no, the kernel-debug packages only add the debug file
> in /usr/lib/debug/boot/
>  I'm talking about installing multiple kernels in //
> (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like
> describe here :
>
> https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues
> in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point.
>
>
Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on
13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are the
/usr/lib/debug bits.


>
> > >
> > >  I think that the only advantage that your solution offers is that if
> > > we remove a componant of base (rcmds for example in 12-CURRENT) those
> > > files would be removed as they are in the userland-base package while
> > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and
> > > will not be deleted in the user computer.
> > >
> >
> >
> > Correct, this is one of the things which prompted us to go this
> direction.
> > Being able to handle crazy mixed WITH/WITHOUT flags was important to us,
> > current pkg base did not handle that so gracefully.
>
>  Can you give me more info on this ? What where the WITH/WITHOUT flags
> that causes problems ?
>


I may have to pick Miwi's brain on this, but I believe some of the issues
we saw were when introducing flags such as WITHOUT_RADIUS. Additionally
there is a runtime problem to solve. I.E. if you change flags mid-stream,
and user updates, there was no clean way on pkg-side to remove those
already installed granular packages. Not without external tooling anyway.



>
> --
> Emmanuel Vadot  
>
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send a

RE: CFT: FreeBSD Package Base

2019-04-29 Thread kris

Correct, this is ZFS only. And it's something we're using specific to FreeNAS / 
TrueOS, which is why I didn't originally mention it as apart of our CFT. 

For UFS, there will need to be additional care taken when doing updates. 

-- 
Kris Moore
Vice President of Engineering
iXsystems, Inc
Ph: (408) 943-4100
Ph: (408) 943-4101
The Groundbreaking TrueNAS M-Series -
Enterprise Storage & Servers Driven By Open Source

-Original Message-
From: Goran Mekić  
Sent: Monday, April 29, 2019 9:43 AM
To: Kris Moore 
Cc: Emmanuel Vadot ; FreeBSD Stable 
; FreeBSD Current ; 
freebsd-pkgb...@freebsd.org; freebsd-...@freebsd.org; 
freebsd-hack...@freebsd.org; freebsd-ports@freebsd.org
Subject: Re: CFT: FreeBSD Package Base

On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote:
> We've written our own tool "sysutils/sysup" in GO which handles this. 
> It performs updates using Boot-Environments to ensure that 
> kernel/world are updated at same time.

If I'm right, UFS doesn't support boot environments, so how would it work for 
UFS based installs?

I personally feel GO is a bit ackward choice of language for something that 
practically should be part of base. At least I would expect OS update/upgrade 
not to require any external package.

Regards,
meka

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"


Re: CFT: FreeBSD Package Base

2019-04-29 Thread Kris Moore
On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot 
wrote:

>
>  Hi Kris,
>
> On Sun, 28 Apr 2019 15:52:21 -0400
>  wrote:
>
> > FreeBSD Community,
> >
> >
> >
> > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and
> 13-current
> > using "TrueOS-inspired" packaged base. These are stock FreeBSD images
> which
> > will allow users to perform all updating via the 'pkg' command directly.
> > Rather than trying to answer all questions in this announcement, we've
> > created a FAQ page with more details. Please refer to this page, and let
> us
> > know if you have additional questions that we can include on that page
> going
> > forward.
> >
>
>  While I appreciate the effort I have some doubt about your
> "re-implementation" of pkgbase. I don't see any improvement compared to
> what is in base currently, I even see downside of your implementation.
>
>  - How do you plan with the need of updating kernel first, reboot and
> updating the rest of the userland after ? (Needed for major and minor
> upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and
> -HEAD branch). This is still a problem with the base pkgbase.
>

We've written our own tool "sysutils/sysup" in GO which handles this. It
performs updates using Boot-Environments to ensure that kernel/world are
updated at same time.




>  - This is even worse because you are using the same repository for
> base and pkg so if a user pkg update and both kernel and pkg(8) needs
> to be updated and pkg use a new syscall or capsicum thing it will be
> updated first and couldn't proceed with the rest of the update (this is
> a supposition, I haven't personally tested).
>

See above.


>  - It seems that multiple kernels isn't supported in your
> implementation, this is already supported in pkgbase but still need
> some love. This is an important point as it will allow user to choose
> easily the kernel that they want to use and will also allow us
> developper to push kernels with new features to help testing.
>

Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get
the Witness-enabled kernel installed alongside non-debugging one.


>  - Since you reduced the granularity on the userland bits it would mean
> that if we use your implementation for -p updates we would download the
> whole userland packages instead of just updating the package that was
> patched. For example with pkgbase, updating from 12.0 to 12.0p1 will
> only update the FreeBSD-runtime package. Yes this package is still big
> to download when you compare to what have changed but until pkg(8) have
> delta pkg supports (and if it will have support, I don't know if
> this is a wish or not) this is the best way to go.
>

Correct, this is by design. We used the in-tree pkg base for nearly a year,
and found that the granularity didn't really offer any savings from a
download or time perspective. Updating 100+ packages took far longer than a
single one, due to all the meta operations. Additionally in real-world
usage, we found that base packages tended to all get updated at the same
time, which took far longer via pkg, since it had to go and perform 100+
fetch operations just to download the base system bits.



>  - I see that you are sorting the plist for kernel and userland based
> on the line length [1], why is that ?


Whoops! I'll fix :)


>
>  I think that the only advantage that your solution offers is that if
> we remove a componant of base (rcmds for example in 12-CURRENT) those
> files would be removed as they are in the userland-base package while
> for pkgbase the FreeBSD-rcmd package will be deleted in the repo and
> will not be deleted in the user computer.
>


Correct, this is one of the things which prompted us to go this direction.
Being able to handle crazy mixed WITH/WITHOUT flags was important to us,
current pkg base did not handle that so gracefully. Additionally we've
added some additional features, such as being able to 'pkg install os/src'
to get system sources used in exact build, as well as being able to rebuild
your local world / kernel packages using ports "make config" framework is
super handy.


>
> >
> > Additionally, I will be hosting a Package Base working group at BSDCan
> 2019,
> > and welcome user and developer attendance to discuss this and other
> ongoing
> > package work:
> >
> >
> >
> > https://wiki.freebsd.org/DevSummit/201905/PackageBase
> >
>
>  I will be present and looking forward to work with you on this.
>
>  Cheers,
>
>  P.S. :  FYI I'm working on pkgbase currently and I will have some
> patches to commit soon (bsdinstall support, memstick creation that
> install a pkgbase aware installaton etc ...).
>

Great! Looking forward to discussion then!


>
> [1] :
>
> https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35
>
> --
> Emmanuel Vadot  
>
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To un

CFT: FreeBSD Package Base

2019-04-28 Thread kris
FreeBSD Community,

 

I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current
using "TrueOS-inspired" packaged base. These are stock FreeBSD images which
will allow users to perform all updating via the 'pkg' command directly.
Rather than trying to answer all questions in this announcement, we've
created a FAQ page with more details. Please refer to this page, and let us
know if you have additional questions that we can include on that page going
forward.

 

Additionally, I will be hosting a Package Base working group at BSDCan 2019,
and welcome user and developer attendance to discuss this and other ongoing
package work:

 

https://wiki.freebsd.org/DevSummit/201905/PackageBase

 

 

FAQ

-

https://trueos.github.io/pkgbase-docs/

 

 

Download Links

-

 

FreeBSD 12-STABLE:

https://pkg.trueos.org/iso/freebsd12-pkgbase/

 

FreeBSD 13-CURRENT:

https://pkg.trueos.org/iso/freebsd-pkgbase/

 

 

-- 

Kris Moore

Vice President of Engineering

iXsystems, Inc

Ph: (408) 943-4100

Ph: (408) 943-4101

The Groundbreaking TrueNAS M-Series -

Enterprise Storage & Servers Driven By Open Source

 

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"