Re: private ports and pkgs versioning

[olli hauer]

On 2016-10-18 14:54, Mark Linimon wrote:
> On Tue, Oct 18, 2016 at 08:49:06AM +0100, Matthew Seaman wrote:
>> Yes, there is a lot of useful stuff in the ports tree to support local
>> ports or even whole local categories of ports.  I can't recall now how I
>> learned about all this stuff -- it may well have been just be a
>> combination of reading Makefiles and hints dropped on mailing lists.  I
>> cannot recall a document describing this stuff anywhere.
> 
> I don't believe that there is one.
> 
> I'm sure there are N locally-grown solutions out in the wild.
> 
> We ought to work together to poll people on what they use.
> 
> As for adding the category, I think there's a quick fix, if you don't
> care about building INDEX.  Add USE_LOCAL_MK=yes to your Makefile
> invocations, and use the patch below.
> 
> Note: I haven't tried this yet, so adding the category to ports/Makefile
> may also be necessary to pacify ports/Mk/bsd.port.subdir.mk (e.g. INDEX.)
> 
> Index: ports/Mk/bsd.local.mk
> ===
> --- ports/Mk/bsd.local.mk (revision 423944)
> +++ ports/Mk/bsd.local.mk (working copy)
> @@ -14,6 +14,8 @@
>  # time should live.
>  #
>  
> +VALID_CATEGORIES+= local
> +
>  .endif # !defined(_POSTMKINCLUDED) && !defined(Local_Pre_Include)
>  
>  .if defined(_POSTMKINCLUDED) && !defined(Local_Post_Include)
> 
> mcl

It's easier for the user to define VALID_CATEGORIES+= in /etc/make.conf, this 
way the own category is not lost in case someone wipes /usr/ports or uses 
portsnap.
For poudriere this should be defined also in $localbase/poudriere/make.conf, 
I'm using this myself for since a long time.

>From my memory (don't nail me down):
I think synth is able to use packages from pkg.FreeBSD.org and can build only 
packages with changed OPTIONS, but to be sure better ask marino@

-- 
olli
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: private ports and pkgs versioning

[Kyle Evans]

On Tue, Oct 18, 2016 at 9:14 AM, Mark Linimon  wrote:
> On Tue, Oct 18, 2016 at 02:52:34PM +0100, Matthew Seaman wrote:
>> Hmmm... looks like this is handled in the 30 or so lines starting at
>> line 1179 of bsd.port.mk
>
> Wouldn't the patch to ports/Makefile just be the following?
>
> .if exists(${.CURDIR}/Makefile.local)
> .include "${.CURDIR}/Makefile.local"
> .endif
>
> Again, I _think_ this is only necessary if you are building INDEX.
>
> mcl
> ___
> freebsd-ports@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

/usr/ports/Makefile.local is already included for consideration @
bsd.port.subdir.mk:142
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: private ports and pkgs versioning

[Mark Linimon]

On Tue, Oct 18, 2016 at 02:52:34PM +0100, Matthew Seaman wrote:
> Hmmm... looks like this is handled in the 30 or so lines starting at
> line 1179 of bsd.port.mk

Wouldn't the patch to ports/Makefile just be the following?

.if exists(${.CURDIR}/Makefile.local)
.include "${.CURDIR}/Makefile.local"
.endif

Again, I _think_ this is only necessary if you are building INDEX.

mcl
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: private ports and pkgs versioning

[Matthew Seaman]

On 10/18/16 13:54, Mark Linimon wrote:
> On Tue, Oct 18, 2016 at 08:49:06AM +0100, Matthew Seaman wrote:
>> Yes, there is a lot of useful stuff in the ports tree to support local
>> ports or even whole local categories of ports.  I can't recall now how I
>> learned about all this stuff -- it may well have been just be a
>> combination of reading Makefiles and hints dropped on mailing lists.  I
>> cannot recall a document describing this stuff anywhere.
> 
> I don't believe that there is one.
> 
> I'm sure there are N locally-grown solutions out in the wild.
> 
> We ought to work together to poll people on what they use.
> 
> As for adding the category, I think there's a quick fix, if you don't
> care about building INDEX.  Add USE_LOCAL_MK=yes to your Makefile
> invocations, and use the patch below.
> 
> Note: I haven't tried this yet, so adding the category to ports/Makefile
> may also be necessary to pacify ports/Mk/bsd.port.subdir.mk (e.g. INDEX.)
> 
> Index: ports/Mk/bsd.local.mk
> ===
> --- ports/Mk/bsd.local.mk (revision 423944)
> +++ ports/Mk/bsd.local.mk (working copy)
> @@ -14,6 +14,8 @@
>  # time should live.
>  #
>  
> +VALID_CATEGORIES+= local
> +
>  .endif # !defined(_POSTMKINCLUDED) && !defined(Local_Pre_Include)
>  
>  .if defined(_POSTMKINCLUDED) && !defined(Local_Post_Include)
> 
> mcl
> 

In many places in the ports you can drop in a Makefile.local that gets
unconditionally included into the existing Makefile at that level.

There are several names that get treated similarly -- so you can have (I
think) Makefile.amd64 which is only included when compiling for that
architecture.  The idea being if you add a whole new file, you won't get
merge conflicts when updating from SVN.  There's a Makefile.inc that
goes one level above that gets included similarly.

Unfortunately it looks like this doesn't apply at the very top level of
the ports, otherwise this would be a really easy way of adding a custom
category.

Hmmm... looks like this is handled in the 30 or so lines starting at
line 1179 of bsd.port.mk

Cheers,

Matthew




signature.asc
Description: OpenPGP digital signature

Re: private ports and pkgs versioning

[Mark Linimon]

On Tue, Oct 18, 2016 at 08:49:06AM +0100, Matthew Seaman wrote:
> Yes, there is a lot of useful stuff in the ports tree to support local
> ports or even whole local categories of ports.  I can't recall now how I
> learned about all this stuff -- it may well have been just be a
> combination of reading Makefiles and hints dropped on mailing lists.  I
> cannot recall a document describing this stuff anywhere.

I don't believe that there is one.

I'm sure there are N locally-grown solutions out in the wild.

We ought to work together to poll people on what they use.

As for adding the category, I think there's a quick fix, if you don't
care about building INDEX.  Add USE_LOCAL_MK=yes to your Makefile
invocations, and use the patch below.

Note: I haven't tried this yet, so adding the category to ports/Makefile
may also be necessary to pacify ports/Mk/bsd.port.subdir.mk (e.g. INDEX.)

Index: ports/Mk/bsd.local.mk
===
--- ports/Mk/bsd.local.mk   (revision 423944)
+++ ports/Mk/bsd.local.mk   (working copy)
@@ -14,6 +14,8 @@
 # time should live.
 #
 
+VALID_CATEGORIES+= local
+
 .endif # !defined(_POSTMKINCLUDED) && !defined(Local_Pre_Include)
 
 .if defined(_POSTMKINCLUDED) && !defined(Local_Post_Include)

mcl
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: private ports and pkgs versioning

[Miroslav Lachman]

Thomas Mueller wrote on 2016/10/18 09:31:


I too might like to compile something not in the ports tree, would like to make a 
category "mystuff" but don't really know how to make it work.

I might get an error message for invalid category, as happened to me once in 
NetBSD with pkgsrc, and I also don't want it to be changed by svn update, or 
cvs update in the case of NetBSD and pkgsrc.

One package of possible interest is Steffen Daode Nurpmeso's s-nail, to be 
renamed to s-mailx, which is an update to heirloom-mailx and mailx or nail; 
included in Arch Linux and OpenBSD ports.

Or maybe I want to test a modification to an existing package.


You can start by modifying existing port.
If you want to and your own local category you should add it to 
/usr/ports/Makefile first (SUBDIR += mycategory)


Then you should add new SUBDIR line to /usr/ports/mycategory/Makefile

Miroslav Lachman
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: private ports and pkgs versioning

[Matthew Seaman]

On 18/10/2016 06:58, Julian Elischer wrote:
> Is there a standard way of naming a pkg that is locally compiled (maybe
> with a different set of options, or a local patch) so that it doesn't
> get confused with generic ports that are from freeBSD.org?

If you're not making local changes to the port, it's usual that you'ld
just call your locally compiled package the same as the generic ports.

If you're creating a new local port, or a variant on one of the standard
ports, then you should use a distinct name.  Using PKGNAMEPREFIX is
probably the simplest way to do that, but obviously that may cause
problems where PKGNAMEPREFIX is already in use -- eg. any python or perl
or apache24 port.  Similarly there's a PKGNAMESUFFIX but similar caveats
apply.  If you go down this route, you'ld also have to chase any
dependency lines in ports you want to use that depend on your local
variant, so you might end up with a quite significantly modified ports tree.

> I want to use mostly standard pkgs but need to compile a few myself
> (this can't be an uncommon requirement).

The usual way of handling this is to build your own repo using poudriere
(or otherwise), and add this to /usr/local/etc/pkg/repos/foo.conf at
higher priority than the generic FreeBSD repos.  pkg(8) will generally
prefer packages from higher priority repos.

Note that building a repo like this with poudriere will include not just
the packages you're interested in but also all of their build- and run-
time dependencies.

> How can I name my variant pkgs so the the pkg program (and ports) can
> still know that it is a satisfactory supplier of prerequisite components?
> 
> e.g.  if a pkg wants a specific rev of libxml2 how much does it use of
> the name libxml2-2.9.2_2 ?

pkg(8) doesn't have a mechanism for only looking for one specific
version of a package -- it's always going to look for the most recent
version of any package.

You could take a local copy of the port and add a PORTEPOCH, but
otherwise freeze the port at your preferred version.  The PORTEPOCH will
automatically make your copy a higher version that what's available from
the general ports tree, so pkg(8) should prefer it.

Note that doing this will foil the mechanisms used by pkg-audit(8) to
warn you of any vulnerabilities in libxml2

> Can (should) I add stuff after the '_'?  If I do will it still recognise
> my pkg file and if I do are there any rules regarding *WHAT* I can put
> there.

Package versions look like this:

aa.aa.aa_b,c

where aa.aa.aa is the upstream version number, or derived from it.  'b'
is the PORTREVISION which is an integer and omitted if zero. 'c' is the
PORTEPOCH -- again an integer and ommitted when zero.  PORTREVISION is
reset to zero whenever the upstream version changes, but PORTEPOCH if
used is there for ever and much be monotonically increasing.  When
comparing two versions, the PORTEPOCH has the greatest significance,
then the upstream version, then the PORTREVISION.

Only the upstream version can contain non-digits -- '.' as a field
separator and single alpha characters in some fields.  The ports will
take the upstream DISTVERSION and automatically produce a PORTVERSION
modified according to these rules.  The point of this transformation is
to ensure that versions sort in a useful way.

So, for example, the lang/perl5.22 port contains:

PORTNAME=   perl
DISTVERSION=${PERL_VERSION}-RC4
PORTREVISION=   0

where:

% make -V PERL_VERSION
5.22.3

and this results in a PKGNAME of:

% pkg info -x perl5
perl5-5.22.3.r4

> lastly is there somewhere I should be looking to read all this
> information rather than pestering the mailing list?

The Porter's Handbook contains a lot of this stuff, and there is a lot
of documentation in the comments in /usr/ports/Mk/bsd.port.mk and other
Makefiles in that directory.

The rules on package version numbers are expressed in code in
pkg-version(8) -- here:

https://github.com/freebsd/pkg/blob/master/libpkg/pkg_version.c

> What I'd like to have is my own depot with something like:
> 
> ===
> 
> libsmi-0.4.8_1.txz
> libxcb-1.11.1.txz
> libxml2-2.9.2_2-mumble.3.txz
> lsof-4.88,8.txz
> lsof-4.90.b,8.txz
> m4-1.4.17_1,1.txz
> 
> ===
> 
> where 'mumble.3' is a locally defined addition, but all the rest of the
> pkgs are straight from pkg.freebsd.org, (or at least compile in default
> form).

That 2.9.2_2-mumble.3 version string doesn't play by the rules.  For a
start, the system will consider the version number to be everything
after the *last* '-' character in the pkg name, not counting any .txz or
similar suffix.  So your version here is 'mumble.3'  which is still
illegal according to the rules.

It also considered everything before the last '-' character to be the
package name so instead of 'libxml2' you've now got 'libxml2-2.9.2_2'
and that won't match properly when trying to resolve dependencies.

You'ld be better off aiming for a version like this:

   libxml2-2.9.2.m3_2.txz

which you could achieve by directly 

Re: private ports and pkgs versioning

[Thomas Mueller]

> Is there a standard way of naming a pkg that is locally compiled (maybe with a
> different set of options, or a local patch) so that it doesn't get confused
> with generic ports that are from freeBSD.org?

> I want to use mostly standard pkgs but need to compile a few myself (this
> can't be an uncommon requirement).

> How can I name my variant pkgs so the the pkg program (and ports) can still
> know that it is a satisfactory supplier of prerequisite components?

> e.g.  if a pkg wants a specific rev of libxml2 how much does it use of the
> name libxml2-2.9.2_2 ?

> Can (should) I add stuff after the '_'?  If I do will it still recognise my
> pkg file and if I do are there any rules regarding *WHAT* I can put there.

> lastly is there somewhere I should be looking to read all this information
> rather than pestering the mailing list?

> What I'd like to have is my own depot with something like:

===

> libsmi-0.4.8_1.txz
> libxcb-1.11.1.txz
> libxml2-2.9.2_2-mumble.3.txz
> lsof-4.88,8.txz
> lsof-4.90.b,8.txz
> m4-1.4.17_1,1.txz

===

> where 'mumble.3' is a locally defined addition, but all the rest of the pkgs
> are straight from pkg.freebsd.org, (or at least compile in default form).

> This "kind-of" works, but the rules of play are not defined anyhere I have
> read, so I don't know if it's going to suddenly fail one day.

> Also it's be really nice if there were a variable I could set to "mumble.3" so
> that I don't have to do a manual rename, because between the time that the
> package is made and it is renamed there is a 'misnamed' package sitting around
> acting as a potential source of confusion.

> thanks,


> Julian

I too might like to compile something not in the ports tree, would like to make 
a category "mystuff" but don't really know how to make it work.

I might get an error message for invalid category, as happened to me once in 
NetBSD with pkgsrc, and I also don't want it to be changed by svn update, or 
cvs update in the case of NetBSD and pkgsrc.

One package of possible interest is Steffen Daode Nurpmeso's s-nail, to be 
renamed to s-mailx, which is an update to heirloom-mailx and mailx or nail; 
included in Arch Linux and OpenBSD ports.

Or maybe I want to test a modification to an existing package.

Tom

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

private ports and pkgs versioning

[Julian Elischer]

Is there a standard way of naming a pkg that is locally compiled 
(maybe with a different set of options, or a local patch) so that it 
doesn't get confused with generic ports that are from freeBSD.org?


I want to use mostly standard pkgs but need to compile a few myself 
(this can't be an uncommon requirement).


How can I name my variant pkgs so the the pkg program (and ports) can 
still know that it is a satisfactory supplier of prerequisite components?


e.g.  if a pkg wants a specific rev of libxml2 how much does it use of 
the name libxml2-2.9.2_2 ?


Can (should) I add stuff after the '_'?  If I do will it still 
recognise my pkg file and if I do are there any rules regarding *WHAT* 
I can put there.


lastly is there somewhere I should be looking to read all this 
information rather than pestering the mailing list?


What I'd like to have is my own depot with something like:

===

libsmi-0.4.8_1.txz
libxcb-1.11.1.txz
libxml2-2.9.2_2-mumble.3.txz
lsof-4.88,8.txz
lsof-4.90.b,8.txz
m4-1.4.17_1,1.txz

===

where 'mumble.3' is a locally defined addition, but all the rest of 
the pkgs are straight from pkg.freebsd.org, (or at least compile in 
default form).


This "kind-of" works, but the rules of play are not defined anyhere I 
have read, so I don't know if it's going to suddenly fail one day.


Also it's be really nice if there were a variable I could set to 
"mumble.3" so that I don't have to do a manual rename, because between 
the time that the package is made and it is renamed there is a 
'misnamed' package sitting around acting as a potential source of 
confusion.


thanks,


Julian





___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"