Re: Option vs. flavor?

[Mathieu Arnold]

Le 17/12/2017 à 15:32, Adam Weinberger a écrit :
>> On 17 Dec, 2017, at 1:48, Yuri  wrote:
>>
>> On 12/16/17 22:06, Adam Weinberger wrote:
>>> Is the port of any use without the data file? If everybody who uses
>>> the port needs the data file, I wouldn't make it an option at all.
>>> The precedent is that ports with huge data files are marked
>>> NO_PACKAGE, so there's no impact on the package builders. Make it
>>> non-optional, mark it as NO_PACKAGE, and then the port works for all
>>> users.
>>>
>>> If, however, the port IS real-world usable without the data files,
>>> I'd turn the data file into a second port. That way, package people
>>> can very easily install the NO_BUILD slave and still install the
>>> main program by pkg.
>>
>>
>> Hi Adam,
>>
>>
>> It is usable without data. Portions of the data are recommended to be
>> downloaded into the specific directory for particular functions. Of
>> course, it's more convenient to just a have all of it installed, and
>> users will need all of it for all functions.
>>
>> This is the large physics package called Geant4 for simulation of the
>> passage of particles through matter. https://reviews.freebsd.org/D13504
>
> Ah okay. Then I'd say, make a slave port that just
> fetches/extracts/installs the data files, and an option in the Geant4
> master port that defaults to off. Set the slave port to NO_PACKAGE.

I do not really see any reason to set NO_PACKAGE. There already are big
packages in our repositories:

-rw-r--r--  2 root  wheel   1,0G 10 déc.  05:49 ufoai-data-2.5.txz
-rw-r--r--  2 root  wheel   1,1G 14 déc.  10:06 texlive-docs-20150523.txz
-rw-r--r--  2 root  wheel   1,3G 10 déc.  15:30 flightgear-data-2017.3.1.txz
-rw-r--r--  2 root  wheel   1,5G 14 déc.  20:24 diaspora-1.1.1_1.txz
-rw-r--r--  2 root  wheel   1,7G 10 déc.  05:20 urbanterror-data-4.2.023.txz


-- 
Mathieu Arnold




signature.asc
Description: OpenPGP digital signature

Re: Option vs. flavor?

[Adam Weinberger]

On 17 Dec, 2017, at 11:10, Yuri  wrote:

On 12/17/17 06:32, Adam Weinberger wrote:
Ah okay. Then I'd say, make a slave port that just  
fetches/extracts/installs the data files, and an option in the Geant4  
master port that defaults to off. Set the slave port to NO_PACKAGE.


Ok, I will do it this way.


I wouldn't go the flavor route, because the purpose of flavors is to  
generate multiple packages. The 4.5GB data files shouldn't have a  
package, so options is the way to go.



I was actually wondering, why the use of flavors is such a bad idea in  
this case? I know that flavors weren't originally intended for this, but  
it might seem that they lend themselves very well to such use.


One can have a virtual flavor, say @withHugeData. It doesn't get built by  
default because it is virtual. At the same time, somebody who needs data  
can always run 'poudriere  thepkg@withHudeData' and build the needed  
flavor of the package.


What would be the downside of such use, besides it being the use of a  
completely different kind than anticipated?


Hi Yuri,

My reasoning is that options exist to let users configure a port to have  
one feature or another. The key point there is that users can choose the  
configuration themselves.


Flavors exist to make it easier to produce multiple packages without making  
slave ports. The key point here is producing multiple packages. Presumably  
you'd set NO_PACKAGE on the massive data, so producing multiple packages  
isn't a benefit. If you're not looking for the builders to produce multiple  
packages, flavors just isn't the optimal paradigm.


A NO_BUILD slave port is really easy for people who are normally pkg-only.  
There's no dependencies, just fetch/extract/install. I'm assuming that  
Geant4 has a number of dependencies, and it's a much bigger deal for people  
used to pkg install'ing everything.


The other thing you might consider is a simple shell script to fetch and  
extract the data files into place. net/GeoIP does it like that (though not  
for size reasons).


# Adam


--
Adam Weinberger
ad...@adamw.org
http://www.adamw.org

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Yuri]

On 12/17/17 06:32, Adam Weinberger wrote:


Ah okay. Then I'd say, make a slave port that just 
fetches/extracts/installs the data files, and an option in the Geant4 
master port that defaults to off. Set the slave port to NO_PACKAGE.


Ok, I will do it this way.


I wouldn't go the flavor route, because the purpose of flavors is to 
generate multiple packages. The 4.5GB data files shouldn't have a 
package, so options is the way to go.



I was actually wondering, why the use of flavors is such a bad idea in 
this case? I know that flavors weren't originally intended for this, but 
it might seem that they lend themselves very well to such use.


One can have a virtual flavor, say @withHugeData. It doesn't get built 
by default because it is virtual. At the same time, somebody who needs 
data can always run 'poudriere  thepkg@withHudeData' and build the 
needed flavor of the package.


What would be the downside of such use, besides it being the use of a 
completely different kind than anticipated?



Yuri

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Adam Weinberger]

On 17 Dec, 2017, at 1:48, Yuri  wrote:

On 12/16/17 22:06, Adam Weinberger wrote:
Is the port of any use without the data file? If everybody who uses the  
port needs the data file, I wouldn't make it an option at all. The  
precedent is that ports with huge data files are marked NO_PACKAGE, so  
there's no impact on the package builders. Make it non-optional, mark it  
as NO_PACKAGE, and then the port works for all users.


If, however, the port IS real-world usable without the data files, I'd  
turn the data file into a second port. That way, package people can very  
easily install the NO_BUILD slave and still install the main program by  
pkg.



Hi Adam,


It is usable without data. Portions of the data are recommended to be  
downloaded into the specific directory for particular functions. Of  
course, it's more convenient to just a have all of it installed, and  
users will need all of it for all functions.


This is the large physics package called Geant4 for simulation of the  
passage of particles through matter. https://reviews.freebsd.org/D13504


Ah okay. Then I'd say, make a slave port that just  
fetches/extracts/installs the data files, and an option in the Geant4  
master port that defaults to off. Set the slave port to NO_PACKAGE.


I wouldn't go the flavor route, because the purpose of flavors is to  
generate multiple packages. The 4.5GB data files shouldn't have a package,  
so options is the way to go.


# Adam


--
Adam Weinberger
ad...@adamw.org
http://www.adamw.org

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Yuri]

On 12/16/17 22:06, Adam Weinberger wrote:


Is the port of any use without the data file? If everybody who uses 
the port needs the data file, I wouldn't make it an option at all. The 
precedent is that ports with huge data files are marked NO_PACKAGE, so 
there's no impact on the package builders. Make it non-optional, mark 
it as NO_PACKAGE, and then the port works for all users.


If, however, the port IS real-world usable without the data files, I'd 
turn the data file into a second port. That way, package people can 
very easily install the NO_BUILD slave and still install the main 
program by pkg. 



Hi Adam,


It is usable without data. Portions of the data are recommended to be 
downloaded into the specific directory for particular functions. Of 
course, it's more convenient to just a have all of it installed, and 
users will need all of it for all functions.


This is the large physics package called Geant4 for simulation of the 
passage of particles through matter. https://reviews.freebsd.org/D13504



Yuri


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Adam Weinberger]

On 15 Dec, 2017, at 18:38, Yuri  wrote:

One port is small by itself, but it semi-optionally requires 4.5GB of  
static data installed.


It is possible to download this data optionally, conditional on the port  
option DATA_FILES which will be "off" by default, so that the users who  
need the data will install it with DATA_FILES=on.


Alternatively, it is possible to create a flavor, something like @withData.


Should the option be preferred, or should the flavor be preferred?


Hi Yuri,

Is the port of any use without the data file? If everybody who uses the  
port needs the data file, I wouldn't make it an option at all. The  
precedent is that ports with huge data files are marked NO_PACKAGE, so  
there's no impact on the package builders. Make it non-optional, mark it as  
NO_PACKAGE, and then the port works for all users.


If, however, the port IS real-world usable without the data files, I'd turn  
the data file into a second port. That way, package people can very easily  
install the NO_BUILD slave and still install the main program by pkg.


# Adam


--
Adam Weinberger
ad...@adamw.org
http://www.adamw.org

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Shane Ambler]

On 17/12/2017 08:32, Yuri wrote:
> On 12/16/17 13:39, Franco Fichtner wrote:
>> Why not use a separate data package as optional dependency? Solves the
>> conditional fetch.
> 
> 
> But with the port option fetch is also conditional. There is no need to
> create an extra-package.

Flavours aren't for option variations, they are for the same code being
linked against multiple versions of other ports, each with different
dependencies - eg python 2.7/3.6 or ruby 2.2/ruby2.4

You could either make a separate port for the data files or add it as an
option to the main port.

Using an option for the data files, you either make it a default option
so that the data is installed by anyone that installs the pkg or have it
off so that anyone who wants the data files needs to build the port
themselves. Having 4.5GB of optional data, I wouldn't suggest having it
as an option that is on, this way the package repos don't need to add
4.5GB of data for each arch that pkgs are built for.

Add a second port for data files - see games/alephone and alephone-data
for an example. To prevent the pkg being added to pkg repos, add
NO_PACKAGE= Data files too big, user to download manually
Using a second data port means the user can download and install the
data without having to compile the program.

Add info about this to pkg-message for the user to read, even if it is
about building the data port to get the extra data.

As for adding it as an option -

OPTIONS_DEFINE= EXTRADATA

EXTRADATA_DISTFILES= extra_data_files.tgz

post-install-EXTRADATA-on:
${COPYTREE_SHARE} ${WRKDIR}/extra_data_files ${STAGEDIR}/${DATADIR}

-- 
FreeBSD - the place to B...Software Developing

Shane Ambler

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Yuri]

On 12/16/17 13:39, Franco Fichtner wrote:

Why not use a separate data package as optional dependency? Solves the 
conditional fetch.



But with the port option fetch is also conditional. There is no need to 
create an extra-package.



Yuri

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Franco Fichtner]

Why not use a separate data package as optional dependency? Solves the 
conditional fetch.

> On 16. Dec 2017, at 22:02, Yuri  wrote:
> 
>> On 12/16/17 12:42, Ben Woods wrote:
>> Is there any reason why you want to avoid the download with the port “make
>> fetch”?
> 
> To not clog the package builder with huge unnecessary data? Or maybe this 
> shouldn't be a concern.
> 
>> This should not impact you if it uses subpackages and you just install the
>> program with pkg.
> 
> 
> But are subpackages already available?
> 
> 
> Yuri
> 
> ___
> freebsd-ports@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-ports
> To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Yuri]

On 12/16/17 12:42, Ben Woods wrote:

Is there any reason why you want to avoid the download with the port “make
fetch”?


To not clog the package builder with huge unnecessary data? Or maybe 
this shouldn't be a concern.



This should not impact you if it uses subpackages and you just install the
program with pkg.



But are subpackages already available?


Yuri

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Ben Woods]

On Sun, 17 Dec 2017 at 8:29 am, Yuri  wrote:

> On 12/16/17 11:14, Ben Woods wrote:
> > The original post was requesting that the port make fetch only downloads
> > the data conditionally based on the option. Not sure how possible that
> is.
>
>
> Yes, I was in doubt too. sub-packages will probably always fetch and
> build, and I am trying to avoid that.
>
> So I would just stick to flavor. @withData should be a virtual flavor,
> that can be built on demand.
>
> IMO, this makes most sense.
>
>
> Yuri


Hi Yuri,

Is there any reason why you want to avoid the download with the port “make
fetch”?

This should not impact you if it uses subpackages and you just install the
program with pkg.

Regards,
Ben

> --

--
From: Benjamin Woods
woods...@gmail.com
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Yuri]

On 12/16/17 11:14, Ben Woods wrote:

The original post was requesting that the port make fetch only downloads
the data conditionally based on the option. Not sure how possible that is.



Yes, I was in doubt too. sub-packages will probably always fetch and 
build, and I am trying to avoid that.


So I would just stick to flavor. @withData should be a virtual flavor, 
that can be built on demand.


IMO, this makes most sense.


Yuri


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Ben Woods]

On Sun, 17 Dec 2017 at 3:29 am, Thierry Thomas  wrote:

> Le sam. 16 déc. 17 à  2:38:51 +0100, Yuri 
>  écrivait :
>
> > One port is small by itself, but it semi-optionally requires 4.5GB of
> > static data installed.
> >
> > It is possible to download this data optionally, conditional on the port
> > option DATA_FILES which will be "off" by default, so that the users who
> > need the data will install it with DATA_FILES=on.
> >
> > Alternatively, it is possible to create a flavor, something like
> @withData.
> >
> >
> > Should the option be preferred, or should the flavor be preferred?
>
> I'd say that this is a case for sub-packages.
> --
> Th. Thomas.
>

I believe Sub-packages would normally still download the data for the port
build (make fetch), but it would not be included main pkg.

The original post was requesting that the port make fetch only downloads
the data conditionally based on the option. Not sure how possible that is.

Regards,
Ben
-- 

--
From: Benjamin Woods
woods...@gmail.com
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Option vs. flavor?

[Thierry Thomas]

Le sam. 16 déc. 17 à  2:38:51 +0100, Yuri 
 écrivait :

> One port is small by itself, but it semi-optionally requires 4.5GB of 
> static data installed.
> 
> It is possible to download this data optionally, conditional on the port 
> option DATA_FILES which will be "off" by default, so that the users who 
> need the data will install it with DATA_FILES=on.
> 
> Alternatively, it is possible to create a flavor, something like @withData.
> 
> 
> Should the option be preferred, or should the flavor be preferred?

I'd say that this is a case for sub-packages.
-- 
Th. Thomas.


signature.asc
Description: PGP signature

Option vs. flavor?

[Yuri]

One port is small by itself, but it semi-optionally requires 4.5GB of 
static data installed.


It is possible to download this data optionally, conditional on the port 
option DATA_FILES which will be "off" by default, so that the users who 
need the data will install it with DATA_FILES=on.


Alternatively, it is possible to create a flavor, something like @withData.


Should the option be preferred, or should the flavor be preferred?


Thanks,

Yuri

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"