[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Chris Rackauckas]

I think some of this I just answered 
here: https://github.com/JuliaLang/julia/issues/18389 . Here's a quote:

I think a good way to go forward is to start making some metapackages held 
> by the Julia orgs. For example, FFT could move out of Base into JuliaMath, 
> and there can be a standard Math.jl which simply re-exports the "Julia 
> standard math library" using Reexport.jl. This would have agreed on 
> packages (with version requirements?) for a specific domain. JuliaMath 
> could have Math.jl metapackage, JuliaStats could have a Stats.jl 
> metapackage, etc.
>
> I think this is better than just lumping them all together into a SciBase 
> because 
> "Science" is huge: the ML people will want the machine learning stack in 
> there, there's a whole statistics stack which replicates base R, if you 
> want to replicate MATLAB/SciPy then you'd want some differential equations 
> solvers which are large packages themselves, arguably PyCall and everything 
> associated is "standard Julia". I think using Math, Stats is both 
> succinct and informative to others if you share your script, and move the 
> discussion of "standards" to a more domain specific level.
>
> Does anyone see any design issues that may arise due to using Reexport 
> here (other than the fact that it's a personal repo)? Or does anyone else 
> have a good design?
>

Metapackages fix your first issue since they can lump a bunch of packages 
together, and import a bunch together as well. I think the Julia 
organizations (JuliaMath, JuliaStats, JuliaOpt, etc.) are good communities 
for deciding a base distribution in their own domains, which is why I am 
think that maybe it should be domain specific: each main JuliaDomain org 
has a Domain.jl  package which you can add with Pkg.add("Domain"), and use 
with `using Domain`. Math, Stats, Opt would then be curated separately by 
people who are engaged in the specific communities (with overlap of course 
since it's open source). A using statement for a few of these still isn't 
too bad, and "using Math, Stats" is both succinct and solves the 
reproducibility problem noted earlier (and that line could easily be added 
a .juliarc file, maybe a function could be added, auto_using("Math"), to 
automatically update the .juliarc? Don't like the name).

For your other issues, the fixes are happening by "upgrading the 
community". The packages are updating rapidly mostly because many of the 
packages themselves are really young and haven't finished adding all of 
their "basic" features. The Orgs have a good approval process. 

As for packages created by non-computing specialists, scientific computing 
is a pretty unique domain where a non-computing specialist may be the only 
person who can implement a specific package at the highest level of 
performance / sophistication. This means it's less about finding software 
developers to do the work, rather it's about teaching and helping domain 
specialists make a package. Julia has already been doing this to some 
extent: requiring CI testing and documentation, and policing the packages a 
bit for version requirements. I think Tony Kelman has done a great job at 
being a good "gatekeeper" to METADATA (I've learned a lot from him, thanks 
Tony!). 

Thus I think the proper way to move forward there is to setup Julia 
software development guidelines, and push for people to follow those 
guidelines. I tried to help out by writing a blog post which explains most 
of the steps 
,
 
but I think a good guideline with "the proper way to make/manage/make a 
Julia package" will be helpful in the long run for non-software development 
experts, and then whenever these principles aren't followed one can just be 
pointed to there with an explanation of why it's a good practice and how to 
do it.

On Thursday, September 15, 2016 at 4:45:25 AM UTC-7, Jeremy Cavanagh wrote:
>
>
> Hi Everyone,
>
> There are many comments with which I agree. Chris has put forward many 
> good ideas, I also use Java and find the install/upgrade system excellent. 
> There is no problem with having lots of APIs in the distribution since they 
> are not loaded into you code unless specifically required. I see a couple 
> of things that are a problem with Julia which I would like to see some 
> changes/improvements in. 
>
> The first is the way that packages are added is really quite tedious and 
> time consuming (I suspect this is a result of its REPL origin) which could 
> possibly be streamlined with some kind of gui based application.
>
> The second is that the group working on the Julia language (unlike Java) 
> are not also providing the bulk of the packages that help to improve the 
> functionality and usefulness of Julia. So how would you decide on a 
> suitable base distribution if the bulk are third party packages? 
>
> There is another "problem" (I can't quite think of the correct term) 

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Jeremy Cavanagh]

Hi Everyone,

There are many comments with which I agree. Chris has put forward many good 
ideas, I also use Java and find the install/upgrade system excellent. There 
is no problem with having lots of APIs in the distribution since they are 
not loaded into you code unless specifically required. I see a couple of 
things that are a problem with Julia which I would like to see some 
changes/improvements in. 

The first is the way that packages are added is really quite tedious and 
time consuming (I suspect this is a result of its REPL origin) which could 
possibly be streamlined with some kind of gui based application.

The second is that the group working on the Julia language (unlike Java) 
are not also providing the bulk of the packages that help to improve the 
functionality and usefulness of Julia. So how would you decide on a 
suitable base distribution if the bulk are third party packages? 

There is another "problem" (I can't quite think of the correct term) which 
Java handles well, and, that is the stability of packages which are 
released/updated in a more controlled manner. For Julia this seems to be 
very random, again due to them being prepared by third parties, I check for 
updates every time I run and rarely do I go for more than a day without one 
package or other being updated. I can't see a solution to this problem 
unless there is some group through which packages are thoroughly vetted and 
given a seal of approval, again similar to Java, before being released to 
the public. 

This may be due to a lack of proper design or requirements definition or 
that the packages are created by non-computing specialists who are not 
familiar with software development as a discipline. Please do not take this 
as a slight it is not intended to be, I have a very high regard for all 
working in science and technology (I studied mechanical engineering at 
university) and appreciate that the reasons they develop packages is 
because they want them for their own field of study and the packages may 
have been developed in a more organic manner, rather than as a generally 
applicable utility for problems of the type.

Re: [julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Mosè Giordano]

2016-09-15 0:10 GMT+02:00 Chris Rackauckas:
> If you use Reexport.jl's @reexport in the top-scope of a module for such a
> metapackage it will actually export all of the functions from inside:
>
> module MyMetapackage # Setup t
>   using Reexport
>   @reexport using DifferentialEquations
>   @reexport using Plots
> end
>
>
>
> using MyMetapackage # Imports both packages
> plot # 3 methods
> solve # 7 methods

Nice!  Then I'd vote in favor of meta-packages, rather than distributions.

It seems we basically came to something already proposed almost two
years ago: https://github.com/JuliaLang/julia/issues/1986#issuecomment-67957609
:-)

Bye,
Mosè

Re: [julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Chris Rackauckas]

If you use Reexport.jl's @reexport in the top-scope of a module for such a 
metapackage it will actually export all of the functions from inside:

module MyMetapackage # Setup t
  using Reexport
  @reexport using DifferentialEquations
  @reexport using Plots
end



using MyMetapackage # Imports both packages
plot # 3 methods
solve # 7 methods


On Wednesday, September 14, 2016 at 3:01:34 PM UTC-7, Mosè Giordano wrote:
>
> 2016-09-14 23:46 GMT+02:00 Chris Rackauckas: 
> > I too am weary about how different distributions would play together. 
> I'd 
> > assume you'd just install with one and that would be it. 
>
> There are many people working in many fields that may want to install 
> different distributions.  I don't think that preventing them to 
> install all of them, or forcing them to choose only one, is a good 
> idea.  Creating many cross-field distributions is not viable.  Some 
> people may discard the idea of installing a distribution just for 
> these reasons. 
>
> > And yes, not 
> > putting "using" in code that you share can lead to some issues, so maybe 
> > "using MyMetapackage" and having the repo online so people can see all 
> of 
> > your packages (at that date/time) would be helpful? 
>
> I think this works better with a meta-package, doesn't it?  The 
> problem is that functions are not exported. 
>
> Bye, 
> Mosè 
>

Re: [julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Mosè Giordano]

2016-09-14 23:46 GMT+02:00 Chris Rackauckas:
> I too am weary about how different distributions would play together. I'd
> assume you'd just install with one and that would be it.

There are many people working in many fields that may want to install
different distributions.  I don't think that preventing them to
install all of them, or forcing them to choose only one, is a good
idea.  Creating many cross-field distributions is not viable.  Some
people may discard the idea of installing a distribution just for
these reasons.

> And yes, not
> putting "using" in code that you share can lead to some issues, so maybe
> "using MyMetapackage" and having the repo online so people can see all of
> your packages (at that date/time) would be helpful?

I think this works better with a meta-package, doesn't it?  The
problem is that functions are not exported.

Bye,
Mosè

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Chris Rackauckas]

I didn't think about using metapackages instead to do "distributions" (with 
Reexport.jl to make using Metapackage import all of the packages). That 
seems like an interesting idea.

I too am weary about how different distributions would play together. I'd 
assume you'd just install with one and that would be it. And yes, not 
putting "using" in code that you share can lead to some issues, so maybe 
"using MyMetapackage" and having the repo online so people can see all of 
your packages (at that date/time) would be helpful?

It really is an interplay between: things keep moving out of Base (like the 
most recent talk is for LinSpace), so the list of "how many usings I need 
for what I am doing" grows. This is both a good thing because Base is 
leaner, but a bad thing because needing to be too explicit about every 
function you want to use is a PITA. Maybe "distributions" is overkill and 
metapackages + a standard library would be better. g's suggestion of 
ways to exclude the StandardLibrary would be helpful, though I don't know 
if an exclude command is possible (though if it was setup to import some 
standard library, julia --barebones seems possible. I don't know if exclude 
is possible because I don't think there's an easy way to remove a package 
after it's been imported).

On Wednesday, September 14, 2016 at 2:03:17 PM UTC-7, Mosè Giordano wrote:
>
> Hi Chris,
>
> what would be the difference between a distribution and a meta-package 
> like those used, for example, in Debian and derivatives: a package without 
> code, that only requires other packages?  In this sense you can create 
> right now a meta-package: just create a repository for a Julia package and 
> populate the REQUIRE file as you wish.  You can specify a Julia version and 
> provide a build script.
>
> I'm above all worried about the interplay between different 
> distributions.  There would be clashing .juliarc's?  Why providing a 
> .juliarc at all?  Automatically running at startup some "using"s & co. is 
> definitely useful, but can lead in the long run to bad habits, like 
> forgetting to put "using" commands in code shared with people not using the 
> same distribution.
>
> Bye,
> Mosè
>
> I think one major point of contention when talking about what should be 
>> included in Base due to competing factors:
>>
>>
>>1. Some people would like a "lean Base" for things like embedded 
>>installs or other low memory applications
>>2. Some people want a MATLAB-like "bells and whistles" approach. This 
>>way all the functions they use are just there: no extra packages to 
>>find/import.
>>3. Some people like having things in Base because it "standardizes" 
>>things. 
>>4. Putting things in Base constrains their release schedule. 
>>5. Putting things in packages outside of JuliaLang helps free up 
>>Travis.
>>
>>
>> The last two concerns have been why things like JuliaMath have sprung up 
>> to move things out of Base. However, I think there is some credibility to 
>> having some form of standardization. I think this can be achieved through 
>> some kind of standard library. This would entail a set of packages which 
>> are installed when Julia is installed, and a set of packages which add 
>> their using statement to the .juliarc. To most users this would be 
>> seamless: they would install automatically, and every time you open Julia, 
>> they would import automatically. There are a few issues there:
>>
>>
>>1.  This wouldn't work with building from source. This idea works 
>>better for binaries (this is no biggie since these users are likely more 
>>experienced anyways)
>>2. Julia would have to pick winners and losers.
>>
>> That second part is big: what goes into the standard library? Would all 
>> of the MATLAB functions like linspace, find, etc. go there? Would the 
>> sparse matrix library be included?
>>
>> I think one way to circumvent the second issue would be to allow for 
>> Julia Distributions. A distribution would be defined by:
>>
>>
>>1. A Julia version
>>2. A List of packages to install (with versions?)
>>3. A build script
>>4. A .juliarc
>>
>> The ideal would be for one to be able to make an executable from those 
>> parts which would install the Julia version with the specified packages, 
>> build the packages (and maybe modify some environment variables / 
>> defaults), and add a .juliarc that would automatically import some packages 
>> / maybe define some constants or checkout branches. JuliaLang could then 
>> provide a lean distribution and a "standard distribution" where the 
>> standard distribution is a more curated library which people can fight 
>> about, but it's not as big of a deal if anyone can make their own. This has 
>> many upsides:
>>
>>
>>1. Julia wouldn't have to come with what you don't want.
>>2. Other than some edge cases where the advantages of Base come into 
>>play (I don't know of a good example, but I 

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[ggggg]

One could imagine a metapackage where 

using StandardLibrary

imports all the goodies (as Gabriel put it), and that wouldn't be too 
painful. Would it be possible to have it the other way around though, where 
all the goodies are imported unless you do something else like

exclude StandardLibrary

or 

julia --barebones

That way the extra work falls to the presumably more experienced users who 
want more control, rather than the potential ex-MATLAB user who just wants 
to take an fft. Of course this only makes sense for the StandardLibrary, 
not for a world with many distributions.

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Mosè Giordano]

Hi Chris,

what would be the difference between a distribution and a meta-package like 
those used, for example, in Debian and derivatives: a package without code, 
that only requires other packages?  In this sense you can create right now 
a meta-package: just create a repository for a Julia package and populate 
the REQUIRE file as you wish.  You can specify a Julia version and provide 
a build script.

I'm above all worried about the interplay between different distributions.  
There would be clashing .juliarc's?  Why providing a .juliarc at all?  
Automatically running at startup some "using"s & co. is definitely useful, 
but can lead in the long run to bad habits, like forgetting to put "using" 
commands in code shared with people not using the same distribution.

Bye,
Mosè

I think one major point of contention when talking about what should be 
> included in Base due to competing factors:
>
>
>1. Some people would like a "lean Base" for things like embedded 
>installs or other low memory applications
>2. Some people want a MATLAB-like "bells and whistles" approach. This 
>way all the functions they use are just there: no extra packages to 
>find/import.
>3. Some people like having things in Base because it "standardizes" 
>things. 
>4. Putting things in Base constrains their release schedule. 
>5. Putting things in packages outside of JuliaLang helps free up 
>Travis.
>
>
> The last two concerns have been why things like JuliaMath have sprung up 
> to move things out of Base. However, I think there is some credibility to 
> having some form of standardization. I think this can be achieved through 
> some kind of standard library. This would entail a set of packages which 
> are installed when Julia is installed, and a set of packages which add 
> their using statement to the .juliarc. To most users this would be 
> seamless: they would install automatically, and every time you open Julia, 
> they would import automatically. There are a few issues there:
>
>
>1.  This wouldn't work with building from source. This idea works 
>better for binaries (this is no biggie since these users are likely more 
>experienced anyways)
>2. Julia would have to pick winners and losers.
>
> That second part is big: what goes into the standard library? Would all of 
> the MATLAB functions like linspace, find, etc. go there? Would the sparse 
> matrix library be included?
>
> I think one way to circumvent the second issue would be to allow for Julia 
> Distributions. A distribution would be defined by:
>
>
>1. A Julia version
>2. A List of packages to install (with versions?)
>3. A build script
>4. A .juliarc
>
> The ideal would be for one to be able to make an executable from those 
> parts which would install the Julia version with the specified packages, 
> build the packages (and maybe modify some environment variables / 
> defaults), and add a .juliarc that would automatically import some packages 
> / maybe define some constants or checkout branches. JuliaLang could then 
> provide a lean distribution and a "standard distribution" where the 
> standard distribution is a more curated library which people can fight 
> about, but it's not as big of a deal if anyone can make their own. This has 
> many upsides:
>
>
>1. Julia wouldn't have to come with what you don't want.
>2. Other than some edge cases where the advantages of Base come into 
>play (I don't know of a good example, but I know there are some things 
>which can't be defined outside of Base really well, like BigFloats? I'm 
> not 
>the expert on this.), most things could spawn out to packages without the 
>standard user ever noticing.
>3. There would still be a large set of standard functions you can 
>assume most people will have.
>4. You can share Julia setups: for example, with my lab I would share 
>a distribution that would have all of the JuliaDiffEq packages installed, 
>along with Plots.jl and some backends, so that way it would be "in the box 
>solve differential equations and plot" setup like what MATLAB provides. I 
>could pick packages/versions that I know work well together, 
>and guarantee their install will work. 
>5. You could write tutorials / run workshops which use a distribution, 
>knowing that a given set of packages will be available.
>6. Anyone could make their setup match yours by looking at the 
>distribution setup scripts (maybe just make a base function which runs 
> that 
>install since it would all be in Julia). This would be nice for some work 
>in progress projects which require checking out master on 3 different 
>packages, and getting some weird branch for another 5. I would give you a 
>succinct and standardized way to specify an install to get there.
>
>
> Side notes:
>
> [An interesting distribution would be that JuliaGPU could provide a full 
> distribution for 

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Steven Sagaert]

for me a distribution is more than just a gobbled together bunch of 
disparate packages: ideally it should have a common style and work with 
common datastructures for input/ouput (between methods) to exchange data. 
That's the real crux of the problem, not the fact that you need to manually 
import packages or not. 

On Tuesday, September 13, 2016 at 4:49:50 PM UTC+2, Randy Zwitch wrote:
>
> "Also, there's a good reason to ask "why fuss with distributions when 
> anyone could just add the packages and add the import statements to their 
> .juliarc?" (though its target audience is for people who don't know details 
> like the .juliarc, but also want Julia to work seamlessly like MATLAB)."
>
> I feel like if you're using MATLAB, it should be a really small step to 
> teach about the .juliarc file, as oppose to the amount of 
> maintenance/fragmentation that comes along with multiple distributions.
>
> Point #1 makes sense for me, if only because that's a use case that can't 
> be accomplished through simple textfile manipulation
>
> On Tuesday, September 13, 2016 at 4:39:15 AM UTC-4, Chris Rackauckas wrote:
>>
>> I think one major point of contention when talking about what should be 
>> included in Base due to competing factors:
>>
>>
>>1. Some people would like a "lean Base" for things like embedded 
>>installs or other low memory applications
>>2. Some people want a MATLAB-like "bells and whistles" approach. This 
>>way all the functions they use are just there: no extra packages to 
>>find/import.
>>3. Some people like having things in Base because it "standardizes" 
>>things. 
>>4. Putting things in Base constrains their release schedule. 
>>5. Putting things in packages outside of JuliaLang helps free up 
>>Travis.
>>
>>
>> The last two concerns have been why things like JuliaMath have sprung up 
>> to move things out of Base. However, I think there is some credibility to 
>> having some form of standardization. I think this can be achieved through 
>> some kind of standard library. This would entail a set of packages which 
>> are installed when Julia is installed, and a set of packages which add 
>> their using statement to the .juliarc. To most users this would be 
>> seamless: they would install automatically, and every time you open Julia, 
>> they would import automatically. There are a few issues there:
>>
>>
>>1.  This wouldn't work with building from source. This idea works 
>>better for binaries (this is no biggie since these users are likely more 
>>experienced anyways)
>>2. Julia would have to pick winners and losers.
>>
>> That second part is big: what goes into the standard library? Would all 
>> of the MATLAB functions like linspace, find, etc. go there? Would the 
>> sparse matrix library be included?
>>
>> I think one way to circumvent the second issue would be to allow for 
>> Julia Distributions. A distribution would be defined by:
>>
>>
>>1. A Julia version
>>2. A List of packages to install (with versions?)
>>3. A build script
>>4. A .juliarc
>>
>> The ideal would be for one to be able to make an executable from those 
>> parts which would install the Julia version with the specified packages, 
>> build the packages (and maybe modify some environment variables / 
>> defaults), and add a .juliarc that would automatically import some packages 
>> / maybe define some constants or checkout branches. JuliaLang could then 
>> provide a lean distribution and a "standard distribution" where the 
>> standard distribution is a more curated library which people can fight 
>> about, but it's not as big of a deal if anyone can make their own. This has 
>> many upsides:
>>
>>
>>1. Julia wouldn't have to come with what you don't want.
>>2. Other than some edge cases where the advantages of Base come into 
>>play (I don't know of a good example, but I know there are some things 
>>which can't be defined outside of Base really well, like BigFloats? I'm 
>> not 
>>the expert on this.), most things could spawn out to packages without the 
>>standard user ever noticing.
>>3. There would still be a large set of standard functions you can 
>>assume most people will have.
>>4. You can share Julia setups: for example, with my lab I would share 
>>a distribution that would have all of the JuliaDiffEq packages installed, 
>>along with Plots.jl and some backends, so that way it would be "in the 
>> box 
>>solve differential equations and plot" setup like what MATLAB provides. I 
>>could pick packages/versions that I know work well together, 
>>and guarantee their install will work. 
>>5. You could write tutorials / run workshops which use a 
>>distribution, knowing that a given set of packages will be available.
>>6. Anyone could make their setup match yours by looking at the 
>>distribution setup scripts (maybe just make a base function which runs 
>> that 
>>

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Randy Zwitch]

"Also, there's a good reason to ask "why fuss with distributions when 
anyone could just add the packages and add the import statements to their 
.juliarc?" (though its target audience is for people who don't know details 
like the .juliarc, but also want Julia to work seamlessly like MATLAB)."

I feel like if you're using MATLAB, it should be a really small step to 
teach about the .juliarc file, as oppose to the amount of 
maintenance/fragmentation that comes along with multiple distributions.

Point #1 makes sense for me, if only because that's a use case that can't 
be accomplished through simple textfile manipulation

On Tuesday, September 13, 2016 at 4:39:15 AM UTC-4, Chris Rackauckas wrote:
>
> I think one major point of contention when talking about what should be 
> included in Base due to competing factors:
>
>
>1. Some people would like a "lean Base" for things like embedded 
>installs or other low memory applications
>2. Some people want a MATLAB-like "bells and whistles" approach. This 
>way all the functions they use are just there: no extra packages to 
>find/import.
>3. Some people like having things in Base because it "standardizes" 
>things. 
>4. Putting things in Base constrains their release schedule. 
>5. Putting things in packages outside of JuliaLang helps free up 
>Travis.
>
>
> The last two concerns have been why things like JuliaMath have sprung up 
> to move things out of Base. However, I think there is some credibility to 
> having some form of standardization. I think this can be achieved through 
> some kind of standard library. This would entail a set of packages which 
> are installed when Julia is installed, and a set of packages which add 
> their using statement to the .juliarc. To most users this would be 
> seamless: they would install automatically, and every time you open Julia, 
> they would import automatically. There are a few issues there:
>
>
>1.  This wouldn't work with building from source. This idea works 
>better for binaries (this is no biggie since these users are likely more 
>experienced anyways)
>2. Julia would have to pick winners and losers.
>
> That second part is big: what goes into the standard library? Would all of 
> the MATLAB functions like linspace, find, etc. go there? Would the sparse 
> matrix library be included?
>
> I think one way to circumvent the second issue would be to allow for Julia 
> Distributions. A distribution would be defined by:
>
>
>1. A Julia version
>2. A List of packages to install (with versions?)
>3. A build script
>4. A .juliarc
>
> The ideal would be for one to be able to make an executable from those 
> parts which would install the Julia version with the specified packages, 
> build the packages (and maybe modify some environment variables / 
> defaults), and add a .juliarc that would automatically import some packages 
> / maybe define some constants or checkout branches. JuliaLang could then 
> provide a lean distribution and a "standard distribution" where the 
> standard distribution is a more curated library which people can fight 
> about, but it's not as big of a deal if anyone can make their own. This has 
> many upsides:
>
>
>1. Julia wouldn't have to come with what you don't want.
>2. Other than some edge cases where the advantages of Base come into 
>play (I don't know of a good example, but I know there are some things 
>which can't be defined outside of Base really well, like BigFloats? I'm 
> not 
>the expert on this.), most things could spawn out to packages without the 
>standard user ever noticing.
>3. There would still be a large set of standard functions you can 
>assume most people will have.
>4. You can share Julia setups: for example, with my lab I would share 
>a distribution that would have all of the JuliaDiffEq packages installed, 
>along with Plots.jl and some backends, so that way it would be "in the box 
>solve differential equations and plot" setup like what MATLAB provides. I 
>could pick packages/versions that I know work well together, 
>and guarantee their install will work. 
>5. You could write tutorials / run workshops which use a distribution, 
>knowing that a given set of packages will be available.
>6. Anyone could make their setup match yours by looking at the 
>distribution setup scripts (maybe just make a base function which runs 
> that 
>install since it would all be in Julia). This would be nice for some work 
>in progress projects which require checking out master on 3 different 
>packages, and getting some weird branch for another 5. I would give you a 
>succinct and standardized way to specify an install to get there.
>
>
> Side notes:
>
> [An interesting distribution would be that JuliaGPU could provide a full 
> distribution for which CUDAnative works (since it requires a different 
> Julia install)]
>
> [A 

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Páll Haraldsson]

On Tuesday, September 13, 2016 at 8:39:15 AM UTC, Chris Rackauckas wrote:
>
>
> This could be a terrible idea, I don't know.
>
 
I don't think so.
 
In the download page there is already a choice of (two extra.. when 
JuliaBox is thought of as such) "distributions", if you will:

"We provide several ways for you to run Julia:

   - In the terminal using the built-in Julia command line.
   - The Juno  integrated development environment 
   (IDE).
   - In the browser on JuliaBox.com" 

That is Juno includes Julia (from memory), so you could have a 
combinatorial explosion.. core packages in a distributions x possible IDEs 
x possible debuggers..

I'm all for a MATLAB like distribution that can compete, with Juno I guess 
(JuliaComputing promotes Ecplise IDE integration..). I hope people can 
agree on something, and avoid a combinatorial explosion.

For myself, I really like julia-lite to be available (it is unofficially).

-- 
Palli.


I think it's great that

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[mmh]

What about sysimage for these pkgs out of base?

On Tuesday, September 13, 2016 at 4:39:15 AM UTC-4, Chris Rackauckas wrote:
>
> I think one major point of contention when talking about what should be 
> included in Base due to competing factors:
>
>
>1. Some people would like a "lean Base" for things like embedded 
>installs or other low memory applications
>2. Some people want a MATLAB-like "bells and whistles" approach. This 
>way all the functions they use are just there: no extra packages to 
>find/import.
>3. Some people like having things in Base because it "standardizes" 
>things. 
>4. Putting things in Base constrains their release schedule. 
>5. Putting things in packages outside of JuliaLang helps free up 
>Travis.
>
>
> The last two concerns have been why things like JuliaMath have sprung up 
> to move things out of Base. However, I think there is some credibility to 
> having some form of standardization. I think this can be achieved through 
> some kind of standard library. This would entail a set of packages which 
> are installed when Julia is installed, and a set of packages which add 
> their using statement to the .juliarc. To most users this would be 
> seamless: they would install automatically, and every time you open Julia, 
> they would import automatically. There are a few issues there:
>
>
>1.  This wouldn't work with building from source. This idea works 
>better for binaries (this is no biggie since these users are likely more 
>experienced anyways)
>2. Julia would have to pick winners and losers.
>
> That second part is big: what goes into the standard library? Would all of 
> the MATLAB functions like linspace, find, etc. go there? Would the sparse 
> matrix library be included?
>
> I think one way to circumvent the second issue would be to allow for Julia 
> Distributions. A distribution would be defined by:
>
>
>1. A Julia version
>2. A List of packages to install (with versions?)
>3. A build script
>4. A .juliarc
>
> The ideal would be for one to be able to make an executable from those 
> parts which would install the Julia version with the specified packages, 
> build the packages (and maybe modify some environment variables / 
> defaults), and add a .juliarc that would automatically import some packages 
> / maybe define some constants or checkout branches. JuliaLang could then 
> provide a lean distribution and a "standard distribution" where the 
> standard distribution is a more curated library which people can fight 
> about, but it's not as big of a deal if anyone can make their own. This has 
> many upsides:
>
>
>1. Julia wouldn't have to come with what you don't want.
>2. Other than some edge cases where the advantages of Base come into 
>play (I don't know of a good example, but I know there are some things 
>which can't be defined outside of Base really well, like BigFloats? I'm 
> not 
>the expert on this.), most things could spawn out to packages without the 
>standard user ever noticing.
>3. There would still be a large set of standard functions you can 
>assume most people will have.
>4. You can share Julia setups: for example, with my lab I would share 
>a distribution that would have all of the JuliaDiffEq packages installed, 
>along with Plots.jl and some backends, so that way it would be "in the box 
>solve differential equations and plot" setup like what MATLAB provides. I 
>could pick packages/versions that I know work well together, 
>and guarantee their install will work. 
>5. You could write tutorials / run workshops which use a distribution, 
>knowing that a given set of packages will be available.
>6. Anyone could make their setup match yours by looking at the 
>distribution setup scripts (maybe just make a base function which runs 
> that 
>install since it would all be in Julia). This would be nice for some work 
>in progress projects which require checking out master on 3 different 
>packages, and getting some weird branch for another 5. I would give you a 
>succinct and standardized way to specify an install to get there.
>
>
> Side notes:
>
> [An interesting distribution would be that JuliaGPU could provide a full 
> distribution for which CUDAnative works (since it requires a different 
> Julia install)]
>
> [A "Data Science Distribution" would be a cool idea: you'd never want to 
> include all of the plotting and statistical things inside of Base, but 
> someone pointing out what all of the "good" packages are that play nice 
> with each other would be very helpful.]
>
> [What if the build script could specify a library path, so that way it can 
> install a setup which doesn't interfere with a standard Julia install?]
>
> This is not without downsides. Indeed, one place where you can look is 
> Python. Python has distributions, but one problem with them is that 
> packages don't 

[julia-users] Re: Idea: Julia Standard Libraries and Distributions

[Steven Sagaert]

I' m in favor of this. In fact I asked for the same thing 
in https://groups.google.com/forum/#!topic/julia-users/3g8zXaXfQqk although 
in a more cryptic way :)

BTW: java already has something like this:  next to the 2 big standard 
distributions javaSE & javaEE (there's also a third specialized javaME but 
that's paying and incompatibel), there are now more fine grained 
distributions called "profiles". In java 9 with modules it will be even 
easier to create more profiles/distributions.


On Tuesday, September 13, 2016 at 10:39:15 AM UTC+2, Chris Rackauckas wrote:
>
> I think one major point of contention when talking about what should be 
> included in Base due to competing factors:
>
>
>1. Some people would like a "lean Base" for things like embedded 
>installs or other low memory applications
>2. Some people want a MATLAB-like "bells and whistles" approach. This 
>way all the functions they use are just there: no extra packages to 
>find/import.
>3. Some people like having things in Base because it "standardizes" 
>things. 
>4. Putting things in Base constrains their release schedule. 
>5. Putting things in packages outside of JuliaLang helps free up 
>Travis.
>
>
> The last two concerns have been why things like JuliaMath have sprung up 
> to move things out of Base. However, I think there is some credibility to 
> having some form of standardization. I think this can be achieved through 
> some kind of standard library. This would entail a set of packages which 
> are installed when Julia is installed, and a set of packages which add 
> their using statement to the .juliarc. To most users this would be 
> seamless: they would install automatically, and every time you open Julia, 
> they would import automatically. There are a few issues there:
>
>
>1.  This wouldn't work with building from source. This idea works 
>better for binaries (this is no biggie since these users are likely more 
>experienced anyways)
>2. Julia would have to pick winners and losers.
>
> That second part is big: what goes into the standard library? Would all of 
> the MATLAB functions like linspace, find, etc. go there? Would the sparse 
> matrix library be included?
>
> I think one way to circumvent the second issue would be to allow for Julia 
> Distributions. A distribution would be defined by:
>
>
>1. A Julia version
>2. A List of packages to install (with versions?)
>3. A build script
>4. A .juliarc
>
> The ideal would be for one to be able to make an executable from those 
> parts which would install the Julia version with the specified packages, 
> build the packages (and maybe modify some environment variables / 
> defaults), and add a .juliarc that would automatically import some packages 
> / maybe define some constants or checkout branches. JuliaLang could then 
> provide a lean distribution and a "standard distribution" where the 
> standard distribution is a more curated library which people can fight 
> about, but it's not as big of a deal if anyone can make their own. This has 
> many upsides:
>
>
>1. Julia wouldn't have to come with what you don't want.
>2. Other than some edge cases where the advantages of Base come into 
>play (I don't know of a good example, but I know there are some things 
>which can't be defined outside of Base really well, like BigFloats? I'm 
> not 
>the expert on this.), most things could spawn out to packages without the 
>standard user ever noticing.
>3. There would still be a large set of standard functions you can 
>assume most people will have.
>4. You can share Julia setups: for example, with my lab I would share 
>a distribution that would have all of the JuliaDiffEq packages installed, 
>along with Plots.jl and some backends, so that way it would be "in the box 
>solve differential equations and plot" setup like what MATLAB provides. I 
>could pick packages/versions that I know work well together, 
>and guarantee their install will work. 
>5. You could write tutorials / run workshops which use a distribution, 
>knowing that a given set of packages will be available.
>6. Anyone could make their setup match yours by looking at the 
>distribution setup scripts (maybe just make a base function which runs 
> that 
>install since it would all be in Julia). This would be nice for some work 
>in progress projects which require checking out master on 3 different 
>packages, and getting some weird branch for another 5. I would give you a 
>succinct and standardized way to specify an install to get there.
>
>
> Side notes:
>
> [An interesting distribution would be that JuliaGPU could provide a full 
> distribution for which CUDAnative works (since it requires a different 
> Julia install)]
>
> [A "Data Science Distribution" would be a cool idea: you'd never want to 
> include all of the plotting and statistical things inside of