I think some of this I just answered
here: https://github.com/JuliaLang/julia/issues/18389 . Here's a quote:
I think a good way to go forward is to start making some metapackages held
> by the Julia orgs. For example, FFT could move out of Base into JuliaMath,
> and there can be a standard Math.jl which simply re-exports the "Julia
> standard math library" using Reexport.jl. This would have agreed on
> packages (with version requirements?) for a specific domain. JuliaMath
> could have Math.jl metapackage, JuliaStats could have a Stats.jl
> metapackage, etc.
> I think this is better than just lumping them all together into a SciBase
> "Science" is huge: the ML people will want the machine learning stack in
> there, there's a whole statistics stack which replicates base R, if you
> want to replicate MATLAB/SciPy then you'd want some differential equations
> solvers which are large packages themselves, arguably PyCall and everything
> associated is "standard Julia". I think using Math, Stats is both
> succinct and informative to others if you share your script, and move the
> discussion of "standards" to a more domain specific level.
> Does anyone see any design issues that may arise due to using Reexport
> here (other than the fact that it's a personal repo)? Or does anyone else
> have a good design?
Metapackages fix your first issue since they can lump a bunch of packages
together, and import a bunch together as well. I think the Julia
organizations (JuliaMath, JuliaStats, JuliaOpt, etc.) are good communities
for deciding a base distribution in their own domains, which is why I am
think that maybe it should be domain specific: each main JuliaDomain org
has a Domain.jl package which you can add with Pkg.add("Domain"), and use
with `using Domain`. Math, Stats, Opt would then be curated separately by
people who are engaged in the specific communities (with overlap of course
since it's open source). A using statement for a few of these still isn't
too bad, and "using Math, Stats" is both succinct and solves the
reproducibility problem noted earlier (and that line could easily be added
a .juliarc file, maybe a function could be added, auto_using("Math"), to
automatically update the .juliarc? Don't like the name).
For your other issues, the fixes are happening by "upgrading the
community". The packages are updating rapidly mostly because many of the
packages themselves are really young and haven't finished adding all of
their "basic" features. The Orgs have a good approval process.
As for packages created by non-computing specialists, scientific computing
is a pretty unique domain where a non-computing specialist may be the only
person who can implement a specific package at the highest level of
performance / sophistication. This means it's less about finding software
developers to do the work, rather it's about teaching and helping domain
specialists make a package. Julia has already been doing this to some
extent: requiring CI testing and documentation, and policing the packages a
bit for version requirements. I think Tony Kelman has done a great job at
being a good "gatekeeper" to METADATA (I've learned a lot from him, thanks
Thus I think the proper way to move forward there is to setup Julia
software development guidelines, and push for people to follow those
guidelines. I tried to help out by writing a blog post which explains most
of the steps
but I think a good guideline with "the proper way to make/manage/make a
Julia package" will be helpful in the long run for non-software development
experts, and then whenever these principles aren't followed one can just be
pointed to there with an explanation of why it's a good practice and how to
On Thursday, September 15, 2016 at 4:45:25 AM UTC-7, Jeremy Cavanagh wrote:
> Hi Everyone,
> There are many comments with which I agree. Chris has put forward many
> good ideas, I also use Java and find the install/upgrade system excellent.
> There is no problem with having lots of APIs in the distribution since they
> are not loaded into you code unless specifically required. I see a couple
> of things that are a problem with Julia which I would like to see some
> changes/improvements in.
> The first is the way that packages are added is really quite tedious and
> time consuming (I suspect this is a result of its REPL origin) which could
> possibly be streamlined with some kind of gui based application.
> The second is that the group working on the Julia language (unlike Java)
> are not also providing the bulk of the packages that help to improve the
> functionality and usefulness of Julia. So how would you decide on a
> suitable base distribution if the bulk are third party packages?
> There is another "problem" (I can't quite think of the correct term) which
> Java handles well, and, that is the stability of packages which are
> released/updated in a more controlled manner. For Julia this seems to be
> very random, again due to them being prepared by third parties, I check for
> updates every time I run and rarely do I go for more than a day without one
> package or other being updated. I can't see a solution to this problem
> unless there is some group through which packages are thoroughly vetted and
> given a seal of approval, again similar to Java, before being released to
> the public.
> This may be due to a lack of proper design or requirements definition or
> that the packages are created by non-computing specialists who are not
> familiar with software development as a discipline. Please do not take this
> as a slight it is not intended to be, I have a very high regard for all
> working in science and technology (I studied mechanical engineering at
> university) and appreciate that the reasons they develop packages is
> because they want them for their own field of study and the packages may
> have been developed in a more organic manner, rather than as a generally
> applicable utility for problems of the type.