Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Le jeu. 20 juil. 2023 à 23:46, Dimitrios Efthymiou a écrit : > > I am not home now, but these are the ones i remember from looking at the > GitHub repo: > AbstractStorelessUnivariateStatistic.java > AbstractUnivariateStatistic.java > WeightedEvaluation.java > Sum.java > FirstMoment.java > Mean.java > SecondMoment.java > StandardDeviation.java > Variance.java > VectorialMean.java Please try what I suggested in my previous message (about 30 lines of code that could be copied into an "internal" package to get the same functionality as the last 2 classes above). Then we can continue discussing (on JIRA) on how to move around roadblocks actually encountered. [You can create a JIRA "sub-task" for each specific problem.] Gilles > > > On Thu, 20 Jul 2023, 22:40 Gilles Sadowski, wrote: > > > Le jeu. 20 juil. 2023 à 23:28, Dimitrios Efthymiou > > a écrit : > > > > > > Unfortunately, i just tried a simple move, but there are deoendencies on > > 3 > > > distance classes > > > > But... those classes are only used by the "clustering" package; they > > are not external dependencies; they would go into the new module > > as first-class citizens. > > [A follow-up issue would be whether those distance classes are > > worth sharing with the other machine learning utility in the module > > "commons-math-neuralnet".] > > > > > and about 12 stats classes, > > > > Which ones? > > > > > because there are transitive > > > dependencies. Not to mention the respective test classes. > > > > Well, of course there is work to do to fix all aspects of the move... > > > > Gilles > > > > > > > > > > > [...] - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
I am not home now, but these are the ones i remember from looking at the GitHub repo: AbstractStorelessUnivariateStatistic.java AbstractUnivariateStatistic.java WeightedEvaluation.java Sum.java FirstMoment.java Mean.java SecondMoment.java StandardDeviation.java Variance.java VectorialMean.java On Thu, 20 Jul 2023, 22:40 Gilles Sadowski, wrote: > Le jeu. 20 juil. 2023 à 23:28, Dimitrios Efthymiou > a écrit : > > > > Unfortunately, i just tried a simple move, but there are deoendencies on > 3 > > distance classes > > But... those classes are only used by the "clustering" package; they > are not external dependencies; they would go into the new module > as first-class citizens. > [A follow-up issue would be whether those distance classes are > worth sharing with the other machine learning utility in the module > "commons-math-neuralnet".] > > > and about 12 stats classes, > > Which ones? > > > because there are transitive > > dependencies. Not to mention the respective test classes. > > Well, of course there is work to do to fix all aspects of the move... > > Gilles > > > > > > > > > [...] > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
I am not home now, but these are the ones i remember from looking at the GitHub repo: AbstractStorelessUnivariateStatistic.java AbstractUnivariateStatistic.java WeightedEvaluation.java Sum.java FirstMoment.java Mean.java SecondMoment.java StandardDeviation.java Variance.java VectorialMean.java On Thu, 20 Jul 2023, 22:40 Gilles Sadowski, wrote: > Le jeu. 20 juil. 2023 à 23:28, Dimitrios Efthymiou > a écrit : > > > > Unfortunately, i just tried a simple move, but there are deoendencies on > 3 > > distance classes > > But... those classes are only used by the "clustering" package; they > are not external dependencies; they would go into the new module > as first-class citizens. > [A follow-up issue would be whether those distance classes are > worth sharing with the other machine learning utility in the module > "commons-math-neuralnet".] > > > and about 12 stats classes, > > Which ones? > > > because there are transitive > > dependencies. Not to mention the respective test classes. > > Well, of course there is work to do to fix all aspects of the move... > > Gilles > > > > > > > > > [...] > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Le jeu. 20 juil. 2023 à 23:28, Dimitrios Efthymiou a écrit : > > Unfortunately, i just tried a simple move, but there are deoendencies on 3 > distance classes But... those classes are only used by the "clustering" package; they are not external dependencies; they would go into the new module as first-class citizens. [A follow-up issue would be whether those distance classes are worth sharing with the other machine learning utility in the module "commons-math-neuralnet".] > and about 12 stats classes, Which ones? > because there are transitive > dependencies. Not to mention the respective test classes. Well, of course there is work to do to fix all aspects of the move... Gilles > > > > > > > [...] - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Unfortunately, i just tried a simple move, but there are deoendencies on 3
distance classes and about 12 stats classes, because there are transitive
dependencies. Not to mention the respective test classes.
On Thu, 20 Jul 2023, 22:22 Gilles Sadowski, wrote:
> Le jeu. 20 juil. 2023 à 21:19, Dimitrios Efthymiou
> a écrit :
> >
> > are you saying that in order to move the ml.clustering classes
> > to the new clustering module, I can take all the dependencies to classes
> > and their transitive dependencies, copy them to the new clustering
> module,
> > but only keep in them the minimum required code for the new module to
> > operate?
>
> To some extent, yes. But the main point is the refactoring. For example,
> your wouldn't copy the code from "linear" after seeing that one can do
> without it. But also note in this case that exposing a "double[][]"
> instead
> may not be the best choice for a long-term API. As was mentioned, it
> would be worth looking at how other libraries provide similar
> functionality.
> The module should solve all issues mentioned in JIRA; it's not just copying
> the classes and removing dependencies.
>
> Gilles
>
> >
> > On Thu, 20 Jul 2023 at 15:27, Gilles Sadowski
> wrote:
> >
> > > Hello.
> > >
> > > Le mer. 19 juil. 2023 à 12:59, Dimitrios Efthymiou
> > > a écrit :
> > > >
> > > > [...]
> > > > 1-- [...]
> > > > 2--As for the atomic refactoring and feature branch, well,
> > > > unless someone moves the Variance class (you said that someone
> > > > is doing it now) and the distance package and whatever other
> > > > dependencies exist within the ml.clustering package,
> > > > there can be no moving of the remaining clustering classes
> > > > to the new clustering module, right?
> > > > 3-- [...]
> > > > 4--I don't know how to continue with the clustering modularisation
> > > > given all those dependencies. Maybe I shouldn't have started this,
> > > > because now I am stuck
> > >
> > > You aren't.
> > >
> > > The dependencies found in "o.a.c.math4.legacy.ml.clustering" are
> > > (1) "MatrixUtils" and "RealMatrix" (from the "linear" package)
> > > (2) "Variance" and "VectorialMean" (from the "stat" package)
> > >
> > > (1) creates the coupling for a single method ("getMembershipMatrix")
> > > that isn't called from anywhere (not even the unit tests). Remove the
> > > method and the dependency towards "linear" vanishes.
> > >
> > > (2) "Variance" can be replaced with a temporary implementation like
> > > (untested copy/paste from "SecondMoment" and "FirstMoment"):
> > > ---CUT---
> > > class Variance {
> > > private int n = 0;
> > > private double dev = 0;
> > > private double nDev = 0;
> > > private double m2 = 0;
> > > private double m1 = 0;
> > >
> > > void increment(final double d) {
> > > ++n;
> > > dev = d - m1;
> > > nDev = dev / n;
> > > m1 += nDev;
> > > m2 += ((double) n - 1) * dev * nDev;
> > > }
> > >
> > > double get() {
> > > return m2;
> > > }
> > > }
> > > ---CUT---
> > > Then, creating a private copy of class "VectorialMean" (replacing,
> > > in the copy, CM exceptions with JDK ones) would complete the
> > > removal of the dependency towards the "stat" package.
> > >
> > > And so on.
> > >
> > > Regards,
> > > Gilles
> > >
> > > > > > [...]
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Le jeu. 20 juil. 2023 à 21:19, Dimitrios Efthymiou
a écrit :
>
> are you saying that in order to move the ml.clustering classes
> to the new clustering module, I can take all the dependencies to classes
> and their transitive dependencies, copy them to the new clustering module,
> but only keep in them the minimum required code for the new module to
> operate?
To some extent, yes. But the main point is the refactoring. For example,
your wouldn't copy the code from "linear" after seeing that one can do
without it. But also note in this case that exposing a "double[][]" instead
may not be the best choice for a long-term API. As was mentioned, it
would be worth looking at how other libraries provide similar functionality.
The module should solve all issues mentioned in JIRA; it's not just copying
the classes and removing dependencies.
Gilles
>
> On Thu, 20 Jul 2023 at 15:27, Gilles Sadowski wrote:
>
> > Hello.
> >
> > Le mer. 19 juil. 2023 à 12:59, Dimitrios Efthymiou
> > a écrit :
> > >
> > > [...]
> > > 1-- [...]
> > > 2--As for the atomic refactoring and feature branch, well,
> > > unless someone moves the Variance class (you said that someone
> > > is doing it now) and the distance package and whatever other
> > > dependencies exist within the ml.clustering package,
> > > there can be no moving of the remaining clustering classes
> > > to the new clustering module, right?
> > > 3-- [...]
> > > 4--I don't know how to continue with the clustering modularisation
> > > given all those dependencies. Maybe I shouldn't have started this,
> > > because now I am stuck
> >
> > You aren't.
> >
> > The dependencies found in "o.a.c.math4.legacy.ml.clustering" are
> > (1) "MatrixUtils" and "RealMatrix" (from the "linear" package)
> > (2) "Variance" and "VectorialMean" (from the "stat" package)
> >
> > (1) creates the coupling for a single method ("getMembershipMatrix")
> > that isn't called from anywhere (not even the unit tests). Remove the
> > method and the dependency towards "linear" vanishes.
> >
> > (2) "Variance" can be replaced with a temporary implementation like
> > (untested copy/paste from "SecondMoment" and "FirstMoment"):
> > ---CUT---
> > class Variance {
> > private int n = 0;
> > private double dev = 0;
> > private double nDev = 0;
> > private double m2 = 0;
> > private double m1 = 0;
> >
> > void increment(final double d) {
> > ++n;
> > dev = d - m1;
> > nDev = dev / n;
> > m1 += nDev;
> > m2 += ((double) n - 1) * dev * nDev;
> > }
> >
> > double get() {
> > return m2;
> > }
> > }
> > ---CUT---
> > Then, creating a private copy of class "VectorialMean" (replacing,
> > in the copy, CM exceptions with JDK ones) would complete the
> > removal of the dependency towards the "stat" package.
> >
> > And so on.
> >
> > Regards,
> > Gilles
> >
> > > > > [...]
-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
are you saying that in order to move the ml.clustering classes
to the new clustering module, I can take all the dependencies to classes
and their transitive dependencies, copy them to the new clustering module,
but only keep in them the minimum required code for the new module to
operate?
On Thu, 20 Jul 2023 at 15:27, Gilles Sadowski wrote:
> Hello.
>
> Le mer. 19 juil. 2023 à 12:59, Dimitrios Efthymiou
> a écrit :
> >
> > [...]
> > 1-- [...]
> > 2--As for the atomic refactoring and feature branch, well,
> > unless someone moves the Variance class (you said that someone
> > is doing it now) and the distance package and whatever other
> > dependencies exist within the ml.clustering package,
> > there can be no moving of the remaining clustering classes
> > to the new clustering module, right?
> > 3-- [...]
> > 4--I don't know how to continue with the clustering modularisation
> > given all those dependencies. Maybe I shouldn't have started this,
> > because now I am stuck
>
> You aren't.
>
> The dependencies found in "o.a.c.math4.legacy.ml.clustering" are
> (1) "MatrixUtils" and "RealMatrix" (from the "linear" package)
> (2) "Variance" and "VectorialMean" (from the "stat" package)
>
> (1) creates the coupling for a single method ("getMembershipMatrix")
> that isn't called from anywhere (not even the unit tests). Remove the
> method and the dependency towards "linear" vanishes.
>
> (2) "Variance" can be replaced with a temporary implementation like
> (untested copy/paste from "SecondMoment" and "FirstMoment"):
> ---CUT---
> class Variance {
> private int n = 0;
> private double dev = 0;
> private double nDev = 0;
> private double m2 = 0;
> private double m1 = 0;
>
> void increment(final double d) {
> ++n;
> dev = d - m1;
> nDev = dev / n;
> m1 += nDev;
> m2 += ((double) n - 1) * dev * nDev;
> }
>
> double get() {
> return m2;
> }
> }
> ---CUT---
> Then, creating a private copy of class "VectorialMean" (replacing,
> in the copy, CM exceptions with JDK ones) would complete the
> removal of the dependency towards the "stat" package.
>
> And so on.
>
> Regards,
> Gilles
>
> > > > [...]
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Hello.
Le mer. 19 juil. 2023 à 12:59, Dimitrios Efthymiou
a écrit :
>
> [...]
> 1-- [...]
> 2--As for the atomic refactoring and feature branch, well,
> unless someone moves the Variance class (you said that someone
> is doing it now) and the distance package and whatever other
> dependencies exist within the ml.clustering package,
> there can be no moving of the remaining clustering classes
> to the new clustering module, right?
> 3-- [...]
> 4--I don't know how to continue with the clustering modularisation
> given all those dependencies. Maybe I shouldn't have started this,
> because now I am stuck
You aren't.
The dependencies found in "o.a.c.math4.legacy.ml.clustering" are
(1) "MatrixUtils" and "RealMatrix" (from the "linear" package)
(2) "Variance" and "VectorialMean" (from the "stat" package)
(1) creates the coupling for a single method ("getMembershipMatrix")
that isn't called from anywhere (not even the unit tests). Remove the
method and the dependency towards "linear" vanishes.
(2) "Variance" can be replaced with a temporary implementation like
(untested copy/paste from "SecondMoment" and "FirstMoment"):
---CUT---
class Variance {
private int n = 0;
private double dev = 0;
private double nDev = 0;
private double m2 = 0;
private double m1 = 0;
void increment(final double d) {
++n;
dev = d - m1;
nDev = dev / n;
m1 += nDev;
m2 += ((double) n - 1) * dev * nDev;
}
double get() {
return m2;
}
}
---CUT---
Then, creating a private copy of class "VectorialMean" (replacing,
in the copy, CM exceptions with JDK ones) would complete the
removal of the dependency towards the "stat" package.
And so on.
Regards,
Gilles
> > > [...]
-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Le mer. 19 juil. 2023 à 17:48, Elliotte Rusty Harold a écrit : > > On Wed, Jul 19, 2023 at 11:38 AM Gilles Sadowski wrote: > > > I think that the page one would look for is this one: > >https://commons.apache.org/proper/commons-math/dependency-info.html > > It's fine to put it there, but even if it's correct it's still too > hard to find. The only way to get to that page is scroll about two > thirds of the way down a sidebar and click one of the four links that > mention "dependencies" that seems as likely to bring you to a list of > commons-math's own dependencies rather than how to add this project as > a dependency. I'm a Maven comitter who's spent way more time in the > depths of Maven Project website generation than 99.9% of Java > programmers and I still couldn't find this the first time I looked for > it. That's de facto evidence that this information is not easy to > find. You are quite right. [IIRC, trying to figure it out from "Maven Central" is even worse.] The "Commons" web site esthetics and ergonomy has never attracted much attention. [After years of a new one being potentially available[1], even the logo did not change (all of the "new" ones will become obsolete in a few months following the ASF rebranding effort).] > > This coordinates belong right up front on > https://commons.apache.org/proper/commons-math/index.html +1 Would you file a report on https://issues.apache.org/jira/projects/COMMONSSITE/ ? > > I am tempted to see about changing the title of that page in the > sidebar from "Dependency Information" to "Maven Coordinates". Those pages are providing dependency info not only for maven: The info is there for (each module too), see e.g. https://commons.apache.org/proper/commons-math/commons-math4-transform/dependency-info.html but it would be nice indeed that the BOM snippet appears prominently on https://commons-math4-transform/dependency-info.html > That > would help a little and it's likely commons-math is hardly the only > project that has this issue. > The web site template is shared by all components. One change will fix them all. ;-) Well, not really: First step is to agree to generate a BOM module like Alex created for "RNG", and that the "Overview" page links to it. That would imply that all projects must become "modular" even if just to have two modules (one "code" and one BOM)... That would be "good" (TM), IMHO. Regards, Gilles [1] https://issues.apache.org/jira/projects/COMMONSSITE/issues/COMMONSSITE-86 - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
On Wed, Jul 19, 2023 at 11:38 AM Gilles Sadowski wrote: > I think that the page one would look for is this one: >https://commons.apache.org/proper/commons-math/dependency-info.html It's fine to put it there, but even if it's correct it's still too hard to find. The only way to get to that page is scroll about two thirds of the way down a sidebar and click one of the four links that mention "dependencies" that seems as likely to bring you to a list of commons-math's own dependencies rather than how to add this project as a dependency. I'm a Maven comitter who's spent way more time in the depths of Maven Project website generation than 99.9% of Java programmers and I still couldn't find this the first time I looked for it. That's de facto evidence that this information is not easy to find. This coordinates belong right up front on https://commons.apache.org/proper/commons-math/index.html I am tempted to see about changing the title of that page in the sidebar from "Dependency Information" to "Maven Coordinates". That would help a little and it's likely commons-math is hardly the only project that has this issue. -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Le mer. 19 juil. 2023 à 16:03, Elliotte Rusty Harold a écrit : > > On Wed, Jul 19, 2023 at 9:53 AM Gilles Sadowski wrote: > > > > org.apache.commons.math4 and org.apache.commons.math3 > > > > > > Although it's not easy to find, > > > > What do you mean? > > Is it something we can fix here? > > > > Probably. I did a google search and hunted around on the web pages at > https://commons.apache.org/proper/commons-math/ > > Nowhere did I find a clear statement that "To import commons-math to > a project use the coordinates org.apache.commons:commons-math3:3.6.1" > or anything like that. I just took another look and I see something > for 4.0 at https://commons.apache.org/proper/commons-math/summary.html > but that's not the first place someone would look, and that's only for > the parent project. Not should the main website be for an unreleased > version. > > I'd put this on both Overview pages and probably the Developer's Guide > page at https://commons.apache.org/proper/commons-math/ I think that the page one would look for is this one: https://commons.apache.org/proper/commons-math/dependency-info.html Unfortunately, the "auto-generating" script/template does not take modular maven projects into account. Rather than assuming a single artefact, the build should somehow generate a BOM that would transitively fetch all the modules. I think that Alex managed to do just that for "Commons RNG".[1] Regards, Gilles [1] https://commons.apache.org/proper/commons-rng/commons-rng-bom/index.html - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
On Wed, Jul 19, 2023 at 9:53 AM Gilles Sadowski wrote: > > org.apache.commons.math4 and org.apache.commons.math3 > > > > Although it's not easy to find, > > What do you mean? > Is it something we can fix here? > Probably. I did a google search and hunted around on the web pages at https://commons.apache.org/proper/commons-math/ Nowhere did I find a clear statement that "To import commons-math to a project use the coordinates org.apache.commons:commons-math3:3.6.1" or anything like that. I just took another look and I see something for 4.0 at https://commons.apache.org/proper/commons-math/summary.html but that's not the first place someone would look, and that's only for the parent project. Not should the main website be for an unreleased version. I'd put this on both Overview pages and probably the Developer's Guide page at https://commons.apache.org/proper/commons-math/ -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Le mer. 19 juil. 2023 à 14:58, Elliotte Rusty Harold a écrit : > > Commons Math 4 and Commons Math 3 have different java packages: > > org.apache.commons.math4 and org.apache.commons.math3 > > Although it's not easy to find, What do you mean? Is it something we can fix here? > it does look like the Maven > coordinates have changed as well. > > so it's effectively a completely new release of a new project that > can coexist with the older project in a classpath. That shouldn't > cause any dependency problems. :-) Thanks, Gilles - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Commons Math 4 and Commons Math 3 have different java packages: org.apache.commons.math4 and org.apache.commons.math3 Although it's not easy to find, it does look like the Maven coordinates have changed as well. so it's effectively a completely new release of a new project that can coexist with the older project in a classpath. That shouldn't cause any dependency problems. -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Hi. Le mer. 19 juil. 2023 à 13:43, Elliotte Rusty Harold a écrit : > > Ok, don't do that unless it's new code in new packages. Otherwise > you're creating a dependency hell for existing clients. It is > extremely developer hostile. Pretty much all of https://jlbp.dev/ > applies but especially > > JLBP-5: Do not include a class in more than one classpath entry > JLBP-6: Rename artifacts and packages together I repeat that it has been "Commons policy" for over 15 years. If I missed something, please use concrete examples, based on the modules shipped with v4.0-beta1, so that we can fix it before the "non-beta" release. Thanks, Gilles > > Debugging the problems this will cause is difficult and painful, even > for someone well-versed in Maven dependency management. > > On Wed, Jul 19, 2023 at 11:37 AM Dimitrios Efthymiou > wrote: > > > > no. I mean creating maven modules inside commons-math, like > > the existing ones: > > commons-math-neuralnet > > commons-math-transform > > > > > [...] - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Hello. Le mer. 19 juil. 2023 à 12:59, Dimitrios Efthymiou a écrit : > > thanks Gilles. > 1--I think I broke the build, because I did not include (correctly) > the dependency on clustering inside the root pom.xml. My local build > succeeds. I hope that the GitHub build succeeds, as well. It doesn't. > 2--As for the atomic refactoring and feature branch, well, > unless someone moves the Variance class (you said that someone > is doing it now) and the distance package and whatever other > dependencies exist within the ml.clustering package, > there can be no moving of the remaining clustering classes > to the new clustering module, right? Wrong. I made a suggestion on JIRA. > 3--I see that the commons-statistics project, for example, > has empty modules. I think the geometry project (I don't remember which one) > has some classes that are still in commons-math Which ones? > because the migration > is not complete. The migration was completed (it took almost two years) before releasing v1.0 of the [Geometry] component. > So, I thought that I coud do the same i.e. move > the clusteirng classes that do not depend on anything The "same" is doing what I suggested (cf. above). > 4--I don't know how to continue with the clustering modularisation > given all those dependencies. Maybe I shouldn't have started this, > because now I am stuck If the work would have only consisted in copying Java files from one directory to another... It would have been done already. Even for packages that didn't have any dependency (see e.g. the "commons-math-transform" module), the port involved looking at the API to make it more "modern" and/or user-friendly. The "clustering" case is more complicated because, in addition to the API changes and enhancements, there are (few) dependencies (to remove), and bugs (to fix). Please look at the JIRA reports. [One main issue is how to represent a point in (problem domain) space and cluster of those, and distance between them, ...] Regards, Gilles >> [...] - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
I see. I didn't suggest I would start creating modules here and there. I just wanted to know if there is a plan to, eventually, put all those legacy packages into their own projects like analysis, linear algebra, special functions, optimisation, etc. That's all. I am not gonna do it. But since on of my favourite things in programming is playing with legacy code and refactoring, I wanted to know if there is a plan for these things. I guess eventually, yes. On Wed, 19 Jul 2023 at 12:43, Elliotte Rusty Harold wrote: > Ok, don't do that unless it's new code in new packages. Otherwise > you're creating a dependency hell for existing clients. It is > extremely developer hostile. Pretty much all of https://jlbp.dev/ > applies but especially > > JLBP-5: Do not include a class in more than one classpath entry > JLBP-6: Rename artifacts and packages together > > Debugging the problems this will cause is difficult and painful, even > for someone well-versed in Maven dependency management. > > On Wed, Jul 19, 2023 at 11:37 AM Dimitrios Efthymiou > wrote: > > > > no. I mean creating maven modules inside commons-math, like > > the existing ones: > > commons-math-neuralnet > > commons-math-transform > > > > On Wed, 19 Jul 2023 at 12:29, Elliotte Rusty Harold > > wrote: > > > > > Could you be precise about what you mean by "modularisation"? It's a > > > very overloaded term. Do you mean Java 9 modules as defined by the > > > JPMS? > > > > > > On Wed, Jul 19, 2023 at 12:33 AM Dimitrios Efthymiou > > > wrote: > > > > > > > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > > > > modularisation of all 14 packages commons-math-legacy has? I think > that > > > > some of them are easy to modularise like optimisation, special and > filter > > > > > > > > > > > > -- > > > Elliotte Rusty Harold > > > elh...@ibiblio.org > > > > > > - > > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > > > > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Ok, don't do that unless it's new code in new packages. Otherwise you're creating a dependency hell for existing clients. It is extremely developer hostile. Pretty much all of https://jlbp.dev/ applies but especially JLBP-5: Do not include a class in more than one classpath entry JLBP-6: Rename artifacts and packages together Debugging the problems this will cause is difficult and painful, even for someone well-versed in Maven dependency management. On Wed, Jul 19, 2023 at 11:37 AM Dimitrios Efthymiou wrote: > > no. I mean creating maven modules inside commons-math, like > the existing ones: > commons-math-neuralnet > commons-math-transform > > On Wed, 19 Jul 2023 at 12:29, Elliotte Rusty Harold > wrote: > > > Could you be precise about what you mean by "modularisation"? It's a > > very overloaded term. Do you mean Java 9 modules as defined by the > > JPMS? > > > > On Wed, Jul 19, 2023 at 12:33 AM Dimitrios Efthymiou > > wrote: > > > > > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > > > modularisation of all 14 packages commons-math-legacy has? I think that > > > some of them are easy to modularise like optimisation, special and filter > > > > > > > > -- > > Elliotte Rusty Harold > > elh...@ibiblio.org > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > > For additional commands, e-mail: dev-h...@commons.apache.org > > > > -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
no. I mean creating maven modules inside commons-math, like the existing ones: commons-math-neuralnet commons-math-transform On Wed, 19 Jul 2023 at 12:29, Elliotte Rusty Harold wrote: > Could you be precise about what you mean by "modularisation"? It's a > very overloaded term. Do you mean Java 9 modules as defined by the > JPMS? > > On Wed, Jul 19, 2023 at 12:33 AM Dimitrios Efthymiou > wrote: > > > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > > modularisation of all 14 packages commons-math-legacy has? I think that > > some of them are easy to modularise like optimisation, special and filter > > > > -- > Elliotte Rusty Harold > elh...@ibiblio.org > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Could you be precise about what you mean by "modularisation"? It's a very overloaded term. Do you mean Java 9 modules as defined by the JPMS? On Wed, Jul 19, 2023 at 12:33 AM Dimitrios Efthymiou wrote: > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > modularisation of all 14 packages commons-math-legacy has? I think that > some of them are easy to modularise like optimisation, special and filter -- Elliotte Rusty Harold elh...@ibiblio.org - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
thanks Gilles. 1--I think I broke the build, because I did not include (correctly) the dependency on clustering inside the root pom.xml. My local build succeeds. I hope that the GitHub build succeeds, as well. 2--As for the atomic refactoring and feature branch, well, unless someone moves the Variance class (you said that someone is doing it now) and the distance package and whatever other dependencies exist within the ml.clustering package, there can be no moving of the remaining clustering classes to the new clustering module, right? 3--I see that the commons-statistics project, for example, has empty modules. I think the geometry project (I don't remember which one) has some classes that are still in commons-math, because the migration is not complete. So, I thought that I coud do the same i.e. move the clusteirng classes that do not depend on anything 4--I don't know how to continue with the clustering modularisation given all those dependencies. Maybe I shouldn't have started this, because now I am stuck On Wed, 19 Jul 2023 at 11:36, Gilles Sadowski wrote: > Hello. > > Le mer. 19 juil. 2023 à 11:21, Dimitrios Efthymiou > a écrit : > > > > I saw 1575, but I didn't see subtasks for all 14 packages. > > Is there a plan to modularise all 14 packages? > > Obviously, it would be good to achieve that, as pointed out > in the release notes of version 4.0-beta1: > https://commons.apache.org/proper/commons-math/changes-report.html > > But there is no "plan", like an ordered list of instructions to > follow from start to end. > The general task is "refactoring". > > > As for the dependencies on core, linear, analysis, well, > > from what I know, the way to modularise a codebase that > > was not designed to be modular, is to start moving classes > > that do not depend on legacy ones, 1-by-1, > > And break the build like it is currently the case with the > "clustering" refactoring? > > > slowly. > > As noted on JIRA[1], the move of an existing functionality into > its own (maven) module should be "atomic" on the "master" > branch. When the refactoring (started on a developer's local > machine) is sufficiently advanced, we can create a feature > branch so that several developers can more easily collaborate. > > > For classes that depend on legacy ones, then we can create > > new analysis and linear modules, create interfaces in them > > that have the methods our new modularised classes need, > > have the legacy classes in analysis and linear implement > > those interfaces, have the legacy module depend on the new > > analysis and linear modules (since they have the new interfaces), > > have the new optimisation module depend on the new > > analysis and linear modules and gradually you can move implementation > > code from the legacy to the new modules. The dependencies > > will be from legacy to the new modules and not the other way > > around. So the process I would try is: > > 1--create module commons-math-optimisation > > 2--create module commons-math-analysis > > 3--create module commons-math-linear-algebra > > 4--see on which analysis classes does optimisation depends? > > 5--see no which specific methods does optimisation depends? > > 6--create interfaces in commons-math-analysis for those classes and their > > methods that optimisation needs > > 7--have the analysis classes implement their respective interfaces from > > commons-math-analysis (maintaining the API) > > 8--have commons-math-legacy depend on commons-math-analysis and > > commons-math-linear-algebra > > 9--use these interfaces from within commons-math-optimisation > > 10-gradual move of methods and classes from commons-math-legacy to > > commons-math-optimisation, commons-math-analysis, > > commons-math-linear-algebra > > Sure! :-} > The devil is in the details... > > One crucial task is to have a way to (optionally) call external > implementations of linear algebra algorithms and data-structures. > I've no idea whether it's possible to adapt all the functionality to a > new design based only on interfaces (and not loose performance). > Unless we can really switch between alternative implementations > this is a lot of work with literally no gain. > Another possibility (also mentioned in [1]) is to isolate the needed > utilities in a "private" toolbox. [However, I'd be *very* reluctant if it > entails copying several hundred or thousand lines.] > > Regards, > Gilles > > [1] > https://issues.apache.org/jira/browse/MATH-1579?focusedCommentId=17744504=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17744504 > > > > > On Wed, 19 Jul 2023 at 09:49, Gilles Sadowski > wrote: > > > > > Hello. > > > > > > Le mer. 19 juil. 2023 à 02:33, Dimitrios Efthymiou > > > a écrit : > > > > > > > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > > > > modularisation of all 14 packages commons-math-legacy has? > > > > > > https://issues.apache.org/jira/browse/MATH-1575 > > > > > > > I think that > > > > some of
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Hello. Le mer. 19 juil. 2023 à 11:21, Dimitrios Efthymiou a écrit : > > I saw 1575, but I didn't see subtasks for all 14 packages. > Is there a plan to modularise all 14 packages? Obviously, it would be good to achieve that, as pointed out in the release notes of version 4.0-beta1: https://commons.apache.org/proper/commons-math/changes-report.html But there is no "plan", like an ordered list of instructions to follow from start to end. The general task is "refactoring". > As for the dependencies on core, linear, analysis, well, > from what I know, the way to modularise a codebase that > was not designed to be modular, is to start moving classes > that do not depend on legacy ones, 1-by-1, And break the build like it is currently the case with the "clustering" refactoring? > slowly. As noted on JIRA[1], the move of an existing functionality into its own (maven) module should be "atomic" on the "master" branch. When the refactoring (started on a developer's local machine) is sufficiently advanced, we can create a feature branch so that several developers can more easily collaborate. > For classes that depend on legacy ones, then we can create > new analysis and linear modules, create interfaces in them > that have the methods our new modularised classes need, > have the legacy classes in analysis and linear implement > those interfaces, have the legacy module depend on the new > analysis and linear modules (since they have the new interfaces), > have the new optimisation module depend on the new > analysis and linear modules and gradually you can move implementation > code from the legacy to the new modules. The dependencies > will be from legacy to the new modules and not the other way > around. So the process I would try is: > 1--create module commons-math-optimisation > 2--create module commons-math-analysis > 3--create module commons-math-linear-algebra > 4--see on which analysis classes does optimisation depends? > 5--see no which specific methods does optimisation depends? > 6--create interfaces in commons-math-analysis for those classes and their > methods that optimisation needs > 7--have the analysis classes implement their respective interfaces from > commons-math-analysis (maintaining the API) > 8--have commons-math-legacy depend on commons-math-analysis and > commons-math-linear-algebra > 9--use these interfaces from within commons-math-optimisation > 10-gradual move of methods and classes from commons-math-legacy to > commons-math-optimisation, commons-math-analysis, > commons-math-linear-algebra Sure! :-} The devil is in the details... One crucial task is to have a way to (optionally) call external implementations of linear algebra algorithms and data-structures. I've no idea whether it's possible to adapt all the functionality to a new design based only on interfaces (and not loose performance). Unless we can really switch between alternative implementations this is a lot of work with literally no gain. Another possibility (also mentioned in [1]) is to isolate the needed utilities in a "private" toolbox. [However, I'd be *very* reluctant if it entails copying several hundred or thousand lines.] Regards, Gilles [1] https://issues.apache.org/jira/browse/MATH-1579?focusedCommentId=17744504=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17744504 > > On Wed, 19 Jul 2023 at 09:49, Gilles Sadowski wrote: > > > Hello. > > > > Le mer. 19 juil. 2023 à 02:33, Dimitrios Efthymiou > > a écrit : > > > > > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > > > modularisation of all 14 packages commons-math-legacy has? > > > > https://issues.apache.org/jira/browse/MATH-1575 > > > > > I think that > > > some of them are easy to modularise like optimisation, > > > > When I list the dependencies towards other packages in "legacy", > > I see > > o.a.c.math4.legacy.core > > o.a.c.math4.legacy.linear > > o.a.c.math4.legacy.analysis > > > > How do you suggest to handle it? > > > > > special > > > > Here, there is only one class, but it should be analysed to > > suggest a better API (and maybe improve performance). > > There is also the question of whether to provide this and other > > special functions with extended precision[1] (and maybe move > > them to [Numbers]; like the gamma family of functions). > > > > > and filter > > > > When I list the dependencies towards other packages in "legacy", > > I see > > o.a.c.math4.legacy.linear > > > > Regards, > > Gilles > > > > > > [1] See section 7.4 in D. Bailey's documentation: > > https://www.davidhbailey.com/dhbpapers/mpfun2020.pdf - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
I saw 1575, but I didn't see subtasks for all 14 packages. Is there a plan to modularise all 14 packages? As for the dependencies on core, linear, analysis, well, from what I know, the way to modularise a codebase that was not designed to be modular, is to start moving classes that do not depend on legacy ones, 1-by-1, slowly. For classes that depend on legacy ones, then we can create new analysis and linear modules, create interfaces in them that have the methods our new modularised classes need, have the legacy classes in analysis and linear implement those interfaces, have the legacy module depend on the new analysis and linear modules (since they have the new interfaces), have the new optimisation module depend on the new analysis and linear modules and gradually you can move implementation code from the legacy to the new modules. The dependencies will be from legacy to the new modules and not the other way around. So the process I would try is: 1--create module commons-math-optimisation 2--create module commons-math-analysis 3--create module commons-math-linear-algebra 4--see on which analysis classes does optimisation depends? 5--see no which specific methods does optimisation depends? 6--create interfaces in commons-math-analysis for those classes and their methods that optimisation needs 7--have the analysis classes implement their respective interfaces from commons-math-analysis (maintaining the API) 8--have commons-math-legacy depend on commons-math-analysis and commons-math-linear-algebra 9--use these interfaces from within commons-math-optimisation 10-gradual move of methods and classes from commons-math-legacy to commons-math-optimisation, commons-math-analysis, commons-math-linear-algebra On Wed, 19 Jul 2023 at 09:49, Gilles Sadowski wrote: > Hello. > > Le mer. 19 juil. 2023 à 02:33, Dimitrios Efthymiou > a écrit : > > > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > > modularisation of all 14 packages commons-math-legacy has? > > https://issues.apache.org/jira/browse/MATH-1575 > > > I think that > > some of them are easy to modularise like optimisation, > > When I list the dependencies towards other packages in "legacy", > I see > o.a.c.math4.legacy.core > o.a.c.math4.legacy.linear > o.a.c.math4.legacy.analysis > > How do you suggest to handle it? > > > special > > Here, there is only one class, but it should be analysed to > suggest a better API (and maybe improve performance). > There is also the question of whether to provide this and other > special functions with extended precision[1] (and maybe move > them to [Numbers]; like the gamma family of functions). > > > and filter > > When I list the dependencies towards other packages in "legacy", > I see > o.a.c.math4.legacy.linear > > Regards, > Gilles > > > [1] See section 7.4 in D. Bailey's documentation: > https://www.davidhbailey.com/dhbpapers/mpfun2020.pdf > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
Re: [math] JIRA ticket for modularisation of all 14 legacy packages
Hello. Le mer. 19 juil. 2023 à 02:33, Dimitrios Efthymiou a écrit : > > Hello everyone. Is there, or gonna be, a dedicated ticket for the > modularisation of all 14 packages commons-math-legacy has? https://issues.apache.org/jira/browse/MATH-1575 > I think that > some of them are easy to modularise like optimisation, When I list the dependencies towards other packages in "legacy", I see o.a.c.math4.legacy.core o.a.c.math4.legacy.linear o.a.c.math4.legacy.analysis How do you suggest to handle it? > special Here, there is only one class, but it should be analysed to suggest a better API (and maybe improve performance). There is also the question of whether to provide this and other special functions with extended precision[1] (and maybe move them to [Numbers]; like the gamma family of functions). > and filter When I list the dependencies towards other packages in "legacy", I see o.a.c.math4.legacy.linear Regards, Gilles [1] See section 7.4 in D. Bailey's documentation: https://www.davidhbailey.com/dhbpapers/mpfun2020.pdf - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
[math] JIRA ticket for modularisation of all 14 legacy packages
Hello everyone. Is there, or gonna be, a dedicated ticket for the modularisation of all 14 packages commons-math-legacy has? I think that some of them are easy to modularise like optimisation, special and filter
