Re: Build failed in Jenkins: commons-validator #61

[sebb]

I think this has happened before.

Jenkins now needs Java 8.
It attempts recovery but that does not work.

On 13 March 2018 at 16:17, Gary Gregory  wrote:
> Thoughts on how to fix this?
>
> Gary
>
> -- Forwarded message -
> From: Apache Jenkins Server 
> Date: Tue, Mar 13, 2018, 09:48
> Subject: Build failed in Jenkins: commons-validator #61
> To: 
>
>
> See 
>
> --
> Started by user ggregory
> [EnvInject] - Loading node environment variables.
> Building remotely on windows-2012-1 (Windows) in workspace <
> https://builds.apache.org/job/commons-validator/ws/>
> Updating https://svn.apache.org/repos/asf/commons/proper/validator/trunk at
> revision '2018-03-13T15:47:38.139 +'
> At revision 1826655
>
> No changes for
> https://svn.apache.org/repos/asf/commons/proper/validator/trunk since the
> previous build
> [EnvInject] - Executing scripts and injecting environment variables after
> the SCM step.
> [EnvInject] - Injecting as environment variables the properties content
> JAVA_1_6_HOME=/home/jenkins/tools/java/latest1.6/
>
> [EnvInject] - Variables injected successfully.
> Parsing POMs
> Established TCP socket on 62574
> maven35-agent.jar already up to date
> maven35-interceptor.jar already up to date
> maven3-interceptor-commons.jar already up to date
> [commons-validator] $ f:\\jenkins\\tools\\java\\latest1.7/bin/java -Xmx2g
> -Xms256m -cp
> f:\jenkins\jenkins-slave\maven35-agent.jar;f:\jenkins\tools\maven\latest3\boot\plexus-classworlds-2.5.2.jar;f:\\jenkins\\tools\\maven\\latest3/conf/logging
> jenkins.maven3.agent.Maven35Main f:\\jenkins\\tools\\maven\\latest3
> F:\jenkins\jenkins-slave\slave.jar
> f:\jenkins\jenkins-slave\maven35-interceptor.jar
> f:\jenkins\jenkins-slave\maven3-interceptor-commons.jar 62574
> <===[JENKINS REMOTING CAPACITY]===>   channel started
> ERROR:
> 
> ERROR: Invalid project setup: jenkins/security/MasterToSlaveCallable :
> Unsupported major.minor version 52.0
> ERROR: [JENKINS-18403][JENKINS-28294] JDK 'JDK 1.7 (latest)' not supported
> to run Maven projects.
> ERROR: Maven projects have to be launched with a Java version greater or
> equal to the minimum version required by the master.
> ERROR: Use the Maven JDK Toolchains (plugin) to build your maven project
> with an older JDK.
> ERROR: Retrying with slave Java and setting compile/test properties to
> point to f:\\jenkins\\tools\\java\\latest1.7.
> ERROR:
> 
> Established TCP socket on 62579
> maven35-agent.jar already up to date
> maven35-interceptor.jar already up to date
> maven3-interceptor-commons.jar already up to date
> [commons-validator] $ "C:\Program Files\Java\jre1.8.0_152/bin/java" -Xmx2g
> -Xms256m -cp
> f:\jenkins\jenkins-slave\maven35-agent.jar;f:\jenkins\tools\maven\latest3\boot\plexus-classworlds-2.5.2.jar;f:\\jenkins\\tools\\maven\\latest3/conf/logging
> jenkins.maven3.agent.Maven35Main f:\\jenkins\\tools\\maven\\latest3
> F:\jenkins\jenkins-slave\slave.jar
> f:\jenkins\jenkins-slave\maven35-interceptor.jar
> f:\jenkins\jenkins-slave\maven3-interceptor-commons.jar 62579
> <===[JENKINS REMOTING CAPACITY]===>   channel started
> Executing Maven:  -B -f <
> https://builds.apache.org/job/commons-validator/ws/pom.xml>
> -Dmaven.repo.local=f:\jenkins\jenkins-slave\maven-repositories\0 -V clean
> install --batch-mode -Dgpg.skip -Prelease -Pjava-1.6
>  [1mApache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
> 2017-10-18T07:58:13Z) [m
> Maven home: F:\jenkins\tools\maven\latest3
> Java version: 1.8.0_152, vendor: Oracle Corporation
> Java home: C:\Program Files\Java\jre1.8.0_152
> Default locale: en_US, platform encoding: Cp1252
> OS name: "windows server 2012 r2", version: "6.3", arch: "amd64", family:
> "windows"
> [INFO] Scanning for projects...
> [INFO]
> [INFO]
> 
> [INFO] Building Apache Commons Validator 1.7-SNAPSHOT
> [INFO]
> 
> [INFO]
> [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @
> commons-validator ---
> [INFO] Deleting 
> [INFO]
> [INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-maven-3) @
> commons-validator ---
> [INFO]
> [INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @
> commons-validator ---
> [INFO]
> [INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @
> commons-validator ---
> [INFO] Executing tasks
>
> main:
>  [copy] Copying 2 files to <
> https://builds.apache.org/job/commons-validator/ws/target\apidocs\META-INF>
> [INFO] Executed tasks
> [INFO]
> [INFO] --- 

Re: [LAZY][VOTE] Release Commons Parent 45 based on RC1.

[sebb]

On 13 March 2018 at 22:59, Gary Gregory  wrote:
> I am looking for a sanity check from other community members... just in
> case ;-)

One reason we can use a lazy vote for CP is that it is completely optional.
If it turns out to be broken, we just stop using it.

This is akin to making a change to trunk, find it causes a problem so
it is changed again or reverted.

If a version of CP is faulty it won't find its way into a formal release.

> Gary
>
> On Mon, Mar 12, 2018 at 6:49 AM, Rob Tompkins  wrote:
>
>> Hello all,
>>
>> This is a [LAZY][VOTE] for releasing Apache Commons Parent 45 (from RC1).
>> [Pardon my miss in commons-parent-44-RC3, I am truly sorry for that].
>>
>> Note on validation:
>>Please read the https://dist.apache.org/repos/dist/dev/commons/commons-
>> parent/RELEASE-NOTES.txt, as
>>there is an erroneous non-failing stack trace that occurs during the
>> build.
>>
>> Tag name:
>>commons-parent-45-RC1
>>
>> Tag URL:
>>https://svn.apache.org/repos/asf/commons/proper/commons-
>> parent/tags/commons-parent-45-RC1/
>>
>> Commit ID the tag points at:
>> 1826542
>>
>> Site Zip:
>>https://dist.apache.org/repos/dist/dev/commons/commons-parent/site.zip
>>
>> Distribution files (committed at revision 25660):
>>https://dist.apache.org/repos/dist/dev/commons/commons-parent/
>>
>> Distribution files hashes (SHA1):
>>commons-parent-45-src.tar.gz
>>(SHA1: a72ab38c3f1b654a753a477071e6d3f9bfc73b50)
>>commons-parent-45-src.zip
>>(SHA1: 343dc5bf8e7d47528bd4b115b6074c7924599f23)
>>
>> These are the Maven artifacts and their hashes:
>>commons-parent-45-site.xml
>>(SHA1: 02b3b54d26d97a72fd55b20d027040ca0daf52b7)
>>commons-parent-45.pom
>>(SHA1: 71201c6eea226cd986f03adb1b6ec326dd860786)
>>
>> KEYS file to check signatures:
>>http://www.apache.org/dist/commons/KEYS
>>
>> Maven artifacts:
>>https://repository.apache.org/content/repositories/
>> orgapachecommons-1315
>>
>> Please select one of the following options[1]:
>>   [ ] +1 Release it.
>>   [ ] +0 Go ahead; I don't care.
>>   [ ] -0 There are a few minor glitches: ...
>>   [ ] -1 No, do not release it because ...
>>
>> This vote will be open at least 72 hours, i.e. until
>> 2018-03-15T13:00:00Z
>> (this is UTC time).
>> 
>>
>> Cheers,
>> -Rob
>>
>> [1] http://apache.org/foundation/voting.html
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>>
>>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [LAZY][VOTE] Release Commons Parent 45 based on RC1.

[Gary Gregory]

I am looking for a sanity check from other community members... just in
case ;-)

Gary

On Mon, Mar 12, 2018 at 6:49 AM, Rob Tompkins  wrote:

> Hello all,
>
> This is a [LAZY][VOTE] for releasing Apache Commons Parent 45 (from RC1).
> [Pardon my miss in commons-parent-44-RC3, I am truly sorry for that].
>
> Note on validation:
>Please read the https://dist.apache.org/repos/dist/dev/commons/commons-
> parent/RELEASE-NOTES.txt, as
>there is an erroneous non-failing stack trace that occurs during the
> build.
>
> Tag name:
>commons-parent-45-RC1
>
> Tag URL:
>https://svn.apache.org/repos/asf/commons/proper/commons-
> parent/tags/commons-parent-45-RC1/
>
> Commit ID the tag points at:
> 1826542
>
> Site Zip:
>https://dist.apache.org/repos/dist/dev/commons/commons-parent/site.zip
>
> Distribution files (committed at revision 25660):
>https://dist.apache.org/repos/dist/dev/commons/commons-parent/
>
> Distribution files hashes (SHA1):
>commons-parent-45-src.tar.gz
>(SHA1: a72ab38c3f1b654a753a477071e6d3f9bfc73b50)
>commons-parent-45-src.zip
>(SHA1: 343dc5bf8e7d47528bd4b115b6074c7924599f23)
>
> These are the Maven artifacts and their hashes:
>commons-parent-45-site.xml
>(SHA1: 02b3b54d26d97a72fd55b20d027040ca0daf52b7)
>commons-parent-45.pom
>(SHA1: 71201c6eea226cd986f03adb1b6ec326dd860786)
>
> KEYS file to check signatures:
>http://www.apache.org/dist/commons/KEYS
>
> Maven artifacts:
>https://repository.apache.org/content/repositories/
> orgapachecommons-1315
>
> Please select one of the following options[1]:
>   [ ] +1 Release it.
>   [ ] +0 Go ahead; I don't care.
>   [ ] -0 There are a few minor glitches: ...
>   [ ] -1 No, do not release it because ...
>
> This vote will be open at least 72 hours, i.e. until
> 2018-03-15T13:00:00Z
> (this is UTC time).
> 
>
> Cheers,
> -Rob
>
> [1] http://apache.org/foundation/voting.html
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>

Re: [parent][release-plugin] One to release the other

[Rob Tompkins]

Yes. I did. Although, it plays mor strangely with the parent people project 
than others because I don’t want to have some properties declared by default. 
The idea there is to have the up version into 45 be non-breaking if that is the 
only change that is made. 

> On Mar 13, 2018, at 3:47 PM, Gary Gregory  wrote:
> 
> Hi Rob,
> 
> When you RC and release the parent POM, do you use your new Maven release
> plugin?
> 
> Gary

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[parent][release-plugin] One to release the other

[Gary Gregory]

Hi Rob,

When you RC and release the parent POM, do you use your new Maven release
plugin?

Gary

Re: [Statistics] Port codes from Commons Math

[Gimhana Nadeeshan]

Hello Devs,

Thanks Gilles and Eric for guidance.

I have cloned the Commons repos and forked the Common's Stat repo. Is it
possible to make pull requests to that repo to be reviewed? Or should I
follow a specific method?

By referring the API docs I got some idea of the separation of modules.

In the current Commons's stat repo there are some classes under the
package  distribution. I think those can be refactored using java 8 in
build statistics functionalities. Please correct me if I wrong.

As Eric said separation of function and streaming implementations is good
idea as designing. (In my point of view, it means method overloading ->
Again correct me if I didn't understand your fact correctly)

And I will share my draft proposal here for your review soon.

Best Regards.

On 13 March 2018 at 20:50, Gilles  wrote:

> Hello.
>
> On Tue, 13 Mar 2018 09:25:19 +0100, Eric Barnhill wrote:
>
>> On Tue, Mar 13, 2018 at 12:47 AM, Gilles 
>> wrote:
>>
>>
>>>
 Where can we find the old code before port into new Commons components?


>>> The code bases are managed by the "git" software; the whole history is
>>> available:
>>>   https://git1-us-west.apache.org/repos/asf?p=commons-math.git;a=log
>>>
>>> [I'd advise to "clone" the repositories on your local computer, and
>>> use the command line tools.]
>>>
>>
>>
>> I believe you will want to clone the commons-math repositories, but then
>> develop your own "fork" of the commons-statistics repository. Gilles can
>> correct me if that is wrong.
>>
>
> Actually, I know only my workflow:
>  $ git clone ...
>  $ git branch ...
>  $ git commit ...
>  $ git push
>
> :-}
>
> I didn't find it very easy to cooperate with developers who
> fork on GitHub and submit PRs.
> I've now found the "git" command that creates a branch from
> a PR, but it would be so much more comfortable to just switch
> directory and do "git pull".
>
> In the context of GSoC, would it be possible to grant some
> privilege to non-committers so that they can update a selected
> "git" repository?
> If not, what is the next easiest way to share a "common space"
> (aka "sandbox") from which it would be easy to copy reviewed
> bits over to the official source repository?
>
>
>>> As
>>>
 you mentioned it will be a good approach to redesign process.


>>> You don't necessarily need to analyze how the code was before
>>> the port/refactoring; looking at how it is now is sufficient,
>>> unless you suspect that something is wrong now and might have
>>> been better before. ;-)
>>>
>>>
>> In particular, the statistics library was designed before Java 8. Java 8
>> however has provided both efficient programming strategies for these
>> statistical methods (in the form of lambdas and streams) as well as some
>> built-in methods providing summary statistics functions (see discussion at
>> http://markmail.org/message/7t2mjaprsuvb3waj).
>>
>
> Very good point, indeed.
> IMO, the new component should be targeted Java 8.
> Even Java 9 (enforcing modularity with JPMS): if by the time we think
> of releasing the code, we still want to avoid "multi-release" JARs it
> will be easy to just remove the "module-info" files (I don't think much
> else Java 9 specific would used by "Commons Statistics").
>
> In fact, given the very slow pace at which new components are being
> brought to releasable state, I'd like to ask whether it would be OK
> to make "incremental" releases?  That would mean: focus on (maven)
> modules that seem close to feature-complete and bug-free, fix the
> remaining issues and perform a release with that module added.
>
> It seems that the expectations were set to high (content-wise given
> the amount of human resources), so that neither CM can be released
> (too many non-fixed issues) nor its "Commons Numbers" spin-off that
> contains many modules, some of which are blocked by lack of consensus
> or dangling discussions.
>
> It probably makes sense, as a design strategy, to separate the function
>> implementation from the streaming implementation. For example, a 2D
>> integer
>> array will probably require a different streaming implementation than a 1D
>> double array, but they can  probably both be passed the same function
>> handle to collect, say, the mean or max value.
>>
>> The role of commons might then be to provide a convenient interface, so
>> that the user can simply call a static method like SummaryStats.mean() and
>> not have to worry about the implementation.
>>
>> The other difficulty I see, is that quantile and median statistics will
>> not
>> be as easy to stream as statistics with a closed-form solution like mean
>> or
>> variance. There may however be great algorithms out there for pulling the
>> median or the 95% quantile out of a stream -- if so they should be used.
>>
>> Eric
>>
>
> Eric,
>
> Would you be the official "mentor" for the GSoC participants that
> are interested in helping with the porting of 

Fwd: Build failed in Jenkins: commons-validator #61

[Gary Gregory]

Thoughts on how to fix this?

Gary

-- Forwarded message -
From: Apache Jenkins Server 
Date: Tue, Mar 13, 2018, 09:48
Subject: Build failed in Jenkins: commons-validator #61
To: 


See 

--
Started by user ggregory
[EnvInject] - Loading node environment variables.
Building remotely on windows-2012-1 (Windows) in workspace <
https://builds.apache.org/job/commons-validator/ws/>
Updating https://svn.apache.org/repos/asf/commons/proper/validator/trunk at
revision '2018-03-13T15:47:38.139 +'
At revision 1826655

No changes for
https://svn.apache.org/repos/asf/commons/proper/validator/trunk since the
previous build
[EnvInject] - Executing scripts and injecting environment variables after
the SCM step.
[EnvInject] - Injecting as environment variables the properties content
JAVA_1_6_HOME=/home/jenkins/tools/java/latest1.6/

[EnvInject] - Variables injected successfully.
Parsing POMs
Established TCP socket on 62574
maven35-agent.jar already up to date
maven35-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
[commons-validator] $ f:\\jenkins\\tools\\java\\latest1.7/bin/java -Xmx2g
-Xms256m -cp
f:\jenkins\jenkins-slave\maven35-agent.jar;f:\jenkins\tools\maven\latest3\boot\plexus-classworlds-2.5.2.jar;f:\\jenkins\\tools\\maven\\latest3/conf/logging
jenkins.maven3.agent.Maven35Main f:\\jenkins\\tools\\maven\\latest3
F:\jenkins\jenkins-slave\slave.jar
f:\jenkins\jenkins-slave\maven35-interceptor.jar
f:\jenkins\jenkins-slave\maven3-interceptor-commons.jar 62574
<===[JENKINS REMOTING CAPACITY]===>   channel started
ERROR:

ERROR: Invalid project setup: jenkins/security/MasterToSlaveCallable :
Unsupported major.minor version 52.0
ERROR: [JENKINS-18403][JENKINS-28294] JDK 'JDK 1.7 (latest)' not supported
to run Maven projects.
ERROR: Maven projects have to be launched with a Java version greater or
equal to the minimum version required by the master.
ERROR: Use the Maven JDK Toolchains (plugin) to build your maven project
with an older JDK.
ERROR: Retrying with slave Java and setting compile/test properties to
point to f:\\jenkins\\tools\\java\\latest1.7.
ERROR:

Established TCP socket on 62579
maven35-agent.jar already up to date
maven35-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
[commons-validator] $ "C:\Program Files\Java\jre1.8.0_152/bin/java" -Xmx2g
-Xms256m -cp
f:\jenkins\jenkins-slave\maven35-agent.jar;f:\jenkins\tools\maven\latest3\boot\plexus-classworlds-2.5.2.jar;f:\\jenkins\\tools\\maven\\latest3/conf/logging
jenkins.maven3.agent.Maven35Main f:\\jenkins\\tools\\maven\\latest3
F:\jenkins\jenkins-slave\slave.jar
f:\jenkins\jenkins-slave\maven35-interceptor.jar
f:\jenkins\jenkins-slave\maven3-interceptor-commons.jar 62579
<===[JENKINS REMOTING CAPACITY]===>   channel started
Executing Maven:  -B -f <
https://builds.apache.org/job/commons-validator/ws/pom.xml>
-Dmaven.repo.local=f:\jenkins\jenkins-slave\maven-repositories\0 -V clean
install --batch-mode -Dgpg.skip -Prelease -Pjava-1.6
 [1mApache Maven 3.5.2 (138edd61fd100ec658bfa2d307c43b76940a5d7d;
2017-10-18T07:58:13Z) [m
Maven home: F:\jenkins\tools\maven\latest3
Java version: 1.8.0_152, vendor: Oracle Corporation
Java home: C:\Program Files\Java\jre1.8.0_152
Default locale: en_US, platform encoding: Cp1252
OS name: "windows server 2012 r2", version: "6.3", arch: "amd64", family:
"windows"
[INFO] Scanning for projects...
[INFO]
[INFO]

[INFO] Building Apache Commons Validator 1.7-SNAPSHOT
[INFO]

[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @
commons-validator ---
[INFO] Deleting 
[INFO]
[INFO] --- maven-enforcer-plugin:1.4.1:enforce (enforce-maven-3) @
commons-validator ---
[INFO]
[INFO] --- build-helper-maven-plugin:3.0.0:parse-version (parse-version) @
commons-validator ---
[INFO]
[INFO] --- maven-antrun-plugin:1.8:run (javadoc.resources) @
commons-validator ---
[INFO] Executing tasks

main:
 [copy] Copying 2 files to <
https://builds.apache.org/job/commons-validator/ws/target\apidocs\META-INF>
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process
(process-resource-bundles) @ commons-validator ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (default) @
commons-validator ---
[INFO] Executing: cmd.exe /X /C "svn --non-interactive info"
[INFO] Working directory: <
https://builds.apache.org/job/commons-validator/ws/>
[INFO] Storing buildNumber: 1826655 at timestamp: 1520956084090

Github usage Was: [Statistics] Port codes from Commons Math

[ajs6f]

> On Mar 13, 2018, at 11:20 AM, Gilles  wrote:
> 
> 
> I didn't find it very easy to cooperate with developers who fork on GitHub 
> and submit PRs. I've now found the "git" command that creates a branch from a 
> PR, but it would be so much more comfortable to just switch directory and do 
> "git pull".
> 

Just as a point of information, it is possible to reverse the Github <- Apache 
mirroring most projects use to be Github -> Apache. What that means is that 
merging PRs from Github becomes one click in the Github UI. 

There are other consequences, of course, especially related to other 
integrations Commons may be using (e.g. integration between Github and JIRA).

Of course, INFRA are the folks to talk to if this sounds interesting. At Apache 
Jena, we looked into it but have taken no action because we still have some 
open questions about when some of our workflow integrations will become 
possible with "reversed mirroring".

Adam Soroka ; aj...@apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Statistics] Port codes from Commons Math

[Gilles]

Hello.

On Tue, 13 Mar 2018 09:25:19 +0100, Eric Barnhill wrote:
On Tue, Mar 13, 2018 at 12:47 AM, Gilles 


wrote:





Where can we find the old code before port into new Commons 
components?




The code bases are managed by the "git" software; the whole history 
is

available:
  https://git1-us-west.apache.org/repos/asf?p=commons-math.git;a=log

[I'd advise to "clone" the repositories on your local computer, and
use the command line tools.]



I believe you will want to clone the commons-math repositories, but 
then
develop your own "fork" of the commons-statistics repository. Gilles 
can

correct me if that is wrong.


Actually, I know only my workflow:
 $ git clone ...
 $ git branch ...
 $ git commit ...
 $ git push

:-}

I didn't find it very easy to cooperate with developers who
fork on GitHub and submit PRs.
I've now found the "git" command that creates a branch from
a PR, but it would be so much more comfortable to just switch
directory and do "git pull".

In the context of GSoC, would it be possible to grant some
privilege to non-committers so that they can update a selected
"git" repository?
If not, what is the next easiest way to share a "common space"
(aka "sandbox") from which it would be easy to copy reviewed
bits over to the official source repository?



As

you mentioned it will be a good approach to redesign process.



You don't necessarily need to analyze how the code was before
the port/refactoring; looking at how it is now is sufficient,
unless you suspect that something is wrong now and might have
been better before. ;-)



In particular, the statistics library was designed before Java 8. 
Java 8

however has provided both efficient programming strategies for these
statistical methods (in the form of lambdas and streams) as well as 
some
built-in methods providing summary statistics functions (see 
discussion at

http://markmail.org/message/7t2mjaprsuvb3waj).


Very good point, indeed.
IMO, the new component should be targeted Java 8.
Even Java 9 (enforcing modularity with JPMS): if by the time we think
of releasing the code, we still want to avoid "multi-release" JARs it
will be easy to just remove the "module-info" files (I don't think much
else Java 9 specific would used by "Commons Statistics").

In fact, given the very slow pace at which new components are being
brought to releasable state, I'd like to ask whether it would be OK
to make "incremental" releases?  That would mean: focus on (maven)
modules that seem close to feature-complete and bug-free, fix the
remaining issues and perform a release with that module added.

It seems that the expectations were set to high (content-wise given
the amount of human resources), so that neither CM can be released
(too many non-fixed issues) nor its "Commons Numbers" spin-off that
contains many modules, some of which are blocked by lack of consensus
or dangling discussions.

It probably makes sense, as a design strategy, to separate the 
function
implementation from the streaming implementation. For example, a 2D 
integer
array will probably require a different streaming implementation than 
a 1D

double array, but they can  probably both be passed the same function
handle to collect, say, the mean or max value.

The role of commons might then be to provide a convenient interface, 
so
that the user can simply call a static method like 
SummaryStats.mean() and

not have to worry about the implementation.

The other difficulty I see, is that quantile and median statistics 
will not
be as easy to stream as statistics with a closed-form solution like 
mean or
variance. There may however be great algorithms out there for pulling 
the
median or the 95% quantile out of a stream -- if so they should be 
used.


Eric


Eric,

Would you be the official "mentor" for the GSoC participants that
are interested in helping with the porting of "o.a.c.math4.stat"?

Thank you,
Gilles


-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [Statistics]Port codes from Commons Math

[Eric Barnhill]

On Tue, Mar 13, 2018 at 12:47 AM, Gilles 
wrote:

>
>>
>> Where can we find the old code before port into new Commons components?
>>
>
> The code bases are managed by the "git" software; the whole history is
> available:
>   https://git1-us-west.apache.org/repos/asf?p=commons-math.git;a=log
>
> [I'd advise to "clone" the repositories on your local computer, and
> use the command line tools.]


I believe you will want to clone the commons-math repositories, but then
develop your own "fork" of the commons-statistics repository. Gilles can
correct me if that is wrong.


>
>
> As
>> you mentioned it will be a good approach to redesign process.
>>
>
> You don't necessarily need to analyze how the code was before
> the port/refactoring; looking at how it is now is sufficient,
> unless you suspect that something is wrong now and might have
> been better before. ;-)
>

In particular, the statistics library was designed before Java 8. Java 8
however has provided both efficient programming strategies for these
statistical methods (in the form of lambdas and streams) as well as some
built-in methods providing summary statistics functions (see discussion at
http://markmail.org/message/7t2mjaprsuvb3waj).

It probably makes sense, as a design strategy, to separate the function
implementation from the streaming implementation. For example, a 2D integer
array will probably require a different streaming implementation than a 1D
double array, but they can  probably both be passed the same function
handle to collect, say, the mean or max value.

The role of commons might then be to provide a convenient interface, so
that the user can simply call a static method like SummaryStats.mean() and
not have to worry about the implementation.

The other difficulty I see, is that quantile and median statistics will not
be as easy to stream as statistics with a closed-form solution like mean or
variance. There may however be great algorithms out there for pulling the
median or the 95% quantile out of a stream -- if so they should be used.

Eric